site_treelogo
  -   Terms of Use and Privacy
Apps | GNU/Linux | IT Prepper | Meta
rss
site_treelogo
  -   Terms of Use and Privacy
Apps | GNU/Linux | IT Prepper | Meta
rss

<<   <   >   >>

2021-06-04 | Apps | Basic Graphite Web Install

These instructions work on L1G3R reference OS with N🚫NIC additions.

Install graphite-web:

divine [ ~ ]$ sudo pip3 install graphite-web
[sudo] password for divine: 
Collecting graphite-web
  Downloading graphite-web-1.1.8.tar.gz (1.2 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1.2 MB 6.4 MB/s 
Collecting Django<3.1,>=1.8
  Downloading Django-3.0.14-py3-none-any.whl (7.5 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 7.5 MB 6.1 MB/s 
Collecting django-tagging==0.4.3
  Downloading django_tagging-0.4.3-py2.py3-none-any.whl (34 kB)
Requirement already satisfied: pytz in /usr/lib/python3.7/site-packages (from graphite-web) (2021.1)
Requirement already satisfied: pyparsing in /usr/lib/python3.7/site-packages/pyparsing-3.0.0b1-py3.7.egg (from graphite-web) (3.0.0b1)
Collecting cairocffi
  Downloading cairocffi-1.2.0.tar.gz (70 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 70 kB 9.1 MB/s 
Requirement already satisfied: urllib3 in /usr/lib/python3.7/site-packages (from graphite-web) (1.26.3)
Requirement already satisfied: scandir in /usr/lib/python3.7/site-packages (from graphite-web) (1.10.0)
Requirement already satisfied: six in /usr/lib/python3.7/site-packages (from graphite-web) (1.15.0)
Collecting asgiref~=3.2
  Downloading asgiref-3.3.4-py3-none-any.whl (22 kB)
Collecting sqlparse>=0.2.2
  Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 42 kB 2.1 MB/s 
Collecting typing-extensions
  Downloading typing_extensions-3.10.0.0-py3-none-any.whl (26 kB)
Requirement already satisfied: cffi>=1.1.0 in /usr/lib/python3.7/site-packages (from cairocffi->graphite-web) (1.14.5)
Requirement already satisfied: pycparser in /usr/lib/python3.7/site-packages (from cffi>=1.1.0->cairocffi->graphite-web) (2.20)
Using legacy 'setup.py install' for graphite-web, since package 'wheel' is not installed.
Using legacy 'setup.py install' for cairocffi, since package 'wheel' is not installed.
Installing collected packages: typing-extensions, sqlparse, asgiref, django-tagging, Django, cairocffi, graphite-web
    Running setup.py install for cairocffi ... done
    Running setup.py install for graphite-web ... done
Successfully installed Django-3.0.14 asgiref-3.3.4 cairocffi-1.2.0 django-tagging-0.4.3 graphite-web sqlparse-0.4.1 typing-extensions-3.10.0.0
WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
You should consider upgrading via the '/usr/bin/python3.7 -m pip install --upgrade pip' command.
divine [ ~ ]$ sudo pip3 install carbon
Collecting carbon
  Downloading carbon-1.1.8.tar.gz (72 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 72 kB 1.9 MB/s 
Collecting Twisted
  Downloading Twisted-21.2.0-py3-none-any.whl (3.1 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3.1 MB 6.7 MB/s 
Collecting txAMQP
  Downloading txAMQP-0.8.2.tar.gz (39 kB)
Collecting cachetools
  Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Requirement already satisfied: urllib3 in /usr/lib/python3.7/site-packages (from carbon) (1.26.3)
Collecting incremental>=16.10.1
  Downloading incremental-21.3.0-py2.py3-none-any.whl (15 kB)
Collecting constantly>=15.1
  Downloading constantly-15.1.0-py2.py3-none-any.whl (7.9 kB)
Collecting attrs>=19.2.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 53 kB 5.7 MB/s 
Collecting zope.interface>=4.4.2
  Downloading zope.interface-5.4.0-cp37-cp37m-manylinux2010_x86_64.whl (251 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 251 kB 13.2 MB/s 
Collecting Automat>=0.8.0
  Downloading Automat-20.2.0-py2.py3-none-any.whl (31 kB)
Collecting hyperlink>=17.1.1
  Downloading hyperlink-21.0.0-py2.py3-none-any.whl (74 kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 74 kB 5.1 MB/s 
Requirement already satisfied: six in /usr/lib/python3.7/site-packages (from Automat>=0.8.0->Twisted->carbon) (1.15.0)
Requirement already satisfied: idna>=2.5 in /usr/lib/python3.7/site-packages (from hyperlink>=17.1.1->Twisted->carbon) (2.10)
Requirement already satisfied: setuptools in /usr/lib/python3.7/site-packages (from zope.interface>=4.4.2->Twisted->carbon) (40.8.0)
Using legacy 'setup.py install' for carbon, since package 'wheel' is not installed.
Using legacy 'setup.py install' for txAMQP, since package 'wheel' is not installed.
Installing collected packages: attrs, zope.interface, incremental, hyperlink, constantly, Automat, Twisted, txAMQP, cachetools, carbon
    Running setup.py install for txAMQP ... done
    Running setup.py install for carbon ... done
Successfully installed Automat-20.2.0 Twisted-21.2.0 attrs-21.2.0 cachetools-4.2.2 carbon constantly-15.1.0 hyperlink-21.0.0 incremental-21.3.0 txAMQP-0.8.2 zope.interface-5.4.0
WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
You should consider upgrading via the '/usr/bin/python3.7 -m pip install --upgrade pip' command.
divine [ ~ ]$ sudo pip3 install whisper
Collecting whisper
  Downloading whisper-1.1.8.tar.gz (38 kB)
Requirement already satisfied: six in /usr/lib/python3.7/site-packages (from whisper) (1.15.0)
Using legacy 'setup.py install' for whisper, since package 'wheel' is not installed.
Installing collected packages: whisper
    Running setup.py install for whisper ... done
Successfully installed whisper-1.1.8
WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
You should consider upgrading via the '/usr/bin/python3.7 -m pip install --upgrade pip' command.

Edit /opt/graphite/webapp/graphite/local_settings.py and set the secret key, time zone, and anything else you need. Here is ours:

SECRET_KEY = 'anythingconsistentlaksjdflkjsdf'
TIME_ZONE = 'UTC'
USE_REMOTE_USER_AUTHENTICATION = True

Create the initial database:

root [ ~ ]# export GRAPHITE_ROOT=/opt/graphite
root [ ~ ]# PYTHONPATH=$GRAPHITE_ROOT/webapp django-admin.py migrate --settings=graphite.settings
Operations to perform:
  Apply all migrations: account, admin, auth, contenttypes, dashboard, events, sessions, tagging, tags, url_shortener
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying account.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying dashboard.0001_initial... OK
  Applying events.0001_initial... OK
  Applying sessions.0001_initial... OK
  Applying tagging.0001_initial... OK
  Applying tagging.0002_on_delete... OK
  Applying tags.0001_initial... OK
  Applying url_shortener.0001_initial... OK

Install mod_wsgi:

sudo pip3 install mod_wsgi
[sudo] password for divine: 
Collecting mod_wsgi
  Downloading mod_wsgi-4.8.0.tar.gz (2.6 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 2.6 MB 9.6 MB/s 
Using legacy 'setup.py install' for mod-wsgi, since package 'wheel' is not installed.
Installing collected packages: mod-wsgi
    Running setup.py install for mod-wsgi ... done
Successfully installed mod-wsgi-4.8.0
WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
You should consider upgrading via the '/usr/bin/python3.7 -m pip install --upgrade pip' command.

Create/edit /etc/httpd/httpd.conf. Here is ours:

Listen 4099
ServerName main.logintegrity.com
LoadModule mpm_event_module /usr/lib/httpd/modules/mod_mpm_event.so
LoadModule authn_file_module /usr/lib/httpd/modules/mod_authn_file.so
LoadModule authn_core_module /usr/lib/httpd/modules/mod_authn_core.so
LoadModule authz_host_module /usr/lib/httpd/modules/mod_authz_host.so
LoadModule authz_groupfile_module /usr/lib/httpd/modules/mod_authz_groupfile.so
LoadModule authz_user_module /usr/lib/httpd/modules/mod_authz_user.so
LoadModule authz_core_module /usr/lib/httpd/modules/mod_authz_core.so
LoadModule access_compat_module /usr/lib/httpd/modules/mod_access_compat.so
LoadModule auth_basic_module /usr/lib/httpd/modules/mod_auth_basic.so
LoadModule reqtimeout_module /usr/lib/httpd/modules/mod_reqtimeout.so
LoadModule filter_module /usr/lib/httpd/modules/mod_filter.so
LoadModule mime_module /usr/lib/httpd/modules/mod_mime.so
LoadModule log_config_module /usr/lib/httpd/modules/mod_log_config.so
LoadModule env_module /usr/lib/httpd/modules/mod_env.so
LoadModule headers_module /usr/lib/httpd/modules/mod_headers.so
LoadModule setenvif_module /usr/lib/httpd/modules/mod_setenvif.so
LoadModule version_module /usr/lib/httpd/modules/mod_version.so
LoadModule unixd_module /usr/lib/httpd/modules/mod_unixd.so
LoadModule status_module /usr/lib/httpd/modules/mod_status.so
LoadModule autoindex_module /usr/lib/httpd/modules/mod_autoindex.so
LoadModule dir_module /usr/lib/httpd/modules/mod_dir.so
LoadModule alias_module /usr/lib/httpd/modules/mod_alias.so
LoadModule wsgi_module '/usr/lib/python3.7/site-packages/mod_wsgi/server/mod_wsgi-py37.cpython-37m-x86_64-linux-gnu.so'

WSGISocketPrefix /var/run/wsgi

<IfModule unixd_module>
User apache
Group apache
</IfModule>
ServerAdmin admin@localhost
<Directory />
    AllowOverride none
    Require all denied
</Directory>
DocumentRoot "/srv/www"
<Directory "/srv/www">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>
<IfModule dir_module>
    DirectoryIndex index.html
</IfModule>
<Files ".ht*">
    Require all denied
</Files>
ErrorLog "/var/log/httpd/error.log"
LogLevel warn

<IfModule log_config_module>
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common
    <IfModule logio_module>
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>
    CustomLog "/var/log/httpd/access.log" common
</IfModule>


<IfModule cgid_module>
</IfModule>

<IfModule headers_module>
    RequestHeader unset Proxy early
</IfModule>

<IfModule mime_module>
    TypesConfig /etc/httpd/mime.types
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
</IfModule>
<IfModule proxy_html_module>
Include /etc/httpd/extra/proxy-html.conf
</IfModule>
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
<VirtualHost *:4099>
    ServerName graphite
    DocumentRoot "/opt/graphite/webapp"
    ErrorLog /opt/graphite/storage/log/webapp/error.log
    CustomLog /opt/graphite/storage/log/webapp/access.log common
    WSGIDaemonProcess graphite-web processes=5 threads=5 display-name='%{GROUP}' inactivity-timeout=120
    WSGIProcessGroup graphite-web
    WSGIApplicationGroup %{GLOBAL}
    WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite-web application-group=%{GLOBAL}
    Alias /static /opt/graphite/webapp/content
    WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi
    <Directory /opt/graphite/static>
            Require all granted
    </Directory>
    <Directory /opt/graphite/webapp>
            Require all granted
    </Directory>
    <Directory /opt/graphite/webapp/content>
            Require all granted
    </Directory>
    <Directory /opt/graphite/conf>
            Require all granted
    </Directory>
</VirtualHost>

Change perms:

chown apache:apache /opt/graphite/storage/graphite.db
chown apache:apache /opt/graphite/storage
chown apache:apache /opt/graphite/storage/log/webapp/*.log

Enter a couple events to test:

curl -X POST http://localhost:8080/events -d '{"what": "dfd map change", "tags" : "furiosa","when": 1392046352}'
curl -X POST http://localhost:8080/events -d '{"what": "dfd map change", "tags" : "furiosa"}'

We modified /opt/graphite/webapp/graphite/templates/events.html for our preferred timestamp:

{% load static %}<html>
  <head>
    <title>Events</title>
    <link rel="stylesheet" type="text/css" href="{% static "css/table.css" %}" />
    <style type="text/css">
    body {
        font-family: sans-serif;
        font-size: 16px;
        margin: 50px;
        max-width: 1200px;
    }
    </style>


    </head>
    <body>
        <div id="title" style="text-align:center">
            <h1>graphite events</h1>
        </div>
        <div class="graphite">
            <div id="main" >
              {% if events %}
                <table class="styledtable" width=100%>
                  <tr><th>when</th><th>what</th><th>tags</th></tr>
                  {% for event in events %}
                      <tr>
                          <td>{{event.when|date:"Y-m-d\TH:i:s:u\Z" }}</td>
                          <td><a href="{% url "events_detail" event.id %}">{{event.what}}</a></td>
                          <td>[&#39;{{ event.tags|join:"&#39;, &#39;"}}&#39;]</td>
                      </tr>
                  {% endfor %}
              {% else %}
                <br/>No events. Add events using
                <a href="{% url "admin:events_event_add" %}">the admin interface</a> or by posting
                (eg, curl -X POST {{ protocol }}://{{ site.domain }}{% url "events" %} -d
                '{"what": "Something Interesting", "tags" : "tag1"}')
              {% endif %}
              </table>
            </div>
        </div>
    </body>
</html>

Same for /opt/graphite/webapp/graphite/templates/event.html:

{% load static %}<html>
  <head>
    <title>{{event.what}}</title>
    <link rel="stylesheet" type="text/css" href="{% static "css/table.css" %}" />
    <style type="text/css">
    body {
        font-family: sans-serif;
        font-size: 16px;
        margin: 50px;
        max-width: 1200px;
    }
    </style>


    </head>
    <body>
        <div id="title" style="text-align:center">
            <h1>{{event.what}}</h1>
        </div>
        <div class="graphite">
            <div id="main" >
              <table class="styledtable" width=100%>
                <tr><td>when</td><td>{{event.when|date:"Y-m-d\TH:i:s:u\Z" }}</td></tr>
                <tr><td>tags</td><td>{{event.tags}}</td></tr>
                <tr><td>data</td><td>{{event.data}}</td></tr>
              </table>
            </div>
        </div>
    </body>
</html>

Here is what the webapp looks like with the above two events:

logging graphite

Articles tagged with graphite on Log Integrity:

2021-06-08: Hybrid DBs