Installing and Deploying QATrack+ on Ubuntu Linux¶

Note

This guide assumes you have at least a basic level of familiarity with Linux and the command line.

New Installation
Upgrading from version 0.2.8
Upgrading from version 0.2.9
Appendix 1: Hosting QATrack+ at a non-root URL

New Installation ¶

This guide is going to walk you through installing everything required to run QATrack+ on an Ubuntu 18.04 LTS (Bionic Beaver) server with Python 3.6, Apache 2.4 as the web server and PostgreSQL 10 (MySQL 5.5) as the database. Installation instructions should be similar on other Linux systems. If you are upgrading an existing installation, please see the sections below on upgrading from v0.2.8 or v0.2.9.

The steps we will be undertaking are:

Prerequisites
Installing and configuring Git
Check out the QATrack+ source code from BitBucket
Installing a Database System
- Installing PostgreSQL
- Installing MySQL (only required if you prefer to use MySQL over Postgres)
Setting up our Python environment (including virtualenv)
- Check your Python version
- Creating our virtual environment
Making sure everything is working up to this point
Installing Apache web server and mod_wsgi
- Installing Apache on Ubuntu 14.04
Final configuration of QATrack+
What Next
Last Word

If you hit an error along the way, stop and figure out why the error is occuring before proceeding with the next step!

Prerequisites ¶

Make sure your existing packages are up to date:

sudo apt-get update
sudo apt-get upgrade

You will need to have the make command and a few other packages available for this deployment. Install install them as follows:

sudo apt-get install make build-essential python3-dev python3-tk

Installing and configuring Git ¶

QATrack+ uses the git version controls system. Ensure you have git installed with the following command:

sudo apt-get install git

and then configure git (substituting your name and email address!)

git config --global user.name "randlet"
git config --global user.email randy@multileaf.ca

Check out the QATrack+ source code from BitBucket ¶

Now that we have git installed we can proceed to grab the latest version of QATrack+. To checkout the code enter the following commands:

mkdir -p ~/web
cd web
git clone https://bitbucket.org/tohccmedphys/qatrackplus.git

Installing a Database System ¶

It is highly recommended that you choose PostgreSQL for your database, however it is possible to use MySQL/MariaDB if you need to.

Installing PostgreSQL ¶

If you do not have an existing database server, you will need to install PostgreSQL locally. Run the following commands:

sudo apt-get install postgresql libpq-dev postgresql-client postgresql-client-common

After that completes, we can create a new Postgres user (db name/user/pwd = qatrackplus/qatrack/qatrackpass) as follows:

cd ~/web/qatrackplus
sudo -u postgres psql < deploy/postgres/create_db_and_role.sql

Now edit /etc/postgresql/10/main/pg_hba.conf (use your favourite editor, e.g. sudo nano /etc/postgresql/10/main/pg_hba.conf, note, if you have a different version of Postgres installed, then you would need to change the 10 in that path e.g. /etc/postgresql/9.3/main/pg_hba.conf) and scroll down to the bottom and change the instances of peer to md5 so it looks like:

# Database administrative login by Unix domain socket
local   all             postgres                                md5

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     md5
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     md5
host    replication     all             127.0.0.1/32            md5
host    replication     all             ::1/128                 md5

and restart the pg server:

sudo service postgresql restart

Installing MySQL (only required if you prefer to use MySQL over Postgres)¶

sudo apt-get install mysql-server libmysqlclient-dev

Note

You should use the InnoDB storage engine for MySQL. If you are using MySQL >= 5.5.5 then it uses InnoDB by default, otherwise if you are using MySQL < 5.5.5 you need to set the default storage engine to InnoDB: https://dev.mysql.com/doc/refman/5.5/en/storage-engine-setting.html

Now we can create and configure a user (db name/user/pwd = qatrackplus/qatrack/qatrackpass) and database for QATrack+:

# if you  set a password during mysql install
sudo mysql -u root -p < deploy/mysql/create_db_and_role.sql

# if you didn't
sudo mysql < deploy/mysql/create_db_and_role.sql

Setting up our Python environment (including virtualenv)¶

Check your Python version ¶

Unlike previous versions of QATrack+, version 0.3.0, runs on Python 3.5+ rather than Python 2.7. Check your version of python3 with the command:

python3 -V

Which should show the result Python 3.5.2 or similar. QATrack+ v0.3.0 is tested on Python versions 3.5.X, & 3.6.X but 3.4.x should also work. In order to keep QATrack+’s Python environment isolated from the system Python, we will run QATrack+ inside a Python Virtual Environment. To create the virtual environment run the following commands:

Creating our virtual environment ¶

sudo apt-get install python3-venv  # use python3.4-venv on Ubuntu 14.04
mkdir -p ~/venvs
python3 -m venv ~/venvs/qatrack3

Anytime you open a new terminal/shell to work with your QATrack+ installation you will want to activate your virtual environment. Do so now like this:

source ~/venvs/qatrack3/bin/activate

Your command prompt should now be prefixed with (qatrack3).

It’s also a good idea to upgrade pip the Python package installer:

pip install --upgrade pip

We will now install all the libraries required for QATrack+ with PostgresSQL (be patient, this can take a few minutes!):

cd ~/web/qatrackplus
pip install -r requirements/postgres.txt

or for MySQL:

cd ~/web/qatrackplus
pip install -r requirements/mysql.txt

Making sure everything is working up to this point ¶

At this point you can run the QATrack+ test suite to ensure your environment is set up correctly:

cd ~/web/qatrackplus
make test_simple

This should take a few minutes to run and should exit with output that looks similar to the following:

Results (88.45s):
passed
skipped
deselected

Installing Apache web server and mod_wsgi ¶

Warning

If you are on Ubuntu 14.04 please complete this section then complete the “Installing Apache on Ubuntu 14.04” section below!

The next step to take is to install and configure the Apache web server. Apache and mod_wsgi can be installed with the following commands:

sudo apt-get install apache2 apache2-dev libapache2-mod-wsgi-py3 python3-dev

Next, lets make sure Apache can write to our logs, media and cache directory:

sudo usermod -a -G www-data $USER
mkdir -p logs
touch logs/{migrate,debug}.log
chmod ug+rwx logs
chmod ug+rwx qatrack/media
chmod a+rw logs/{migrate,debug}.log

mkdir -p qatrack/cache
sudo chown -R www-data:www-data qatrack/cache/
sudo chmod -R g+rwxs qatrack/cache/

Now we can remove the default Apache config file and copy over the QATrack+ config file:

Danger

If you already have other sites running using the default config file you will want to edit it to include the directives relevant to QATrack+ rather than deleting it. Seek help if you’re unsure!

make qatrack_daemon.conf
sudo rm /etc/apache2/sites-enabled/000-default.conf

Installing Apache on Ubuntu 14.04 ¶

The process for installing Apache on Ubuntu 14.04 is a bit more complicated. If you can upgrade to 18.04 it is recommended you do so. Otherwise, read on (ref https://askubuntu.com/a/569551).

First uninstall the existing mod-wsgi-py3 package and make sure apache-dev is installed:

sudo apt-get remove libapache2-mod-wsgi-py3
sudo apt-get install apache2-dev
source ~/venvs/qatrack3/bin/activate
pip install mod_wsgi

Now install mod_wsgi into Apache:

sudo ~/venvs/qatrack3/bin/mod_wsgi-express install-module

which will result in two lines like:

LoadModule wsgi_module "/usr/lib/apache2/modules/mod_wsgi-py34.cpython-34m.so"
WSGIPythonHome "/home/ubuntu/venvs/qatrack3"

Write the first line to /etc/apache2/mods-available/wsgi_express.load and the second line to /etc/apache2/mods-available/wsgi_express.conf:

echo 'LoadModule wsgi_module "/usr/lib/apache2/modules/mod_wsgi-py34.cpython-34m.so"' | sudo tee --append /etc/apache2/mods-available/wsgi_express.conf
echo 'WSGIPythonHome "/home/ubuntu/venvs/qatrack3"' | sudo tee --append /etc/apache2/mods-available/wsgi_express.load

Now enable the wsgi_express module and restart Apache:

sudo a2enmod wsgi_express
sudo service apache2 restart

Final configuration of QATrack+¶

Next we need to tell QATrack+ how to connect to our database and (optionally) set some configuration options for your installation.

Create your local_settings.py file by copying the example from deploy/{postgres|mysql}/local_settings.py:

cp deploy/postgres/local_settings.py qatrack/local_settings.py
# or #
cp deploy/mysql/local_settings.py qatrack/local_settings.py

then open the file in a text editor. There are many available settings and they are documented within the example file and more completely on the settings page. Directions for setting up email are also included on that page.

However, the two most important settings are DATABASES and ALLOWED_HOSTS: which should be set like the following (switch the ENGINE to mysql if required):

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3'
        'NAME': 'qatrackplus',                      # Or path to database file if using sqlite3.
        'USER': 'qatrack',                      # Not used with sqlite3.
        'PASSWORD': 'qatrackpass',                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
    }
}


# Change XX.XXX.XXX.XX to your servers IP address and/or host name e.g. ALLOWED_HOSTS = ['54.123.45.1', 'yourhostname']
ALLOWED_HOSTS = ['XX.XXX.XXX.XX']

Once you have got those settings done, we can now create the tables in our database and install the default data:

python manage.py migrate
python manage.py loaddata fixtures/defaults/*/*

You also need to create a super user so you can login and begin configuring your Test Lists:

python manage.py createsuperuser

and finally we need to collect all our static media files in one location for Apache to serve and then restart Apache:

python manage.py collectstatic
sudo service apache2 restart

You should now be able to log into your server at http://yourserver/.

What Next ¶

Check the the settings page for any available customizations you want to add to your QATrack+ installation (don’t forget to restart Apache after changing any settings!)
Automate the backup of your QATrack+ installation.
Read the Administration Guide, User Guide, and Tutorials.

Last Word ¶

There are a lot of steps getting everything set up so don’t be discouraged if everything doesn’t go completely smoothly! If you run into trouble, please get in touch with me on the mailing list and I can help you out.

Upgrading from version 0.2.8 ¶

In order to upgrade from version 0.2.8 you must first uprade to version 0.2.9. If you hit an error along the way, stop and figure out why the error is occuring before proceeding with the next step! If you want assistance with the process, please post to to the Mailing List.

Activate your virtual environment
Backing up your database
Checking out version 0.2.9
Update your existing virtual environment
Migrate your database

Activate your virtual environment ¶

As usual, you will first want to activate your existing virtual environment:

source ~/venvs/qatrack/bin/activate

Backing up your database ¶

It is extremely important you back up your database before attempting to upgrade. You can either use your database to dump a backup file:

pg_dump -U <username> --password <dbname> > backup-0.2.8-$(date -I).sql   # e.g. pg_dump -U qatrack --password qatrackdb > backup-0.2.8-$(date -I).sql

# or for MySQL

mysqldump --user <username> --password <dbname> > backup-0.2.8-$(date -I).sql  # e.g. mysqldump --user qatrack --password qatrackdb > backup-0.2.8-$(date -I).sql

or generate a json dump of your database (possibly extremely slow!):

cd ~/web/qatrackplus
python manage.py dumpdata --natural > backup-0.2.8-$(date -I).json

Checking out version 0.2.9 ¶

First we must check out the code for version 0.2.9:

git fetch origin
git checkout v0.2.9.1

Warning

If you get any errors using git (e.g. trying to check out v0.2.9.1) that you don’t know how to handle, please stop and get help!

Update your existing virtual environment ¶

There were a number of changes in dependencies for version 0.2.9 so we need to update our virtual env:

pip install --upgrade pip
pip install -r requirements/base.txt

Migrate your database ¶

Note

You should use the InnoDB storage engine for MySQL. If you are using MySQL >= 5.5.5 then it uses InnoDB by default, otherwise if you are using MySQL < 5.5.5 you need to set the default storage engine to InnoDB: https://dev.mysql.com/doc/refman/5.5/en/storage-engine-setting.html

The next step is to migrate the 0.2.8 database schema to 0.2.9:

python manage.py syncdb
python manage.py migrate

Assuming that proceeds without errors you can proceed to Upgrading from version 0.2.9 below.

Upgrading from version 0.2.9 ¶

The steps below will guide you through upgrading a version 0.2.9 installation to 0.3.0. If you hit an error along the way, stop and figure out why the error is occuring before proceeding with the next step!

Verifying your Python 3 version
Backing up your database
Checking out version 0.3.0
Create and activate your new virtual environment
Migrate your database
- Check the migration log
Update your local_settings.py file
Update your Apache configuration
Last Word

Verifying your Python 3 version ¶

Unlike QATrack+ v0.2.9 which runs on Python 2.7, QATrack+ 0.3.0 only runs on Python version 3.5 or 3.6 (and probably 3.4!). You will need to ensure you have one of those Python versions installed:

python3 -V
# should result in e.g.
Python 3.5.2

If you don’t see either Python 3.4.X, 3.5.X or, 3.6.X then you will need to install Python 3 on your system (beyond the scope of this document).

Backing up your database ¶

It is extremely important you back up your database before attempting to upgrade. You can either use your database to dump a backup file:

pg_dump -U <username> --password <dbname> > backup-0.2.9-$(date -I).sql   # e.g. pg_dump -U qatrack --password qatrackdb > backup-0.2.9-$(date -I).sql

# or for MySQL

mysqldump --user <username> --password <dbname> > backup-0.2.9-$(date -I).sql  # e.g. mysqldump --user qatrack --password qatrackdb > backup-0.2.9-$(date -I).sql

or generate a json dump of your database (possibly extremely slow!):

source ~/venvs/qatrack/bin/activate
python manage.py dumpdata --natural > backup-0.2.9-$(date -I).json
deactivate

Checking out version 0.3.0 ¶

First we must check out the code for version 0.3.0:

git fetch origin
git checkout v0.3.0.19

Create and activate your new virtual environment ¶

If you currently have a virtualenv activated, deactivate it with the deactivate command:

deactivate

We need to create a new virtual environment with the Python 3 interpreter:

sudo apt-get install python3-venv
python3 -m venv ~/venvs/qatrack3
source ~/venvs/qatrack3/bin/activate

and we can then install the required python libraries:

pip install -r requirements/postgres.txt  # or requirements/mysql.txt

Migrate your database ¶

Note

You should use the InnoDB storage engine for MySQL. If you are using MySQL >= 5.5.5 then it uses InnoDB by default, otherwise if you are using MySQL < 5.5.5 you need to set the default storage engine to InnoDB: https://dev.mysql.com/doc/refman/5.5/en/storage-engine-setting.html

The next step is to update the v0.2.9 schema to v0.3.0

python manage.py migrate --fake-initial

and load some initial service log data:

python manage.py loaddata fixtures/defaults/units/*
python manage.py loaddata fixtures/defaults/service_log/*

Check the migration log ¶

During the migration above you may have noticed some warnings like:

Note: if any of the following tests process binary files (e.g. images, dicom files etc) rather than plain text, you must edit the calculation and replace ‘FILE’ with ‘BIN_FILE’. Tests:

Test name 1 (test-1)

Test name 2 (test-2)

…

This data is also available in the logs/migrate.log file. Because the way Python handles text encodings / files has changed in Python 3, you will need to update any upload test that handles binary data by changing the FILE reference in the calculation procedure to BIN_FILE. For example change:

data = FILE.read()
# do something with data

to:

data = BIN_FILE.read()
# do something with data

You may have also seen warnings like:

The test named ‘yourtestname’ with ID=1234 needs to be updated to be

compatible with Python 3.

While most Test calculation procedures will be compatible with both Python 2 and Python 3, there have been some syntactical changes in the language which may require you to update a calculation procedure to be Python 3 compatible.

Update your local_settings.py file ¶

Now is a good time to review your local_settings.py file. There are a few new settings that you may want to configure. The settings are documented in the settings page.

Update your Apache configuration ¶

First, lets make sure Apache can write to our logs and media directory:

sudo usermod -a -G www-data $USER
mkdir -p logs
touch logs/{migrate,debug}.log
chmod ug+rwx logs
chmod ug+rwx qatrack/media
chmod a+rw logs/{migrate,debug}.log

Since we are now using a different Python virtual environment we need to update the WSGIPythonHome variable. Open your Apache config file (either /etc/apach2/sites-available/qatrack.conf or /etc/apache2/sites-available/default.conf or /etc/apache2/httpd.conf) and set the virtualenv path correctly:

WSGIPythonHome /home/YOURUSERNAME/venvs/qatrack3

# or for daemon mode

WSGIDaemonProcess qatrackplus python-home=/home/YOURUSERNAMEHERE/venvs/qatrack3 python-path=/home/YOURUSERNAMEHERE/web/qatrackplus

and then restart Apache:

sudo service apache2 restart

Last Word ¶

There are a lot of steps getting everything set up so don’t be discouraged if everything doesn’t go completely smoothly! If you run into trouble, please get in touch on the mailing list.

Appendix 1: Hosting QATrack+ at a non-root URL ¶

If you want to host QATrack+ somewhere other than the root of your server (e.g. you want to host the QATrack+ application at http://myserver/qatrackplus/), you will need to ensure mod_rewrite is enabled:

sudo a2enmod rewrite
sudo service apache2 restart

and you will need to include the following lines in your qatrack/local_settings.py file

FORCE_SCRIPT_NAME = "/qatrackplus"
LOGIN_EXEMPT_URLS = [r"^qatrackplus/accounts/", r"qatrackplus/api/*"]
LOGIN_REDIRECT_URL = "/qatrackplus/qa/unit/"
LOGIN_URL = "/qatrackplus/accounts/login/"

and edit /etc/apache/sites-available/qatrack.conf so that the WSGIScriptAlias is set correctly:

WSGIScriptAlias /qatrackplus /home/YOURUSERNAMEHERE/web/qatrackplus/qatrack/wsgi.py