Specify Linux Server Installation Guide

:open_book: This guide provides comprehensive command-line instructions for IT administrators to install a self-managed Specify 7 instance on a Linux server using Docker.

Webinar: Installing Specify on a Linux Server

Transcript:
Installing Specify on a Linux Server Webinar Transcript (1).vtt (53.8 KB)

Command-Line Conventions

When following this guide, rely on these formatting conventions to interpret the commands:

  • $: Indicates shell input that you must type.

  • #: Indicates privileged shell input requiring sudo.

  • No symbol: Text without a preceding symbol represents the output returned by the computer.

  • $VARIABLE_NAME: Fully capitalized text preceded by a dollar sign represents a system-specific variable you must replace with your own value.

  • file.txt: A slide with a title that looks like a filename by itself will only contain the contents of said file as the body of the slide.

For more in-depth instructions for using the command-line, explore Ubuntu’s The Linux command line for beginners - Ubuntu Desktop documentation or Dave Eddy’s Complete Bash Scripting Course available on YouTube.

Supported Distributions and Deployment Architecture

  • Linux Distributions: Ubuntu Server is the officially supported distribution. Other Linux distributions can be used (for example, Rocky Linux has been used to meet institutional security requirements), but Specify will not troubleshoot OS-level issues on unsupported distributions.

  • Container Platform: Since Linux is not natively equipped to handle the containers that accompany Specify, a separate platform must be used to run containers. Docker is the primary development container platform for Specify 7. Podman offers better security configuration (rootless by default) but Specify typically requires root permissions; instructions for rootless Podman are available in the repository. Specify’s containers interact as shown below:

  • Deployment Options:

    • All-in-one: Orchestrates all containers into a single environment, best for single instances maintaining one database (which can hold multiple collections).

    • Specify Cloud: The deployment structure used by Specify for hosted instances, designed for institutions managing multiple separate databases or multiple Specify instances.

Prerequisites and Server Preparation

The server must have SSH configured for remote access, Git, Docker (or Podman), and a text editor like Vim or Nano installed.

  1. Install dependencies: Run sudo apt install docker-compose-v2 git.

  2. Configure Docker permissions: Add your user to the Docker group by running sudo usermod -aG docker "$USER". You must completely log out of the server and log back in for these user permissions to take effect.

  3. Generate SSH Key: Execute ssh-keygen. It is highly recommended to set a passphrase during this process. Once generated, copy the contents of the .pub file into a new SSH key within your GitHub account settings.

  4. Clone the Repository: You must email your GitHub username and institution to support@specifysoftware.org to request access to the private docker-compositions repository. Once granted access, clone it via git clone git@github.com:specify/docker-compositions.git and navigate to the docker/all-in-one/ directory. After cloning the repository, the structure should look similar to the image below. NOTE: Please write alt text

Modifying Configuration Files

Configuration requires editing both specify.env and docker-compose.yml. Many environment variables containing “host” in their name can be left to default container names, except for the asset server.

1. Configuring specify.env

  • Database Settings: Verify DATABASE_NAME accurately reflects your desired database name, or your existing Specify 6 database name if migrating.

  • Users and Passwords: You must set strong, unique passwords for the MASTER_PASSWORD (provisions users), MIGRATOR_PASSWORD (runs database migrations), and APP_USER_PASSWORD (handles day-to-day Specify operations). Do not reuse passwords across these users. To generate secure passwords via the command line, run this pipeline head -c 30 /dev/urandom | base64 and copy the output. head -c will read the specified number of bytes from a file. /dev/urandom is a special file on linux systems that is populated with random data. base64 will encode that random data into a set alphabet with 64 characters (like how english has 26).

  • Secret Key: The SECRET_KEY is utilized for web logins and must be a unique, hard-to-compute string.

  • Host URLs: Restrict ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS to the exact IP addresses or URLs (e.g., specify.kunhm.org) authorized to access your network. The host segment of the ASSET_SERVER_URL must identically match the URL users will type into their browser.

  • Asset Server Key: The ASSET_SERVER_KEY does not strictly need to be cryptographically secure, but it must be unique to prevent collisions, so a secure key is recommended.

  • Time Zone: Set TIME_ZONE to your local timezone (referencing /usr/share/zoneinfo/, a directory on most linux systems that contains timezone info for countries and cities; find a country and city that is in your timezone) to ensure timestamps are accurate during data exports.

  • Variables to Leave Alone: Do not modify REPORT_RUNNER, CELERY, or SPECIFY7_CONFIG_DIR unless instructed by support staff. Keep LOG_LEVEL at warning and SP7_DEBUG set to false, as modifying these can slow down server performance.

2. Configuring docker-compose.yml

  • MariaDB Matching: Under mariadb.environment, ensure MYSQL_ROOT_PASSWORD matches the root password in your .env file, and that MYSQL_USER and MYSQL_PASSWORD exactly match the master name and password variables. If these mismatch, Specify will fail to connect.

  • Specify 7 Image: Leave the image version string as specifyconsortium/specify7-service:v7 to ensure you receive automatic updates. Do not delete the text preceding the colon, as Docker requires this string to locate the container on Docker Hub. Ensure extra_hosts mappings correctly point to the host gateway.

  • Asset Server: Under asset-server.environment, ensure SERVER_NAME matches your web URL, ATTACHMENT_KEY matches the key in the .env file, and DEBUG_MODE is set to false. Note that the default asset server key variable uses underscores in the .env file but not in the compose file.

MariaDB Setup and Privileges

If you are deploying an existing Specify 6 database, place the .sql dump file into the seed-database folder and add MYSQL_DATABASE=<your_database_name> under the MariaDB environment section of the compose file.

If you do not have a seed database, you must manually grant privileges:

  1. Start only the MariaDB container by executing docker compose up mariadb (or -d for detached).

  2. Identify the container name using docker ps.

  3. Open an interactive terminal within the container by running docker exec -it <mariadb_container_name> mariadb -uroot -p$MYSQL_ROOT_PASSWORD. Ensure there is no space between the -p flag and the password variable.

  4. Run the following queries (replacing variables with your specific names and passwords) to allow the master user to create additional users, check user status, update privileges, and access the main Specify database:

  • GRANT CREATE USER, RELOAD ON \*.\* TO $MASTER_USER@’%’ IDENTIFIED BY ‘$MASTER_PASSWORD’

  • GRANT SELECT ON MYSQL.\* TO $MASTER_USER@’%’ IDENTIFIED BY ‘$MASTER_PASSWORD’

  • GRANT ALL ON '$DATABASE_NAME'.\* TO $MASTER_USER@’%’ IDENTIFIED BY ‘$MASTER_PASSWORD' WITH GRANT OPTION;

Note on Configuration Errors: If you realize your master passwords do not match across files during startup, execute docker compose down -v to destroy the volumes and reset the state to a blank slate, then repeat the MariaDB setup.

Launching the Application

  • Starting Up: Run docker compose pull followed by docker compose up to initialize the system.

  • Django Migrations: Specify will immediately begin running Django migrations. If there is no seed database, it will run initial migration 0001 to build the entire database schema from scratch, which will take several minutes depending on server specifications.

  • System Monitoring: During migrations, navigating to the URL will yield a “502 bad gateway” or “connection refused” error from NGINX because Specify is not yet accepting connections. To verify the system is actively working, open a split terminal (using software like tmux) and run the top command to monitor the Python process consuming CPU resources.

  • Migration Completion: You will see migration numbers that appear non-uniform or out of order; this is normal as they are applying to different Django apps. The setup is complete when the logs display “no duplicates found” and NGINX spawns a Gunicorn worker, at which point the browser will load the guided setup page.

Post-Installation Operations

  • Security Measures: Secure the server by configuring HTTPS traffic, enabling a firewall, and locking down the ALLOWED_HOSTS variable to solely the domain expected for access.

  • Automated Backups: Utilize a cron job to automate database backups. Compress these backups to reduce file size from gigabytes to megabytes. Retain daily backups for one week, Friday backups for one month, and the first Friday backup of each month indefinitely.

  • Update Plan: Stay informed of updates via the Speciforum. To update, execute git pull on the compositions repository followed by docker pull. Updates can be scheduled via a cron job or applied manually, but you should always run a database backup prior to initiating any update process.

Additional Documentation

Webinar Slide Deck

Installing Specify on a Linux Server Webinar Slides.pdf (1.2 MB)

La implementación actual basada en MariaDB/MySQL ha demostrado ser estable, rápida y adecuada para el volumen de información gestionado por Specify. Sin embargo, considerando que el núcleo de la aplicación descansa sobre Django, una evolución hacia una capa de persistencia más independiente del motor de base de datos podría ofrecer mayor flexibilidad institucional. Ceteris paribus, esto abriría la puerta a motores como PostgreSQL, reconocido por sus capacidades avanzadas y estándares, o SQL Server en entornos corporativos Microsoft, manteniendo al mismo tiempo la fortaleza y sencillez operativa que hoy ofrece MariaDB. :+1: