This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Deployment

Horreum Deployment Guides

Get Horreum running locally and upload your first data

1 - Bare-metal

Install Horreum on a bare-metal machine

This guide documents production installation without helper *-compose scripts.

Setup database

You need PostgreSQL 12 (or later) installed; setup is out of scope of this guide.

PostgreSQL must be installed with SSL support. In short, you’ll need to setup at least the following SSL properties: ssl=on, ssl_cert_file and ssl_key_file. Horreum will also use the server certificate file in order to verify the connection.

Create a new database for the application called horreum and limited-privilege user appuser with a secure password:

export PGHOST=127.0.0.1
export PGPORT=5432
export PGUSER=dbadmin
export PGPASSWORD="Curr3ntAdm!nPwd"
psql -c "CREATE DATABASE horreum" postgres
export PGDATABASE=horreum
psql -c "CREATE ROLE \"appuser\" noinherit login password 'SecurEpaSSw0!2D';" postgres

Now you need to setup a Keycloak user and database:

psql -c "CREATE ROLE \"keycloakuser\" noinherit login password 'An0th3rPA55w0rD';"
psql -c "CREATE DATABASE keycloak WITH OWNER = 'keycloakuser';"

Keycloak setup

For complete Keycloak setup please refer to Keycloak Getting Started - you can also use existing Keycloak instance.

Get the realm definition and import it:

REALM_CONFIG=$(mktemp horreum-docker-compose.XXXX.yaml)
curl https://raw.githubusercontent.com/Hyperfoil/Horreum/master/infra/keycloak-horreum.json \
    -s -o $REALM_CONFIG
./bin/standalone.sh \
    -Dkeycloak.profile.feature.upload_scripts=enabled \
    -Dkeycloak.migration.action=import \
    -Dkeycloak.migration.provider=singleFile \
    -Dkeycloak.migration.file=$REALM_CONFIG \
    -Dkeycloak.migration.strategy=IGNORE_EXISTING

When Keycloak starts you should access its admin console and adjust URLs for clients horreum and horreum-ui:

  • Root URL (rootUrl)
  • Valid Redirect URIs (redirectUris) - make sure to include the /* to match all subpaths
  • Admin URL (adminUrl)
  • Web Oridins (webOrigins)

After that create the role __user_reader, go to ‘Role Mappings’ tab, select realm-management in Client Roles and give this user the view-users role. Make sure that this user has the offline_access Realm Role as well.

Now you can create team roles, users and assign them appropriately. For correct integration with Grafana please remember to set email for each user (this will be used purely to match Grafana identities).

You should also open horreum client, switch to ‘Credentials’ tab and record the Secret (UUID identifier).

Starting Horreum

Horreum is a Quarkus application and is configured using one of these:

  • Java system properties when starting the application
  • Exported environment variables
  • Environment variables definition in .env file in the current working directory

You should set up these variables:

# --- DATABASE ---
# These two URLs should be identical
QUARKUS_DATASOURCE_MIGRATION_JDBC_URL=jdbc:postgresql://db.local:5432/horreum
QUARKUS_DATASOURCE_JDBC_URL=jdbc:postgresql://db.local:5432/horreum
# This is the regular user Horreum will use for DB access
QUARKUS_DATASOURCE_USERNAME=appuser
QUARKUS_DATASOURCE_PASSWORD=SecurEpaSSw0!2D
# This is the user that has full control over 'horreum' database
QUARKUS_DATASOURCE_MIGRATION_USERNAME=dbadmin
QUARKUS_DATASOURCE_MIGRATION_PASSWORD=Curr3ntAdm!nPwd
# The database server SSL certificate
QUARKUS_DATASOURCE_JDBC_ADDITIONAL-JDBC-PROPERTIES_SSLROOTCERT=server.crt
# As an alternative, certificate validation can be disabled with
# QUARKUS_DATASOURCE_JDBC_ADDITIONAL-JDBC-PROPERTIES_SSLMODE=require
# Secret generated during database setup: run `SELECT * FROM dbsecret` as DB admin
HORREUM_DB_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxx

# --- KEYCLOAK ---
# This URL must be accessible from Horreum, but does not have to be exposed to the world
QUARKUS_OIDC_AUTH_SERVER_URL=https://keycloak.local/auth/realms/horreum
# You might need to set this property to the external Keycloak URL
QUARKUS_OIDC_TOKEN_ISSUER=https://keycloak.example.com
# Secret found in Keycloak console
QUARKUS_OIDC_CREDENTIALS_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
# Make sure to include the /auth path. This URL must be externally accessible.
HORREUM_KEYCLOAK_URL=https://keycloak.example.com/auth
# Keycloak URL for internal access
HORREUM_KEYCLOAK_MP_REST_URL=https://keycloak.local

# --- Grafana ---
HORREUM_GRAFANA_ADMIN_PASSWORD=g12aPHana+Pwd
# External Grafana URL
HORREUM_GRAFANA_URL=https://grafana.example.com:3443
# Internal Grafana URL. This should be secured as ATM Horreum sends the credentials using Basic auth.
HORREUM_GRAFANA_MP_REST_URL=https://grafana.local:3443

# --- HTTP ---
# You might also want to set the IP the webserver is listening to
QUARKUS_HTTP_HOST=123.45.67.89
QUARKUS_HTTP_PORT=443
# Any production instance should be run over secured connections
QUARKUS_HTTP_SSL_CERTIFICATE_KEY_STORE_FILE=/path/to/keystore.jks
QUARKUS_HTTP_SSL_CERTIFICATE_KEY_STORE_PASSWORD=keystore-password
QUARKUS_HTTP_INSECURE_REQUESTS=disabled
# If you share the certificate for Horreum and Keycloak/Grafana disable HTTP/2 to avoid connection coalescing
QUARKUS_HTTP_HTTP2=false
# URL for Horreum external access (advertised in permanent links etc.)
HORREUM_URL=https://horreum.example.com
# Internal URL for services that load data from Horreum (e.g. Grafana)
HORREUM_INTERNAL_URL=https://horreum.local:8443

# --- Mailserver ---
QUARKUS_MAILER_FROM=horreum@horreum.example.com
QUARKUS_MAILER_HOST=smtp.example.com
QUARKUS_MAILER_PORT=25
QUARKUS_MAILER_START_TLS=disabled
QUARKUS_MAILER_LOGIN=disabled

# --- Other ---
# By default webhook notifications that fail to verify TLS integrity fail; set this to ignore verification result.
# HORREUM_HOOK_TLS_INSECURE=true

# This is an offline token for the __user_reader user. You can obtain that with
# curl -s -X POST https://keycloak.example.com/auth/realms/horreum/protocol/openid-connect/token \
#      -H 'content-type: application/x-www-form-urlencoded' \
#      -d 'username=__user_reader&password='$PASSWORD'&grant_type=password&client_id=horreum-ui&scope=offline_access' \
#     | jq -r .refresh_token
HORREUM_KEYCLOAK_USER_READER_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

With all this in you can finally start Horreum as any other Java application (note: dependencies in repo/target/lib/ must be present):

java -jar repo/target/repo-1.0.0-SNAPSHOT-runner.jar

2 - OpenShift/Kubernetes

Install Horreum on a OpenShift/Kubernetes cluster

Deploying Horreum for production in OpenShift/Kubernetes is easy using Horreum operator; you can install it through the Operator Marketplace/Operator Hub. This operator installs Horreum and the services it depends on - PostgreSQL database and Keycloak SSO.

Installation steps

  1. From OperatorHub install horreum in your namespace.
  2. Create a pvc in the namespace where horreum operator was installed (oc apply -f <your_pvc_filename>.yaml -n <your_horreum_namespace>). The pvc is required by the postgres database and must be in place before the database pod is created by the operator. PVC should have the same name as specified in the CR definition below. See example pvc definition:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: horreum-postgres
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeMode: Filesystem
  1. Create horreum CR, so that all pods will be created by the horreum operator (oc apply -f <your_cr_filename>.yaml -n <your_horreum_namespace>). See an example of the horreum resource:
apiVersion: hyperfoil.io/v1alpha1
kind: Horreum
metadata:
  name: example-horreum
spec:
  route:
    host: horreum.apps.mycloud.example.com
  keycloak:
    route:
      host: keycloak.apps.mycloud.example.com
  grafana:
    route:
      host: grafana.apps.mycloud.example.com
  postgres:
    persistentVolumeClaim: horreum-postgres
  report:
    route:
      host: hyperfoil-report.apps.mycloud.example.com
    persistentVolumeClaim: hyperfoil-report

For detailed description of all properties refer to the CRD.

When using persistent volumes make sure that the access rights are set correctly and the pods have write access; in particular the PostgreSQL database requires that the mapped directory is owned by user with id 999.

If you’re planning to use secured routes (edge termination) it is recommended to set the tls: my-tls-secret at the first deploy; otherwise it is necessary to update URLs for clients horreum and horreum-ui in Keycloak manually. Also the Horreum pod needs to be restarted after Keycloak route update.

Currently you must set both Horreum and Keycloak route host explicitly, otherwise you could not log in (TODO).

When the horreum resource gets ready, login into Keycloak using administrator credentials (these are automatically created if you don’t specify existing secret) and create a new user in the horreum realm, a new team role (with -team suffix) and assign it to the user along with other appropriate predefined roles. Administrator credentials can be found using this:

NAME=$(oc get horreum -o jsonpath='{$.items[0].metadata.name}')
oc get secret $NAME-keycloak-admin -o json | \
    jq '{ user: .data.username | @base64d, password: .data.password | @base64d }'

For details of roles in Horreum please refer to user management.

Hyperfoil integration

For your convenience this operator creates also a config map (*-hyperfoil-upload) that can be used in Hyperfoil resource to upload Hyperfoil results to this instance - you can use it directly or merge that into another config map you use for post-hooks. However, it is necessary to define & mount a secret with these keys:

# Credentials of the user you've created in Keycloak
HORREUM_USER=user
HORREUM_PASSWORD=password
# Role for the team the user belongs to (something you've created)
HORREUM_GROUP=engineers-team

oc create secret generic hyperfoil-horreum \
    --from-literal=HORREUM_USER=$HORREUM_USER \
    --from-literal=HORREUM_PASSWORD=$HORREUM_PASSWORD \
    --from-literal=HORREUM_GROUP=$HORREUM_GROUP \

Then set it up in the hyperfoil resource:

apiVersion: hyperfoil.io/v1alpha1
kind: Hyperfoil
metadata:
  name: example-hyperfoil
  namespace: hyperfoil
spec:
  # ...
  postHooks: example-horreum-hyperfoil-upload
  secretEnvVars:
    - hyperfoil-horreum

This operator automatically inserts a webhook to convert test results into Hyperfoil report; In order to link from test to report you have to add a schema (matching the URI used in your Hyperfoil version, usually something like http://hyperfoil.io/run-schema/0.8 and add it an extractor info with JSON path .info. Subsequently go to the test and add a view component with header ‘Report’, accessor you’ve created in the previous step and this rendering script (replacing the hostname):

(value, all) => {
  let info = JSON.parse(value);
  return (
    '<a href="http://example-horreum-report-hyperfoil.apps.mycloud.example.com/' +
    all.id +
    "-" +
    info.id +
    "-" +
    info.benchmark +
    '.html" target=_blank>Show</a>'
  );
};