Browser Performance Testing (PREMIUM)
If your application offers a web interface and you are using GitLab CI/CD, you can quickly determine the performance impact of pending code changes.
GitLab uses Sitespeed.io, a free and open source
tool for measuring the performance of web sites, and has built a simple
which outputs the results in a file called
performance.json. This plugin
outputs the performance score for each page that is analyzed.
Going a step further, GitLab can show the Performance report right in the merge request widget area (see below).
For instance, consider the following workflow:
- A member of the marketing team is attempting to track engagement by adding a new tool.
- With browser performance metrics, they see how their changes are impacting the usability of the page for end users.
- The metrics show that after their changes the performance score of the page has gone down.
<head>which affects loading page speed.
- They ask a front end developer to help them, who sets the library to load asynchronously.
- The frontend developer approves the merge request and authorizes its deployment to production.
How it works
First of all, you need to define a job in your
.gitlab-ci.yml file that generates the
Performance report artifact.
For more information on how the Performance job should look like, check the
example on Configuring Browser Performance Testing.
GitLab then checks this report, compares key performance metrics for each page between the source and target branches, and shows the information right on the merge request.
If the Performance report doesn't have anything to compare to, no information
will be displayed in the merge request area. That is the case when you add the
Performance job in your
.gitlab-ci.yml for the very first time.
Consecutive merge requests will have something to compare to, and the Performance
report will be shown properly.
Configuring Browser Performance Testing
First, you need GitLab Runner with docker-in-docker build.
Once you set up the Runner, add a new job to
.gitlab-ci.yml that generates the
For GitLab 12.4 and later, to define the
performance job, you must
that's provided as a part of your GitLab installation.
For GitLab versions earlier than 12.4, you can copy and use the job as defined
in that template.
CAUTION: Caution: The job definition provided by the template does not support Kubernetes yet. For a complete example of a more complex setup that works in Kubernetes, see here.
Add the following to your
include: template: Verify/Browser-Performance.gitlab-ci.yml performance: variables: URL: https://example.com
CAUTION: Caution: The job definition provided by the template is supported in GitLab 11.5 and later versions. It also requires GitLab Runner 11.5 or later. For earlier versions, use the previous job definitions.
The above example will create a
performance job in your CI/CD pipeline and will run
sitespeed.io against the webpage you defined in
URL to gather key metrics.
The GitLab plugin for sitespeed.io
is downloaded in order to save the report as a Performance report artifact
that you can later download and analyze. Due to implementation limitations we always
take the latest Performance artifact available.
The full HTML sitespeed.io report will also be saved as an artifact, and if you have GitLab Pages enabled, it can be viewed directly in your browser.
It is also possible to customize options by setting the
For example, this is how to override the number of runs sitespeed.io
will make on the given URL:
include: template: Verify/Browser-Performance.gitlab-ci.yml performance: variables: URL: https://example.com SITESPEED_OPTIONS: -n 5
For further customization options for sitespeed.io, including the ability to provide a list of URLs to test, please see the Sitespeed.io Configuration documentation.
TIP: Tip: Key metrics are automatically extracted and shown in the merge request widget.
Performance testing on Review Apps
The above CI YML is great for testing against static environments, and it can be extended for dynamic environments. There are a few extra steps to take to set this up:
performancejob should run after the dynamic environment has started.
- In the
reviewjob, persist the hostname and upload it as an artifact so it's available to the
performancejob (the same can be done for static environments like staging and production to unify the code path). Saving it as an artifact is as simple as
echo $CI_ENVIRONMENT_URL > environment_url.txtin your job's
- In the
performancejob, read the previous artifact into an environment variable, in this case
$URLbecause this is what our sitespeed.io command uses for the URL parameter. Because Review App URLs are dynamic, we define the
- You can now run the sitespeed.io container against the desired hostname and paths.
.gitlab-ci.yml file would look like:
stages: - deploy - performance include: template: Verify/Browser-Performance.gitlab-ci.yml review: stage: deploy environment: name: review/$CI_COMMIT_REF_SLUG url: http://$CI_COMMIT_REF_SLUG.$APPS_DOMAIN script: - run_deploy_script - echo $CI_ENVIRONMENT_URL > environment_url.txt artifacts: paths: - environment_url.txt only: - branches except: - master performance: dependencies: - review before_script: - export URL=$(cat environment_url.txt)
Previous job definitions
Before GitLab 11.5, Performance job and artifact had to be named specifically
to automatically extract report data and show it in the merge request widget.
While these old job definitions are still maintained they have been deprecated
and may be removed in next major release, GitLab 12.0.
You are advised to update your current
.gitlab-ci.yml configuration to reflect that change.
For GitLab 11.4 and earlier, the job should look like:
performance: stage: performance image: docker:git variables: URL: https://example.com services: - docker:stable-dind script: - mkdir gitlab-exporter - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js - mkdir sitespeed-results - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL - mv sitespeed-results/data/performance.json performance.json artifacts: paths: - performance.json - sitespeed-results/