Update the documentation with an upgrade guide

Closes #28 (closed)

Merge request reports

Loading
+693 −0
Changes for doc/getting_started/upgrade_guide/2025Q4_2026Q2.md: 693 added lines, 0 removed lines.
Original line number Diff line number Diff line
# OpenSlice Upgrade Guide from 2025Q4 to 2026Q2


This guide upgrades a running OpenSlice instance from release 2025Q4 to release 2026Q2. Follow the
section that matches how the instance was installed: [Kubernetes](#upgrading-a-kubernetes-deployment) or [Docker Compose](#upgrading-a-docker-compose-deployment).

The two paths differ in where your configuration lives. Docker Compose keeps it in generated files
that sit beside templates the new release updates, so every upgrade asks you to merge. Helm keeps it
in a values file outside the chart, which the upgrade never touches.


### What changes in this release

The complete difference between the two tags, across both deployment methods, as perceived in the main project:

| File | Change |
| ---- | ------ |
| `compose/docker-compose.yaml.configure` | `SECRETS_CONTROLLER_ENABLE` added to the `osom` service |
| `compose/deploy.sh` | now clones `org.etsi.osl.mcp.backend` |
| `compose/docker-images-build.sh` | removed |
| `compose/readme_old.txt` | removed |
| `kubernetes/helm/openslice/Chart.yaml` | chart 3.0.0 to 4.1.0, appVersion 2026Q2 |
| `kubernetes/helm/openslice/values.yaml` | `osom.secretsController.enable` added |
| `kubernetes/helm/openslice/templates/osom.yaml` | passes that value through |
| `config.prod.default.json`, `config.theming.default.json` | version string |
| `pom.xml` | module versions |

No service is added or removed, no infrastructure image changes and the Keycloak realm export is untouched. This is a routine upgrade.

You can confirm the same on your own checkout:

```bash
cd org.etsi.osl.main
git fetch --all --tags
git diff --stat tags/2025Q4 tags/2026Q2 -- compose kubernetes pom.xml
```

> **Note:** Always write `tags/` in front of a release name. Installing with
> `git clone --branch 2025Q4` leaves a local branch of the same name beside the tag, so the bare
> name is ambiguous and git resolves it to the tag with only a warning. The collision affects
> 2025Q4, never 2026Q2.

---

## Upgrading a Kubernetes deployment

The chart is at `[repo-root]/kubernetes/helm/openslice`. The examples below use `openslice` as both
the namespace and the Helm release name; substitute your own if they differ.

### 1. Prepare the environment

```bash
helm list -n openslice                              # CHART openslice-3.0.0, APP VERSION 2025Q4
helm get values openslice -n openslice > my-values.yaml
```

If that file contains `null`, your settings live in the chart's own `values.yaml`. This is the usual
outcome, because the 2025Q4 deployment guide instructs you to edit that file directly. Two problems
follow: `git checkout` refuses to move while a tracked file carries your edits, and an upgrade fed an
empty values file resets `rooturl`, `storageClass`, `oscreds` and every image tag to the shipped
defaults.

Move your settings out of the chart and hand the chart's own file back to git:

```bash
cd kubernetes/helm/openslice
git diff values.yaml              # every line here belongs in my-values.yaml
cp -n values.yaml ~/values.yaml.mine
git checkout -- values.yaml
git status --porcelain            # values.yaml must no longer appear
cd ../../..
```

> **Important Note:** `cp -n` is deliberate. If you run this step a second time, a plain `cp` would
> copy the file that `git checkout --` has already restored, replacing your saved settings with the
> shipped defaults. Supplying those to the upgrade is precisely the failure this step exists to
> prevent.

Now transcribe the settings from that comparison into `my-values.yaml`, for example:

```yaml
rooturl: http://<your-domain>
storageClass: <your-storage-class>

oscreds:
  mysql:
    keycloak:
      password: "<your-keycloak-db-password>"
      adminpassword: "<your-keycloak-admin-password>"
```

> **Note:** Do not carry the image tags across at this point. They are handled in step 5.

Back up the databases from the MySQL StatefulSet, and snapshot the volume behind
`<release>-mysql-portal-claim0` as well if your storage class supports it:

```bash
kubectl exec -n openslice sts/openslice-mysql -- \
  sh -c 'mysqldump -u root -p"$MYSQL_ROOT_PASSWORD" --single-transaction --routines --events \
    --databases osdb ostmfdb metricodb keycloak' \
  > openslice-backup-$(date +%F).sql

ls -l openslice-backup-*.sql      # a few MB, never zero
```

> **Note:** `--single-transaction` matters because the services continue writing while the dump
> runs, and this file is what the rollback procedure depends on.

### 2. Download the new release

```bash
git fetch --all --tags
git checkout tags/2026Q2
git describe --tags               # 2026Q2
```

### 3. Refresh the Web UI configuration

The chart mounts `config.prod.json` and `config.theming.json`, the copies you made at install time.
The checkout cannot modify them, so they still carry the 2025Q4 version string, and each of them
places a version on screen: `PORTALVERSION` in the first, and a release name hardcoded inside
`FOOTER_HTML` in the second.

```bash
cd kubernetes/helm/openslice/files/org.etsi.osl.tmf.web/src/assets/config
diff config.prod.default.json    config.prod.json
diff config.theming.default.json config.theming.json
```

The new release changed one line in each: the version. Where a comparison shows only that line, copy
the new default over. Where it also shows edits of your own, keep your file and move the version
inside it instead, exactly as described in the Docker Compose section.

```bash
grep PORTALVERSION config.prod.json     # must read 2026Q2
cd ../../../../../../..
```

> **Note:** `theming.scss` and `files/org.etsi.osl.portal.web/src/js/config.js` are identical in both
> releases. `config.js` must nevertheless exist, otherwise `helm upgrade` fails with
> `config.js is required`.

Skipping this step does not prevent the upgrade from succeeding, but the portal footer continues to
name 2025Q4.

### 4. Reconcile your values file

Your values file sits outside the chart, so the upgrade does not touch it:

```bash
git diff tags/2025Q4 tags/2026Q2 -- kubernetes/helm/openslice/values.yaml
```

The release adds one key, and it carries a default:

```yaml
osom:
  secretsController:
    enable: false
```

`templates/osom.yaml` reads it as `.Values.osom.secretsController.enable | default "false"` and the
chart's own `values.yaml` supplies it, so a values file that never mentions it is perfectly valid.
Nothing is renamed or removed in this release.

> **Important Note:** `rooturl` matters most. Every service's Keycloak issuer URL is templated from
> it, as are the allowed origins, the Kroki URL and the portal-web `config.js`. The issuer mismatch
> that affects Docker Compose deployments cannot occur here as long as your values file still sets
> it.

### 5. Pin the image tags

The 2025Q4 chart ships `tag: "latest"` with `pullPolicy: Always` for all 13 OpenSlice images, and an
installation that followed the deployment guide never changed them. Unless you pinned them yourself,
that is the state of your deployment, and three consequences follow.

**You cannot determine which release is running.** `helm list` reports the chart's `appVersion`, not
the contents of the images, so it reports 2025Q4 regardless. `kubectl get deploy` shows only
`:latest`. The only reliable signal is the digest each pod resolved:

```bash
kubectl get pods -n openslice \
  -o jsonpath='{range .items[*]}{.status.containerStatuses[0].image}{"  "}{.status.containerStatuses[0].imageID}{"\n"}{end}' \
  | grep osl/code | sort -u
```

Identical digests across pods mean they were pulled together. Differing digests mean a pod restarted
later and collected a newer build than its neighbours.

**Part of the deployment may have upgraded itself already.** With `pullPolicy: Always`, every pod
restart re-pulls `latest`. A node reboot or an evicted pod after 2026Q2 was published moves that
service to 2026Q2 on its own, with no upgrade performed and no record in `helm history`.

**A rollback will not work.** `helm rollback` restores the previous manifests, but those manifests
also specify `latest`, so the pods re-pull the newest images and you return to where you started.
Until the tags are pinned, your database backup is the only means of return.

To pin them, add the following to your values file. You could also leave an empty string at the specified tags
as each template resolves its tag as `{{ .Values.image.<component>.tag | default .Chart.AppVersion }}`, so an 
empty string falls back to the chart's `appVersion`, which at 2026Q2 is `2026Q2`:

```yaml
image:
  bugzilla:   { tag: "2026Q2" }
  centrallog: { tag: "2026Q2" }
  manoclient: { tag: "2026Q2" }
  osom:       { tag: "2026Q2" }
  portalapi:  { tag: "2026Q2" }
  osscapi:    { tag: "2026Q2" }
  oasapi:     { tag: "2026Q2" }
  portalweb:  { tag: "2026Q2" }
  tmfweb:     { tag: "2026Q2" }
  cridge:     { tag: "2026Q2" }
  metrico:    { tag: "2026Q2" }
  mcpserver:  { tag: "2026Q2" }
  mcpbackend: { tag: "2026Q2" }
```

> **Note:** Leave `artemis`, `blockdiag`, `keycloak`, `kroki` and `mysql` unchanged. They carry
> explicit upstream versions, unchanged in this release, and do not follow the chart's appVersion.

This resolves all three problems permanently. From this upgrade onwards every pod resolves to the
chart's `appVersion`, so what is deployed is what you checked out, restarts are stable, and a
rollback returns the previous code rather than the same images again.

### 6. Render the chart before upgrading

A mistake caught at this point costs nothing to correct:

```bash
helm template openslice ./kubernetes/helm/openslice -n openslice -f my-values.yaml \
  | grep -oE "labs\.etsi\.org:5050/osl/code/[^\"]*" | sed 's/.*://' | sort | uniq -c
```

All 13 images must read `2026Q2`. A result of `latest` means the chart never received your values
file; `2025Q4` means the checkout did not take effect.

### 7. Deploy

```bash
helm upgrade openslice ./kubernetes/helm/openslice \
  -n openslice \
  -f my-values.yaml \
  --atomic --timeout 15m

kubectl -n openslice rollout status deploy --timeout=15m
kubectl -n openslice get pods
```

`--atomic` rolls the release back if the new pods never become ready. It has no effect on database
schema changes; your backup covers those.

Allow several minutes. The TMF API and Assurance API pods are the last to report ready, and Hibernate
applies schema changes as they start.

> **Note:** `rollout status` occasionally gives up on one of those pods shortly before it settles.
> Inspect the pod itself rather than reacting to that timeout.

Keycloak, MySQL and Artemis remain running throughout, because this release changes none of their
images, so nothing requires restarting afterwards. The two Compose steps for that have no equivalent
here: the `tmfweb` and `portalweb` images carry the front ends, and the ingress routes to Services
rather than to pod addresses.

### 8. Validate the upgrade

```bash
helm list -n openslice            # CHART openslice-4.1.0, APP VERSION 2026Q2

kubectl -n openslice get deploy \
  -o jsonpath='{range .items[*]}{.spec.template.spec.containers[0].image}{"\n"}{end}' \
  | grep osl/code | sed 's/.*://' | sort | uniq -c

kubectl exec -n openslice sts/openslice-mysql -- \
  sh -c 'mysql -u root -p"$MYSQL_ROOT_PASSWORD" -N -B \
    -e "select table_schema, count(*) from information_schema.tables \
        where table_schema in (\"osdb\",\"ostmfdb\",\"metricodb\",\"keycloak\") group by table_schema"'
```

Then open the portal, check the version in the footer, sign in, and confirm that your Service
Specifications and Product Offerings are listed.

> **Note:** A footer that still names 2025Q4 means step 3 was skipped. The upgrade itself is
> unaffected.

### Rolling back a Kubernetes deployment

```bash
helm rollback openslice -n openslice
```

Helm rolls back the manifests; your data remains as 2026Q2 left it. This returns you to 2025Q4 code
only if revision 1 carried pinned image tags. If you upgraded from an installation that used
`latest`, the restored manifests also specify `latest` and the pods re-pull 2026Q2, in which case the
database dump is your only means of return.

If Hibernate has already altered the schema, restore the dump first:

```bash
kubectl exec -i -n openslice sts/openslice-mysql -- \
  sh -c 'mysql -u root -p"$MYSQL_ROOT_PASSWORD"' < openslice-backup-YYYY-MM-DD.sql
helm rollback openslice -n openslice
```

> **Important Note:** A rollback moves the Helm release and nothing else. Your working copy remains
> on 2026Q2, where the next `helm upgrade` from that directory would quietly ship the new release
> again. Return the source tree as well:

```bash
git checkout tags/2025Q4
cd kubernetes/helm/openslice/files/org.etsi.osl.tmf.web/src/assets/config
cp config.prod.default.json    config.prod.json
cp config.theming.default.json config.theming.json
```

---

## Upgrading a Docker Compose deployment

> **Important Note:** Run every `docker compose` and `deploy.sh` command as the **same user that
> installed the deployment**, and never mix `sudo` with plain invocations. The MySQL service mounts
> `~/mysql/data`, and `~` expands for whoever runs the command, so switching to `sudo` silently
> points MySQL at an empty directory and it initialises a new, empty database. The deployment then
> appears healthy while your catalogue is empty. For the same reason, `deploy.sh` run under `sudo`
> leaves `org.etsi.osl.tmf.web/dist/` owned by `root`, and every later build you run as yourself
> fails to write there.

You can confirm at any time which directory is actually attached:

```bash
docker inspect amysql --format '{{range .Mounts}}{{if eq .Destination "/var/lib/mysql"}}{{.Source}}{{end}}{{end}}'
```

### 1. Prepare the environment

Back up the four databases and record the table counts, so that you have a baseline to compare
against after the upgrade:

```bash
docker exec amysql sh -c \
  'mysqldump -u root -p"$MYSQL_ROOT_PASSWORD" --databases osdb ostmfdb metricodb keycloak' \
  > ~/openslice-backup-$(date +%F).sql

docker exec amysql sh -c 'mysql -u root -p"$MYSQL_ROOT_PASSWORD" -N -B \
  -e "select table_schema, count(*) from information_schema.tables \
      where table_schema in (\"osdb\",\"ostmfdb\",\"metricodb\",\"keycloak\") group by table_schema"'
```

> **Note:** Both commands must stay inside `sh -c`. `MYSQL_ROOT_PASSWORD` exists inside the
> container, not in your shell, so without the single quotes your shell expands it to nothing and
> mysql fails with `Access denied for user 'root'@'localhost' (using password: NO)`.

> **Disclaimer:** The MySQL data folder is a bind mount, not a backup. `docker compose down -v`
> leaves it in place, so it is not protection against a schema migration.

### 2. Download the new release

`docker-compose.yaml` and `nginx/nginx.conf` are git-ignored and survive the checkout. Two files that
the 2025Q4 deployment guide asks you to edit are tracked by git, and a tracked file carrying your
edits can stop a checkout:

* `compose/mysql-init/01-databases.sql`, if you changed the `portaluser` or `keycloak` passwords.
* `compose/keycloak-init/realm-export.json`, if you seeded the realm from the file rather than
  through the Keycloak admin console.

```bash
cd org.etsi.osl.main
git status --porcelain       # a leading " M" marks a tracked file you changed
cp -n compose/mysql-init/01-databases.sql       ~/01-databases.sql.mine
cp -n compose/keycloak-init/realm-export.json   ~/realm-export.json.mine

git checkout tags/2026Q2
```

Neither file changed between 2025Q4 and 2026Q2, so the checkout succeeds and carries your edits
across. Detached HEAD is expected.

> **Note:** Keep the copies regardless. On a release that does modify one of these files the
> checkout aborts with `Your local changes would be overwritten by checkout`, and you would then
> restore the file with `git checkout -- <file>` and reapply your edit afterwards. `cp -n` is
> deliberate: without it, running this step twice would overwrite your saved copy with the file that
> was already restored.

> **Important Note:** Neither file is read at runtime once the deployment exists. `mysql-init` runs
> only against an empty data directory and Keycloak imports the realm only when it is absent from
> the database. However `01-databases.sql` must keep agreeing with `DB_PASSWORD` in
> `docker-compose.yaml`, or a future deployment built on a fresh volume will not start.

### 3. Reconcile the Docker Compose file

Compare what the new release changed against what you changed:

```bash
cd compose
git diff tags/2025Q4 tags/2026Q2 -- docker-compose.yaml.configure   # the new release
diff docker-compose.yaml.configure docker-compose.yaml              # your deployment
diff nginx/nginx.conf.default nginx/nginx.conf
```

The new release changes three lines, on the `osom` service:

```diff
-        "logging.level.org.etsi.osl.osom" : "INFO"
+        "logging.level.org.etsi.osl.osom" : "INFO",
+        "SECRETS_CONTROLLER_ENABLE": false
```

Your own half is considerably larger. A deployment that took every option offered by the 2025Q4
guide shows around 70 changed lines. Apply the three new lines to your file by hand, and never copy
the template over your own file.

```bash
cp docker-compose.yaml docker-compose.yaml.2025Q4.bak
# add the two osom lines by hand, then:
diff docker-compose.yaml.2025Q4.bak docker-compose.yaml
```

That comparison must show only the `osom` change. Then confirm that none of your own settings were
lost. The following must return `0`:

```bash
grep -c "keycloak:8080" docker-compose.yaml
```

### 4. Pin the image tags

The templates ship `:latest`. Left as they are, every release you build carries the same tag, you
cannot tell afterwards which release is deployed, and you cannot roll back to the images you had.

```bash
sed -i -E "s#(labs\.etsi\.org:5050/osl/code/org\.etsi\.osl\.[a-z.]+):[A-Za-z0-9_.]+#\1:2026Q2#" \
  docker-compose.yaml

grep -oE 'org\.etsi\.osl\.[a-z.]+:[A-Za-z0-9_.]+' docker-compose.yaml | sort -u
```

Eleven lines, all reading `2026Q2`.

> **Note:** The expression matches whatever tag is currently present, so it works whether you are
> coming from `:latest` or from a previous pin. An expression anchored on `:latest` would work once
> and then silently match nothing on every later upgrade.

The build in the next step tags what it produces with these names, so afterwards `docker ps` reports
the release you are running, and the 2025Q4 images remain on disk for a rollback. Older images are also available online by tag, so there is no compulsory need to keep them locally.

### 5. Build the new release

`deploy.sh` moves every `org.etsi.osl.*` checkout to 2026Q2 and rebuilds the jars and the TMF portal
bundle. Run it from the folder that holds the checkouts, not from inside one of them.

```bash
cd ..
wget -O deploy.sh \
  https://labs.etsi.org/rep/osl/code/org.etsi.osl.main/-/raw/2026Q2/compose/deploy.sh
chmod +x deploy.sh
sed -i 's/docker run -it --rm/docker run --rm/' deploy.sh
./deploy.sh 2026Q2
```

> **Note:** The `sed` is required if you are working over a non-interactive SSH session. `deploy.sh`
> invokes `docker run -it`, which aborts when no TTY is attached.

Two messages during the run look like failures and are not:

* `You are not currently on a branch` appears once per repository. The script runs `git checkout
  2026Q2` followed by `git pull`, and pulling on a detached HEAD is not possible. The checkout has
  already succeeded.
* The Maven build of each module takes several minutes on a small host.

The 2026Q2 script also clones `org.etsi.osl.mcp.backend`, which the 2025Q4 script did not. If your
2025Q4 installation needed that repository cloned by hand, the upgrade picks it up automatically.

Confirm that every checkout moved before building images:

```bash
for d in org.etsi.osl.*/; do
  printf "%-34s %s\n" "$(basename $d)" "$(cd "$d" && git describe --all)"
done
```

Fourteen lines, all reading `tags/2026Q2`.

> **Important Note:** A checkout that silently stayed behind is the one failure that survives to the
> end of the upgrade. The images would carry the 2026Q2 tag you pinned while containing 2025Q4 code,
> and every version check in this guide would still report 2026Q2.

### 6. Configure the Web UI

Nginx serves both portals from bind-mounted host folders, which `deploy.sh` has just rebuilt, so
there is nothing to copy. Two files under `org.etsi.osl.tmf.web/src/assets/config` carry a version
string, both are git-ignored, and nothing in the upgrade updates them: `deploy.sh` writes
`config.prod.json` only when it is missing, and never writes `config.theming.json` at all.

* `config.prod.json` holds `PORTALVERSION`.
* `config.theming.json` holds `FOOTER_HTML`, a block of raw HTML with the release name hardcoded
  inside the text. The footer renders that verbatim whenever the file exists. A deployment that
  skipped the optional branding step has no such file, and its footer falls back to `PORTALVERSION`.

The `.default` file beside each one is tracked by git and the checkout has already updated it, so
the 2026Q2 string is available. Compare them:

```bash
cd org.etsi.osl.tmf.web/src/assets/config
diff config.prod.default.json    config.prod.json
diff config.theming.default.json config.theming.json
```

The new release changed exactly one line in each: the version. Judge the two files separately.

**If a comparison shows only the version line**, that file is unmodified and you can take the new
default:

```bash
cp config.prod.default.json    config.prod.json      # only if its diff was version-only
cp config.theming.default.json config.theming.json   # only if its diff was version-only
```

**If a comparison also shows edits of your own**, keep your file and move the version inside it:

```bash
sed -i 's/"PORTALVERSION":"2025Q4"/"PORTALVERSION":"2026Q2"/' config.prod.json
sed -i 's/running OSL version 2025Q4/running OSL version 2026Q2/' config.theming.json
```

> **Important Note:** Do not run both. Copying the default and then applying the `sed` appears
> harmless, because the default already carries the new version, but the copy has already replaced
> your `TITLE`, `FOOTER_HTML` and any other branding with the shipped text, and the `sed` cannot
> restore it.

`org.etsi.osl.portal.web/src/js/config.js` needs no change between these two releases.

Nginx serves the `dist/` bundle, never `src/`, and only the Angular build regenerates it, so until
you run the build nothing you edited reaches the browser:

```bash
cd org.etsi.osl.tmf.web
stat -c '%n owner=%U' dist/org.etsi.osl.tmf.web
sudo chown -R $(id -un):$(id -gn) .          # only if the previous line reported root
docker run -u $(id -u) --rm -v "$PWD":/app trion/ng-cli:14.2.9 ng build --configuration production
grep -o '"PORTALVERSION":"[^"]*"' dist/org.etsi.osl.tmf.web/assets/config/config.prod.json
```

That last line must read `2026Q2` before you continue.

> **Note:** A `dist/` directory left owned by `root` by an earlier `sudo ./deploy.sh` is the usual
> reason a rebuild appears to do nothing at all. The build fails to write and the old bundle stays
> in place, which is indistinguishable from forgetting the step.

### 7. Deploy

```bash
cd org.etsi.osl.main/compose
docker compose --profile prod up -d
docker compose --profile prod restart
docker compose restart nginx
```


> **Important Note:** Restart everything, not only the backends. A source build normally leaves
> MySQL, Keycloak and Artemis untouched, in which case the restart costs a few seconds. But if
> anything has re-pulled the third-party images, Compose recreates the `amysql` container, and
> Keycloak and the Spring services keep JDBC connection pools pointing at the container that went
> away. The result is `401` responses from the token endpoint and a partially populated catalogue
> from the TMF API, with the data itself perfectly intact. That failure closely resembles data loss,
> which is why the restart is worth taking every time.

> **Note:** Restart nginx last and on its own. It resolves its upstream host names once, when it
> loads its configuration, so a restart issued while the backends are still coming up leaves it
> holding addresses that have already changed.

Allow several minutes before judging the result. The TMF API creates several hundred tables on its
first start with the new release and returns `502` for the whole of that time:

```bash
docker logs openslice-scapi 2>&1 | grep -E "Started .*Boot in|APPLICATION FAILED"
```

> **Note:** If Keycloak enters a restart loop during the upgrade, recreate it rather than restarting
> it. Every start then prints only `User with username 'admin' already added to ...
> keycloak-add-user.json` and exits, because that file lives in the container filesystem and
> survives restarts. `docker compose --profile prod up -d --force-recreate keycloak` clears it. This
> is safe: the realm lives in MySQL and is only imported when absent from the database.

### 8. Validate the upgrade

```bash
curl -s http://<your-domain>/assets/config/config.prod.json    | grep PORTALVERSION
curl -s http://<your-domain>/assets/config/config.theming.json | grep -o "running OSL version [A-Za-z0-9]*"

docker exec amysql sh -c 'mysql -u root -p"$MYSQL_ROOT_PASSWORD" -N -B \
  -e "select table_schema, count(*) from information_schema.tables \
      where table_schema in (\"osdb\",\"ostmfdb\",\"metricodb\",\"keycloak\") group by table_schema"'

curl -s -o /dev/null -w "%{http_code}\n" \
  -d client_id=osapiWebClientId -d client_secret=secret \
  -d username=<user> -d password=<password> -d grant_type=password \
  http://<your-domain>/auth/realms/openslice/protocol/openid-connect/token
```

Both version strings must read `2026Q2`, the table counts must match the baseline recorded in step 1,
and the token request must return `200`. Then sign in to the portal and confirm that your Service
Specifications and Product Offerings are listed.

> **Note:** Read the second `curl` even when the first one passes. `PORTALVERSION` is not what a
> branded portal displays in its footer, so checking it alone reports success while the page on
> screen still names 2025Q4.

Two results are worth interpreting rather than acting on directly:

* **Either version string still reads 2025Q4** while the backends are on 2026Q2: the Angular build
  did not take effect. Compare the served value with `src/assets/config/config.prod.json`. Identical
  means the source was never edited; different means the build did not run, or could not write to a
  root-owned `dist/`. The remedy is in step 6.
* **A `401`, or a catalogue that appears empty**: this is the recreated-MySQL behaviour described in
  step 7, not lost data. Restart the stack before investigating datasource configuration. An empty
  catalogue accompanied by healthy table counts instead indicates the `sudo` mount swap described at
  the start of this section.

### Rolling back a Docker Compose deployment

```bash
cd org.etsi.osl.main/compose
docker compose --profile prod down
cp docker-compose.yaml.2025Q4.bak docker-compose.yaml

# only if the schema has already changed
docker compose --profile prod up -d mysql-portal && sleep 60
docker exec -i amysql sh -c 'mysql -u root -p"$MYSQL_ROOT_PASSWORD"' \
  < ~/openslice-backup-YYYY-MM-DD.sql

docker compose --profile prod up -d          # no --build: reuse the 2025Q4 images
docker compose --profile prod restart
docker compose restart nginx
```

The 2025Q4 images remain on disk under whatever tag that release's Compose file named, and restoring
the backup file points at them again, so `up -d` without `--build` reuses them. On a first upgrade
from a stock installation that tag is `:latest`.

> **Note:** The Web UI is the exception. `dist/` now holds the 2026Q2 bundle, so to roll the portal
> back as well, run `./deploy.sh 2025Q4` again from the folder holding the checkouts.

---

## Common local modifications

These are the settings that the 2025Q4 deployment guide asks you to configure, together with those
that deployments commonly add themselves. On Kubernetes they live in your values file and pass
through the upgrade untouched. The list matters for Docker Compose, where step 3 asks you to merge
the new release's three lines into a file that holds all of them.

Unless stated otherwise, these are found in `docker-compose.yaml`.

| Modification | Symptom if lost |
| ------------ | --------------- |
| MySQL `portaluser` and `keycloak` passwords, in `compose/mysql-init/01-databases.sql` and `DB_PASSWORD` | Keycloak cannot reach its database on any deployment built from a fresh volume |
| `KEYCLOAK_PASSWORD` for the administration console | reverts to the shipped default on a fresh Keycloak database |
| Keycloak issuer changed from `http://keycloak:8080` to your own URL | every authenticated request returns `401` |
| `KEYCLOAK_FRONTEND_URL` on the `keycloak` service | the issuer follows the request `Host` header again |
| `bugzillaurl`, `bugzillakey`, `main_operations_product` | the Bugzilla connector silently stops working |
| `SPRING_AI_OLLAMA_*` and the system prompt on `oslmcpbackend` | the assistant addresses the wrong LLM |
| A kubeconfig placed in `compose/kubedir/` (untracked, survives the checkout) | CRIDGE loses access to its cluster |
| A pinned third-party image updated, for example Portainer | the container restarts continuously |
| Services removed from the `prod` profile, and their `location` blocks in `nginx.conf` | they start again and fail against dependencies you never deployed, or nginx refuses to start |
| `server_name` in `nginx.conf` | no effect while it remains the only server block |
| Host and API URLs in `org.etsi.osl.portal.web/src/js/config.js` | the portal addresses `localhost` from the browser |
| `TITLE` and branding in `config.prod.json`, `theming.scss`, `config.theming.json` | your branding reverts to the shipped text |
| The release name hardcoded in `config.theming.json`, inside `FOOTER_HTML` | the footer continues to name the release you branded on, whatever `PORTALVERSION` reports |
| Edits to the tracked `compose/keycloak-init/realm-export.json` | the checkout aborts on any release that changes that file |

`deploy.sh` leaves all of these untouched, so the only item this upgrade requires you to correct by
hand is the version string in the two Web UI configuration files.

### The Keycloak issuer on Docker Compose

Keycloak constructs the `iss` claim of a token from the `Host` header of the request, so a token
obtained through nginx carries your host name while the shipped template expects
`http://keycloak:8080`. The two disagree. Signing in still succeeds, and then every authenticated
request returns `401`.

Two changes are required, and losing either one breaks authentication:

1. Replace `http://keycloak:8080` with your deployment URL in `docker-compose.yaml`.
2. Set `KEYCLOAK_FRONTEND_URL` on the `keycloak` service.

> **Important Note:** Do not rewrite `proxy_pass http://keycloak:8080/auth` in `nginx.conf`. That
> occurrence is the internal proxy target on the Docker network and must remain a container name.

> **Note:** The backends read the OpenID configuration at start-up, so they must be able to reach
> that URL. On a cloud virtual machine behind NAT the host frequently cannot reach its own public
> address. Whatever you configured for this at installation time is host-level configuration that
> the upgrade leaves alone, so confirm it is still in place.

On Kubernetes the chart handles all of this from `rooturl`.
+17 −0
Changes for doc/getting_started/upgrade_guide/introduction.md: 17 added lines, 0 removed lines.
Original line number Diff line number Diff line
# Upgrade Guide

**Intended Audience: OpenSlice Administrators**

This section walks you through upgrading a running OpenSlice instance from one release to the next.

Guides are version-specific and assume you are moving between two adjacent releases. If your
deployment is more than one release behind, work through each guide in order rather than skipping
ahead.

- [2025Q4 to 2026Q2](./2025Q4_2026Q2.md)

## Before you begin

**Back up your databases.** The services run Hibernate with `ddl-auto: update`, so the schema
migrates itself on first start of the new release and nothing migrates it back. The backup commands
for each deployment method are the first step of their respective sections below.
 No newline at end of file
+3 −0
Changes for mkdocs.yml: 3 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -121,6 +121,9 @@ nav:
      - NFV Orchestrator Configuration: ./getting_started/configuration/nfvo_config.md
      - Advanced topics:
        - Consuming Services From External OSS: ./getting_started/configuration/consuming_services_from_external_partners.md
    - Upgrade guide: 
        - Introduction: ./getting_started/upgrade_guide/introduction.md
        - 2025Q4 -> 2026Q2: ./getting_started/upgrade_guide/2025Q4_2026Q2.md
  - NaaS:
    - Introduction: ./naas/introduction.md
    - Services:
Loading