feat(r2): deploy, federate and test OOP Release 2 on existing or fresh k3s clusters

---

# Example inventory for playbooks/scenarios/r2_existing_cluster: one host per operator.

# Copy to inventory/r2.local.yml (gitignored) and adapt. Secrets go in Ansible Vault, e.g.

# inventory/host_vars/<host>/vault.yml created with `ansible-vault create`.

all:

  children:

    oop_platform_operators:

      hosts:

        # Helm and kubectl run here against the kubeconfig. A remote host works the same

        # (ansible_host, ansible_user, ...) when the kubeconfig lives there.

        operator_a:

          ansible_connection: local

          ansible_python_interpreter: "{{ ansible_playbook_python }}"

          oop_platform_kubeconfig: ~/.kube/operator-a.yaml

          oop_platform_public_host: 192.0.2.10

          oop_platform_federation_id: operator-a

          oop_platform_country_code: ES

          oop_platform_mcc: "214"

          oop_platform_mncs: ["07"]

          oop_platform_federation_clients: [operator-b]

        operator_b:

          ansible_connection: local

          ansible_python_interpreter: "{{ ansible_playbook_python }}"

          oop_platform_kubeconfig: ~/.kube/operator-b.yaml

          oop_platform_public_host: 192.0.2.20

          oop_platform_federation_id: operator-b

          oop_platform_country_code: ES

          oop_platform_mcc: "214"

          oop_platform_mncs: ["01"]

          oop_platform_federation_clients: [operator-a]

      vars:

        # Published CI builds of each develop branch. Bump them together.

        oop_platform_fm_image_tag: ci-develop-8b791159

        oop_platform_oeg_image_tag: ci-develop-469e1962

        oop_platform_srm_image_tag: ci-develop-3200f256

        # No Release 2 portal image is published yet: build one, or turn the portal off.

        oop_platform_portal_enabled: false

        # The R2 charts are not on develop yet.

        oop_platform_chart_version: develop

        # In the vault, per host (never in plain text):

        # oop_platform_fm_partner_secret_key: <32 bytes, base64url>

        # oop_platform_oeg_callback_secret_key: <32 bytes, base64url>

        # oop_platform_postgres_password: ...

        # oop_platform_db_passwords: {keycloak: ..., portal: ..., oeg: ..., srm: ..., fm: ...}

        # oop_platform_keycloak_admin_password: ...

---

# Playbook: install k3s

# Description: A single-node k3s cluster on each host in k3s_nodes, plus Helm and the

#   Python Kubernetes client, ready for playbooks/scenarios/r2_existing_cluster. A host

#   that already runs k3s keeps its cluster as it is.

# Usage: ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/deploy.yml

- name: Install k3s

  hosts: k3s_nodes

  gather_facts: true

  roles:

    - role: k3s

---

# Playbook: remove k3s

# Description: Runs the k3s uninstaller on each host in k3s_nodes. That deletes every

#   workload and volume in the cluster, whoever installed them, so it needs

#   -e k3s_confirm_uninstall=true.

# Usage: ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/undeploy.yml \

#          --limit <host> -e k3s_confirm_uninstall=true

- name: Remove k3s

  hosts: k3s_nodes

  gather_facts: true

  tasks:

    - name: Refuse without confirmation

      ansible.builtin.assert:

        that: k3s_confirm_uninstall | default(false) | bool

        fail_msg: >-

          This deletes the whole k3s cluster on {{ inventory_hostname }}, every workload and

          volume included. Re-run with -e k3s_confirm_uninstall=true.

    - name: Uninstall k3s

      ansible.builtin.include_role:

        name: k3s

        tasks_from: uninstall

ansible-playbook playbooks/scenarios/full_oop_existing_cluster/deploy.yml --limit example_existing_cluster

```

### `r2_existing_cluster`

Release 2 (Postgres, NATS, Keycloak realms, portal) on a cluster you already run. Each

host in `oop_platform_operators` is one operator: the `oop-platform` role installs the

`oop-platform` helm chart from `oop/deploy/helm` against that host's kubeconfig, then

checks Keycloak's issuers, OEG and FM through the public address. See

`inventory/r2.example.yml` for the variables each operator needs.

```bash

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/deploy.yml

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/undeploy.yml

```

A plain Ubuntu VM gets its cluster first from `playbooks/infrastructure/k3s/deploy.yml`.

It installs single-node k3s at a pinned version (`k3s_version`), Helm and the Python

Kubernetes client, and gives the SSH user a `~/.kube/config`. Put the hosts in a

`k3s_nodes` group. A host that already runs k3s keeps its cluster, version and flags as

they are. `k3s/undeploy.yml` removes the whole cluster with everything in it, so it asks

for `-e k3s_confirm_uninstall=true`.

```bash

ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/deploy.yml

```

The chart generates FM's and OEG's encryption keys, the database passwords and the

Keycloak admin password on first install, and keeps them in Secrets in the namespace.

That's fine until the cluster dies, at which point every partner secret FM stored becomes

unreadable. For anything you can't lose, put your own values in Ansible Vault

(`oop_platform_fm_partner_secret_key`, `oop_platform_oeg_callback_secret_key`,

`oop_platform_db_passwords`, ...). The role refuses to swap a key the cluster already

uses, because the data it encrypted would be lost; it prints the command to read the key

in use so you can move that one into the vault instead.

`undeploy.yml` leaves those Secrets and the database volume in place on purpose, so the

next deploy picks up the same keys and data.

Once the operators are up, federate them and run the end-to-end tests:

```bash

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/federate.yml

pip install -r tests/r2_e2e/requirements.txt

ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/e2e.yml

```

`federate.yml` registers each operator in every other one it shares federation clients

with, then establishes the federation both ways. It reads the client secrets from the

partner's `keycloak-credentials` Secret, so it needs access to all the clusters; a partner

you can't reach that way goes in `oop_platform_partners` with its secret from the vault

(see the role defaults). It talks to Federation Manager's internal API through

`kubectl exec` in the FM pod and passes the secrets on stdin. Running it again changes

nothing.

`e2e.yml` runs `tests/r2_e2e` against the inventory: Keycloak issuers, OEG and EWBI

authentication, the federation lifecycle across two operators (including a Federation

Manager restart), and app registration. The tests create throwaway Keycloak clients in

the `oop` realm and delete them afterwards. Known gaps are marked `xfail(strict=True)`

with the reason, so they show up in the report and fail the run as soon as they're

fixed, as a prompt to remove the mark. Pass pytest arguments with

`-e e2e_pytest_args='-k federation'`.

## Undeploy

Each scenario provides:

---

# Playbook: OOP Release 2 on an existing Kubernetes cluster

# Description: Installs or upgrades the oop-platform helm chart for each operator in

#   oop_platform_operators, then checks it through its public addresses.

# Usage: ansible-playbook -i inventory/hosts.local.yml playbooks/scenarios/r2_existing_cluster/deploy.yml

- name: Deploy OOP Release 2 on an existing cluster

  hosts: oop_platform_operators

  gather_facts: true

  roles:

    - role: oop-platform

      vars:

        oop_platform_state: present