Commit 505e2bd0 authored by Sergio Gimenez's avatar Sergio Gimenez
Browse files

Merge branch 'feat/r2-existing-cluster' into 'master'

feat(r2): deploy, federate and test OOP on k3s

See merge request !3
parents 04df7a73 e6bdbd90
Loading
Loading
Loading
Loading
Loading
+2 −0
Changes for README.md: 2 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -16,6 +16,7 @@ This repository contains the public Ansible automation for the ETSI OOP stack.

- `docs/getting-started.md`: environment, inventory, secrets, first deployment
- `docs/deployment.md`: working deployment, undeploy, verification, smoke test flow
- `ansible/playbooks/scenarios/r2_existing_cluster/README.md`: Release 2 on your own clusters (k3s, Helm chart, federation, e2e tests)

## Quick start

@@ -40,6 +41,7 @@ deploying. Set `lite2edge_image` as well when using the `lite2edge` profile.

- `playbooks/scenarios/dual_oop/deploy.yml`: two complete Operator Platforms on two hosts
- `playbooks/scenarios/full_oop/deploy.yml`: one complete Operator Platform on one host
- `playbooks/scenarios/r2_existing_cluster/`: Release 2, one platform per operator on its own cluster, federated and tested; see its `README.md`

## Kubeconfigs

+124 −0
Changes for ansible/inventory/r2.example.yml: 124 added lines, 0 removed lines.
Original line number Diff line number Diff line
---
# Example inventory for playbooks/scenarios/r2_existing_cluster: one host per operator.
# Copy to inventory/r2.local.yml (gitignored) and adapt. Secrets go in Ansible Vault, e.g.
# inventory/host_vars/<host>/vault.yml created with `ansible-vault create`.
# What each variable does: playbooks/scenarios/r2_existing_cluster/README.md.

all:
  children:
    oop_platform_operators:
      hosts:
        # Helm and kubectl run here against the kubeconfig. A remote host works the same
        # (ansible_host, ansible_user, ...) when the kubeconfig lives there.
        operator_a:
          ansible_connection: local
          ansible_python_interpreter: "{{ ansible_playbook_python }}"
          oop_platform_kubeconfig: ~/.kube/operator-a.yaml
          oop_platform_public_host: 192.0.2.10
          oop_platform_federation_id: operator-a
          oop_platform_country_code: ES
          oop_platform_mcc: "214"
          oop_platform_mncs: ["07"]
          oop_platform_federation_clients: [operator-b]
          srm_zone_name: Barcelona

        operator_b:
          ansible_connection: local
          ansible_python_interpreter: "{{ ansible_playbook_python }}"
          oop_platform_kubeconfig: ~/.kube/operator-b.yaml
          oop_platform_public_host: 192.0.2.20
          oop_platform_federation_id: operator-b
          oop_platform_country_code: ES
          oop_platform_mcc: "214"
          oop_platform_mncs: ["01"]
          oop_platform_federation_clients: [operator-a]
          srm_zone_name: Madrid

      vars:
        # Published CI builds of each develop branch. Bump them together.
        oop_platform_fm_image_tag: ci-develop-8b791159
        oop_platform_oeg_image_tag: ci-develop-469e1962
        oop_platform_srm_image_tag: ci-develop-3200f256
        # No Release 2 portal image is published yet: build one, or turn the portal off.
        oop_platform_portal_enabled: false
        # The Release 2 charts are on this branch until oop/deploy/helm!15 merges;
        # helm develop still has the Release 1 (MongoDB) charts.
        oop_platform_chart_version: refactor/helm-2.0
        # Plain HTTP between the operators: test networks only.
        oop_platform_allow_insecure_partner_endpoints: true

        # What every partner in this inventory may use here (federate.yml).
        oop_platform_partner_agreement:
          zones: ["{{ srm_zone_name }}"]
          terms:
            reservedComputeResources:
              - {cpuArchType: ISA_X86_64, numCPU: "2", memory: 4096}
            computeResourceQuotaLimits:
              - {cpuArchType: ISA_X86_64, numCPU: "4", memory: 8192}
            flavoursSupported:
              - flavourId: small
                cpuArchType: ISA_X86_64
                supportedOSTypes:
                  - {architecture: x86_64, distribution: UBUNTU, version: OS_VERSION_UBUNTU_2204_LTS, license: OS_LICENSE_TYPE_FREE}
                numCPU: 1
                memorySize: 1024
                storageSize: 10

        # SRM has no zone until it is given one: here, one zone per operator, with apps
        # deployed by Helm into namespace oop-apps of the operator's own cluster.
        oop_platform_extra_values:
          srm:
            srmcontroller:
              workloads:
                namespaces: [oop-apps]
              controlPaths:
                - control_path_ref: k8s-local
                  adapter: kubernetes
                  config:
                    namespace: oop-apps
              topology: |
                version: openop.srm.bootstrap/v1
                platform_instance_id: {{ oop_platform_federation_id }}
                control_paths:
                  - ref: k8s-local
                    adapter_family: compute
                    adapter_type: kubernetes
                    adapter_contract_version: v1
                    infrastructure:
                      namespace: oop-apps
                zones:
                  - ref: {{ oop_platform_federation_id }}-zone
                    name: {{ srm_zone_name }}
                    kind: resource
                    state: active
                    domains:
                      - ref: {{ oop_platform_federation_id }}-compute
                        name: {{ srm_zone_name }} compute
                        kind: compute
                        state: active
                        capabilities:
                          - ref: deploy-workload
                            name: Deploy Workload
                            kind: workload
                            state: active
                            control_path_bindings:
                              - ref: deploy-workload-via-k8s-local
                                control_path_ref: k8s-local
                                is_default: true
                                state: active

        # In the vault, per host (never in plain text):
        # oop_platform_fm_partner_secret_key: <32 bytes, base64url>
        # oop_platform_oeg_callback_secret_key: <32 bytes, base64url>
        # oop_platform_postgres_password: ...
        # oop_platform_db_passwords: {keycloak: ..., portal: ..., oeg: ..., srm: ..., fm: ...}
        # oop_platform_keycloak_admin_password: ...

    # Plain Ubuntu VMs: playbooks/infrastructure/k3s/deploy.yml installs k3s first. A host
    # in both groups is the k3s server and the operator host (use ansible_host, not local).
    # k3s_nodes:
    #   hosts:
    #     operator_a:
    #     operator_a_agent:
    #       ansible_host: 192.0.2.11
    #       k3s_server: operator_a
+13 −0
Changes for ansible/playbooks/infrastructure/k3s/deploy.yml: 13 added lines, 0 removed lines.
Original line number Diff line number Diff line
---
# Playbook: install k3s
# Description: k3s on each host in k3s_nodes: a server, or an agent joining the server
#   named in k3s_server. Servers also get Helm and the Python Kubernetes client, ready for
#   playbooks/scenarios/r2_existing_cluster. A host that already runs k3s keeps its
#   cluster as it is.
# Usage: ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/deploy.yml

- name: Install k3s
  hosts: k3s_nodes
  gather_facts: true
  roles:
    - role: k3s
+23 −0
Changes for ansible/playbooks/infrastructure/k3s/undeploy.yml: 23 added lines, 0 removed lines.
Original line number Diff line number Diff line
---
# Playbook: remove k3s
# Description: Runs the k3s uninstaller on each host in k3s_nodes. That deletes every
#   workload and volume in the cluster, whoever installed them, so it needs
#   -e k3s_confirm_uninstall=true.
# Usage: ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/undeploy.yml \
#          --limit <host> -e k3s_confirm_uninstall=true

- name: Remove k3s
  hosts: k3s_nodes
  gather_facts: true
  tasks:
    - name: Refuse without confirmation
      ansible.builtin.assert:
        that: k3s_confirm_uninstall | default(false) | bool
        fail_msg: >-
          This deletes the whole k3s cluster on {{ inventory_hostname }}, every workload and
          volume included. Re-run with -e k3s_confirm_uninstall=true.

    - name: Uninstall k3s
      ansible.builtin.include_role:
        name: k3s
        tasks_from: uninstall
+17 −0
Changes for ansible/playbooks/scenarios/README.md: 17 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -35,6 +35,23 @@ One complete Operator Platform on a host that already has a running cluster.
ansible-playbook playbooks/scenarios/full_oop_existing_cluster/deploy.yml --limit example_existing_cluster
```

### `r2_existing_cluster`

Release 2 (PostgreSQL, NATS, Keycloak realms, portal) on clusters you already run, or on
plain Ubuntu VMs after `playbooks/infrastructure/k3s/deploy.yml`. Each host in
`oop_platform_operators` is one operator with its own cluster. `deploy.yml` installs the
`oop-platform` Helm chart, `federate.yml` registers the operators with each other and
federates them, and `e2e.yml` runs `tests/r2_e2e`. What each step sets up, the variables
and the secrets are in [`r2_existing_cluster/README.md`](r2_existing_cluster/README.md).

```bash
ansible-playbook -i inventory/r2.local.yml playbooks/infrastructure/k3s/deploy.yml   # plain VMs only
ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/deploy.yml
ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/federate.yml
ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/e2e.yml
ansible-playbook -i inventory/r2.local.yml playbooks/scenarios/r2_existing_cluster/undeploy.yml
```

## Undeploy

Each scenario provides:
Loading