diff --git a/doc/releasenotes.md b/doc/releasenotes.md index cfe9e5ea0abaaf9276396640003d56360c62524e..8aad1ad760bbd553fd673fadf634515461181929 100644 --- a/doc/releasenotes.md +++ b/doc/releasenotes.md @@ -7,6 +7,19 @@ ### **Technical Debt Solved** +#### **Certificate Generation Reworked for Multi-Instance Deployments (Common Vault)** + +The certificate generation and storage flow has been updated to support **multiple CAPIF instances sharing the same Vault**. +Previously, the startup and certificate provisioning logic assumed a single CAPIF instance and stored certificates from fixed Vault paths, which caused collisions when deploying more than one instance using a common Vault. + +With this change: + +- Certificates and keys are now **generated at the service level** (e.g., NGINX generates it's own key and CSR locally). +- Vault is now used only as a **signing authority (CA)** to sign incoming CSRs and to store the resulting artifacts, avoiding Vault-specific instance coupling and enabling the same Vault to serve other CAPIF deployments. +- A **unique CCF identifier (ccf_id)** is used as the namespace key to store and retrieve CAPIF certificates. +- CAPIF-related certificates are stored under instance-scoped Vault paths (e.g. `secret/capif//...`) to prevent overwriting assets across deployments. +- Startup scripts and tooling were updated to obtain and use the correct `ccf_id` dynamically, ensuring each instance loads the correct certificate material. + #### **NGINX Configuration improved** The NGINX configuration included in the OpenCAPIF deployment has been improved. diff --git a/doc/vault/vault.md b/doc/vault/vault.md new file mode 100644 index 0000000000000000000000000000000000000000..7b2db5221f5c2a60888a78eaa34cb1f964ff7d28 --- /dev/null +++ b/doc/vault/vault.md @@ -0,0 +1,317 @@ +# CAPIF Certificate Generation Architecture + +## Summary + +This document describes the changes made to the certificate generation and management architecture in CAPIF, implemented in the `OCF182-certs-generation` branch. The main objective is to allow a single Vault server to serve multiple CAPIF instances efficiently and securely. + +## Main Changes + +### Previous Architecture + +In the previous implementation, Vault was responsible for: + +- Generating the root CA and intermediate CA +- **Generating** service certificates +- Storing and distributing certificates + +**Problems:** + +- Vault generated service certificates (less secure) +- Difficult scalability for multiple CAPIF instances +- Strong coupling between Vault and each CAPIF instance + +### New Architecture + +In the new implementation: + +``` +┌──────────────────────────────────────────────────────────────┐ +│ VAULT │ +│ ┌────────────────────────────────────────────────────────┐ │ +│ │ 1. Generates Root CA │ │ +│ │ 2. Generates Intermediate CA │ │ +│ │ 3. Configures signing role │ │ +│ │ 4. Stores CA bundle in secret/ca │ │ +│ └────────────────────────────────────────────────────────┘ │ +│ │ +│ Functions: │ +│ ✓ SIGNS certificates (receives CSR) │ +│ ✓ STORES signed certificates │ +└──────────────────────────────────────────────────────────────┘ + ▲ + │ + ┌────────────────────┼────────────────────┐ + │ │ │ + ▼ ▼ ▼ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ CAPIF #1 │ │ CAPIF #2 │ │ CAPIF #N │ +│ CCF_ID: A │ │ CCF_ID: B │ │ CCF_ID: C │ +└──────────────┘ └──────────────┘ └──────────────┘ +``` + +**Benefits:** + +- ✅ Enhanced security: each service generates its own private key +- ✅ Scalability: one Vault serves multiple CAPIF instances +- ✅ Isolation: certificates organized by CCF_ID +- ✅ PKI best practices: separation between generation and signing + +## Modified Components + +### 1. Vault (`services/vault/vault_prepare_certs.sh`) + +**Main changes:** + +- No longer **generates** service certificates, only PKI infrastructure +- Generates only root CA and intermediate CA +- Configures PKI signing endpoint at `/v1/pki_int/sign/my-ca` +- Stores CA bundle in `secret/ca` (KV v2) +- Configures `my-ca` signing role with flexible policies that allow signing any CSR + +### 2. NGINX (`services/nginx/nginx_prepare.sh`) + +**New primary certificate generation component:** + +Implemented flow: + +1. Fetches CA certificate from Vault (`secret/ca`) +2. Generates its own private key (`server.key`) if it doesn't exist +3. Creates a CSR (Certificate Signing Request) with CAPIF hostname +4. Sends CSR to Vault for signing +5. Receives signed certificate (`server.crt`) +6. Extracts public key from certificate (`server_pub.pem`) +7. Obtains unique CCF_ID from Helper +8. Stores all certificates in Vault under `secret/capif/${CCF_ID}/nginx` + +**Important note:** This script includes retries with timeout to handle ordered service startups. + +### 3. Register (`services/register/register_prepare.sh`) + +**Similar pattern to NGINX:** + +- Generates its own private key (`register_key.key`) +- Creates CSR with complete organization information +- Obtains CA from Vault +- Requests certificate signing from Vault via PKI endpoint +- Saves signed certificate locally + +### 4. Helper Service (`services/helper/helper_service/app.py`) + +**Modifications at application startup:** + +- Creates a dedicated `certs/` directory with restrictive permissions +- Generates key pair and CSR for superadmin certificate using pyOpenSSL +- Saves private key locally with 600 permissions +- Requests certificate signing from Vault +- Downloads and stores signed superadmin certificate +- Downloads and saves CA root from Vault + +**Security improvements:** permission validation and robust error handling. + +### 5. API Services (Invoker, Provider, Security) + +**Modified `prepare_*.sh` scripts:** + +New consumption pattern (no longer generate certificates): + +1. Query Helper to obtain CAPIF instance's CCF_ID +2. Retrieve server's public key from Vault using path `secret/capif/${CCF_ID}/nginx` +3. Save public key to local location +4. Implement robust retries to wait for NGINX to generate and store certificates + +**Affected files:** + +- `prepare_invoker.sh` +- `prepare_provider.sh` +- `prepare_security.sh` + +## Certificate Organization in Vault + +### KV (Key-Value v2) Structure + +**Root level:** + +- `secret/ca`: Stores intermediate CA certificate (accessible by all CAPIF instances) + +**Per-CAPIF level:** + +- `secret/capif//nginx/`: Contains certificates for each CAPIF instance + - `server_crt`: Server certificate + - `server_key`: Server private key + - `server_pub`: Extracted public key + - `ca`: Copy of CA bundle + +**Note:** In the future, sub-levels for `register/` and `helper/` will be added under each CCF_ID. + +### PKI Endpoints + +**Root PKI Engine (`pki/`):** + +- `root/generate/internal`: Generates root CA +- `config/urls`: Configures issuance and CRL URLs +- `root/sign-intermediate`: Signs intermediate CA + +**Intermediate PKI Engine (`pki_int/`):** + +- `intermediate/generate/internal`: Generates intermediate CA CSR +- `intermediate/set-signed`: Installs signed intermediate certificate +- `roles/my-ca`: Defines signing role with its policies +- `sign/my-ca`: Endpoint used by services to sign their CSRs + +## Deployment Flow + +### 1. Vault Initialization + +**Execute once per Vault cluster:** + +- In Docker: run `vault_prepare_certs.sh` script +- In Kubernetes: apply ConfigMap and Job from `helm/vault-job/vault-job.yaml` + +**Expected result:** + +- ✅ Root CA generated +- ✅ Intermediate CA generated and signed +- ✅ `my-ca` role configured +- ✅ CA bundle stored in `secret/ca` + +### 2. CAPIF Instance Deployment + +Each CAPIF instance (identified by unique CCF_ID): + +1. **Helper starts** → Generates unique CCF_ID +2. **NGINX starts:** + + - Generates `server.key` and `server.csr` + - Requests signing from Vault + - Receives `server.crt` + - Stores in `secret/capif/${CCF_ID}/nginx` +3. **Register starts:** + + - Generates registration certificate + - Requests signing from Vault +4. **API Services start:** + + - Retrieve certificates from Vault using CCF_ID + - Use public key for validation + +## Advantages of New Architecture + +### Security + +1. **Private keys never transit**: each service generates its private key locally +2. **Principle of least privilege**: Vault only has CA role, doesn't manage service keys +3. **Isolation per CAPIF**: certificates separated by CCF_ID + +### Scalability + +1. **One Vault, multiple CAPIF**: total decoupling +2. **Namespace per instance**: `secret/capif/${CCF_ID}/` +3. **No collisions**: unique CCF_ID guarantees separation + +### Operations + +1. **Simplified rotation**: each CAPIF can regenerate its certificates independently +2. **Improved auditing**: can track which CAPIF requested which certificate +3. **Independent deployment**: a CAPIF can restart without affecting others + +## Helm/Kubernetes Compatibility + +The `helm/vault-job/vault-job.yaml` file has been significantly simplified: + +**Main changes:** + +- ❌ No longer generates certificates for specific services +- ✅ Only executes basic PKI setup (Root CA, Intermediate CA, signing role) +- ✅ Lighter: 144 lines removed +- ✅ ConfigMap contains a self-contained script that enables PKI engines, generates CAs and configures signing role +- ✅ Job runs only once when deploying Vault namespace + +## Docker Compose Configuration + +**Changes in `services/docker-compose-capif.yml`:** + +**NGINX:** + +- No longer mounts external certificate volumes +- Certificates are generated dynamically in `/etc/nginx/certs` at startup +- New environment variables: `VAULT_HOSTNAME`, `VAULT_PORT`, `VAULT_ACCESS_TOKEN`, `CAPIF_HOSTNAME` + +**Helper:** + +- Manages its own certificate directory internally +- `certs/` directory is automatically created by `app.py` at startup +- No longer requires external volumes for certificates + +## Migration from Previous Architecture + +### Migration steps: + +1. **Backup existing certificates**: Export current certificates from Vault (optional, for history) +2. **Deploy new Vault**: Execute updated `vault_prepare_certs.sh` script +3. **Clean old certificates**: Remove certificate directories in NGINX and Register services +4. **Redeploy CAPIF services**: Execute `run.sh` so services automatically generate their new certificates + +### Important considerations: + +- ⚠️ **Downtime**: There will be a service interruption during migration +- ⚠️ **New CCF_ID**: Each instance will get a new unique CCF_ID +- ⚠️ **Regenerated certificates**: All service certificates will be new +- ✅ **No API changes**: CAPIF's public interface remains the same +- ✅ **Compatibility**: Existing clients will continue working after obtaining new CA + +## Testing + +**Recommended verifications:** + +1. **Verify CA in Vault**: Query `secret/data/ca` with curl to confirm CA bundle is available +2. **Start CAPIF**: Execute startup script and observe logs from each service +3. **Verify NGINX certificate**: Use `openssl x509` inside container to inspect generated certificate +4. **Verify storage in Vault**: Obtain CCF_ID from Helper and query path `secret/capif/${CCF_ID}/nginx` in Vault +5. **Run test suite**: Run `run_capif_tests.sh` to validate complete functionality + +## Troubleshooting + +### Error: "Unable to retrieve CA certificate from Vault" + +**Cause:** Vault has not been properly initialized with PKI infrastructure + +**Solution:** Execute Vault preparation script (`vault_prepare_certs.sh`) inside container + +### Error: "Failed to sign server certificate" + +**Cause:** `my-ca` signing role is not configured or Vault token doesn't have sufficient permissions + +**Solution:** Verify role exists and token has write permissions on `pki_int/sign/my-ca` + +### Error: "Unable to retrieve CCF_ID from Helper" + +**Cause:** Helper service hasn't completed initialization yet + +**Solution:** Wait 30-60 seconds. Service scripts implement automatic retries with incremental delays + +## References + +### Modified Files (Branch OCF182-certs-generation) + +- [`services/vault/vault_prepare_certs.sh`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/vault/vault_prepare_certs.sh) - PKI Setup +- [`services/nginx/nginx_prepare.sh`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/nginx/nginx_prepare.sh) - NGINX certs generation +- [`services/register/register_prepare.sh`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/register/register_prepare.sh) - Register certs generation +- [`services/helper/helper_service/app.py`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/helper/helper_service/app.py) - Superadmin cert generation +- [`services/TS29222_CAPIF_API_Invoker_Management_API/prepare_invoker.sh`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/TS29222_CAPIF_API_Invoker_Management_API/prepare_invoker.sh) - Cert consumption +- [`services/TS29222_CAPIF_API_Provider_Management_API/prepare_provider.sh`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/TS29222_CAPIF_API_Provider_Management_API/prepare_provider.sh) - Cert consumption +- [`services/TS29222_CAPIF_Security_API/prepare_security.sh`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/TS29222_CAPIF_Security_API/prepare_security.sh) - Cert consumption +- [`helm/vault-job/vault-job.yaml`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/helm/vault-job/vault-job.yaml) - Kubernetes Job +- [`services/docker-compose-capif.yml`](https://labs.etsi.org/rep/ocf/capif/-/blob/staging/services/docker-compose-capif.yml) - Docker Configuration + +### Related Documentation + +- [HashiCorp Vault PKI Secrets Engine](https://developer.hashicorp.com/vault/docs/secrets/pki) +- [OpenSSL CSR Generation](https://www.openssl.org/docs/man1.1.1/man1/openssl-req.html) +- [CAPIF Specification TS 29.222](https://www.3gpp.org/DynaReport/29222.htm) + +--- + +**Document Version**: 1.0 +**Date**: February 2026 +**Branch**: OCF182-certs-generation +**Author**: CAPIF Team diff --git a/mkdocs.yml b/mkdocs.yml index aaf20b7a0f279aa9ffd0a422bacef72d4e31097c..44fd63df92b39eced724145fef05b5616233fd89 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -98,6 +98,7 @@ nav: - Event Filter: ./event-filter/event-filter.md - Dynamic Configuration: ./configuration/configuration.md - Event Reporting Information: ./event-req/event-req.md + - Vault Certificate Management: ./vault/vault.md - SDK: - Introduction: ./sdk/sdk_introduction.md - Requirements: ./sdk/sdk_requirements.md