Commit b9e1dacc authored by Sergio Gimenez's avatar Sergio Gimenez
Browse files

docs: align FM persistence model

parent c78a13b9
Loading
Loading
Loading
Loading
+68 −29
Changes for fm/persistence-model.md: 68 added lines, 29 removed lines.
Original line number Diff line number Diff line
@@ -44,7 +44,8 @@ Keep one distinction in mind, the FM analogue of OEG's:
1. **Who our federation partners are** — partner profiles, the OAuth2 client identity that maps an
   inbound token back to a partner, the credentials we use to obtain tokens for outbound calls, and
   connection status.
2. **What each partnership is allowed to do** — federation agreements (permitted API types, zones,
2. **How each directed federation is identified and what it is allowed to do** — federation
   contexts, agreements (permitted API types, zones,
   usage limits, validity) and the partner-app → local `service_specification_id` mappings
   (ADR-0018) that drive SRM dispatch, plus the routing rules that resolve identifiers to partners.
3. **What has crossed the boundary** — the audit record of every inbound and outbound EWBI
@@ -68,6 +69,11 @@ types, permitted zones, usage limits, validity window, and — crucially — the
that resolve a partner-facing `appId` / API family to a local `service_specification_id` (ADR-0018).
It is the resolution authority for SRM dispatch. Stored in `federation_agreements`.

**FederationContext** — one directed OPG.04 federation relationship. FM issues the opaque
`federationContextId` for inbound federation; the partner issues it for outbound federation. The
context stores direction, lifecycle status, governing agreement, and the partner status callback
URL. Stored in `federation_contexts`.

**RoutingRule** — a mapping from an identifier range (`msisdn_prefix`, `ip_cidr`) to a partner OP,
used for outbound federation routing. Stored in `routing_rules`.

@@ -76,19 +82,21 @@ partner; lets FM validate that a federated deployment targets an authorized zone
`shared_resource_catalogues`.

**FederationTransaction** — the durable audit record of one EWBI interaction (inbound or outbound).
Carries the opaque partner-facing `federation_operation_id`, the internal `operation_id` (SRM
correlation), the internal `correlation_id`, the external `federation_correlation_id`, status, and
sanitized request/response summaries. Written `pending` **before** any DataBus publish; the async
retry and SRM-correlation anchor. Stored in `federation_transactions`.
Carries the directed context, optional federation-side and SRM operation handles, idempotency data,
status, and sanitized request/response summaries. For asynchronous commands it is written `pending`
**before** DataBus publish; synchronous queries are recorded after the HTTP round-trip. Stored in
`federation_transactions`.

### Relationships

```text
partner_ops            1 ──── * federation_agreements
partner_ops            1 ──── * federation_contexts
partner_ops            1 ──── * routing_rules
partner_ops            1 ──── * shared_resource_catalogues
partner_ops            1 ──── * federation_transactions
federation_agreements  1 ──── * federation_transactions
federation_contexts    1 ──── * federation_transactions
```

### Entity ownership (stores vs. references)
@@ -99,14 +107,15 @@ anything owned by SRM is held only as a UUID reference.
| # | Entity | Table | FM owns? | Rationale |
|---|---|---|---|---|
| 1 | Partner OP profiles | `partner_ops` | ✅ Yes | Identity + OAuth2 client credentials + connection status for each federated operator. |
| 2 | Federation agreements | `federation_agreements` | ✅ Yes | Permissions + `appId → service_specification_id` mappings (ADR-0018); the self-contained federation contract. |
| 3 | Routing rules | `routing_rules` | ✅ Yes | Resolve MSISDN prefix / IP CIDR → partner for outbound routing. |
| 4 | Shared resource catalogues | `shared_resource_catalogues` | ✅ Yes | Zone/capability availability exchanged with partners; validates federated deployment targets. |
| 5 | Federation transactions | `federation_transactions` | ✅ Yes | Audit + dispute-resolution record; correlates EWBI handle ↔ internal `operation_id`. |
| 6 | Edge cloud zones | *(none)* | ❌ No | SRM authoritative. FM reads via `GET /internal/zones`; stores only zone UUIDs when catalogued. |
| 7 | Service specifications / manifests | *(none)* | ❌ No | SRM owns the catalog. FM stores only the `service_specification_id` reference inside an agreement. |
| 8 | Service instance detail | *(none)* | ❌ No | SRM owns the instance record. FM keeps only `operation_id`/result summary in a transaction. |
| 9 | Capacity / reservations | *(none)* | ❌ No | SRM holds no stored capacity (ADR-0002); FM's `POST /internal/capacity-check` is best-effort, not persisted. |
| 2 | Federation contexts | `federation_contexts` | ✅ Yes | Directed OPG.04 context identity, direction, lifecycle, agreement, and callback URL. |
| 3 | Federation agreements | `federation_agreements` | ✅ Yes | Permissions + `appId → service_specification_id` mappings (ADR-0018); the self-contained federation contract. |
| 4 | Routing rules | `routing_rules` | ✅ Yes | Resolve MSISDN prefix / IP CIDR → partner for outbound routing. |
| 5 | Shared resource catalogues | `shared_resource_catalogues` | ✅ Yes | Zone/capability availability exchanged with partners; validates federated deployment targets. |
| 6 | Federation transactions | `federation_transactions` | ✅ Yes | Audit + dispute-resolution record; correlates context and internal `operation_id`. |
| 7 | Edge cloud zones | *(none)* | ❌ No | SRM authoritative. FM reads via `GET /internal/zones`; stores only zone UUIDs when catalogued. |
| 8 | Service specifications / manifests | *(none)* | ❌ No | SRM owns the catalog. FM stores only the `service_specification_id` reference inside an agreement. |
| 9 | Service instance detail | *(none)* | ❌ No | SRM owns the instance record. FM keeps only `operation_id`/result summary in a transaction. |
| 10 | Capacity / reservations | *(none)* | ❌ No | SRM holds no stored capacity (ADR-0002); FM's `POST /internal/capacity-check` is best-effort, not persisted. |

### What is intentionally not persisted

@@ -143,12 +152,28 @@ anything owned by SRM is held only as a UUID reference.
| `supported_apis` | JSONB | NOT NULL, DEFAULT '[]' | API families the partner supports |
| `sla_config` | JSONB | DEFAULT '{}' | SLA parameters (max rps, max concurrent) |
| `status` | VARCHAR(20) | NOT NULL, DEFAULT 'pending' | `pending`, `active`, `suspended`, `decommissioned` |
| `last_heartbeat_at` | TIMESTAMPTZ | | Last successful heartbeat |
| `last_health_check_at` | TIMESTAMPTZ | | Last successful OPG.04 `GetFederationHealth` check |
| `created_at` | TIMESTAMPTZ | NOT NULL, DEFAULT now() | |
| `updated_at` | TIMESTAMPTZ | NOT NULL, DEFAULT now() | |

Indexes: `idx_partner_ops_mcc_mnc`, `idx_partner_ops_status`, `idx_partner_ops_oauth2_client_id`.

#### `federation_contexts`

| Column | Type | Constraints | Description |
|---|---|---|---|
| `id` | UUID | PK, DEFAULT gen_random_uuid() | Internal row identity |
| `partner_op_id` | UUID | NOT NULL, FK → partner_ops(id) | Bilateral partner |
| `agreement_id` | UUID | FK → federation_agreements(id) | Governing agreement |
| `direction` | VARCHAR(10) | NOT NULL | `inbound` when FM issued the context; `outbound` when the partner issued it |
| `federation_context_id` | VARCHAR(255) | NOT NULL | Opaque OPG.04 context identifier; not assumed to be a UUID |
| `status_callback_url` | TEXT | | OPG.04 `partnerStatusLink` for status callbacks |
| `status` | VARCHAR(20) | NOT NULL | Directed federation lifecycle state |
| `created_at` | TIMESTAMPTZ | NOT NULL, DEFAULT now() | |
| `updated_at` | TIMESTAMPTZ | NOT NULL, DEFAULT now() | |

Constraint: `UNIQUE (partner_op_id, direction, federation_context_id)`.

#### `federation_agreements`

| Column | Type | Constraints | Description |
@@ -205,26 +230,33 @@ Indexes: `idx_shared_catalogue_partner_direction` on `(partner_op_id, direction)
| `partner_op_id` | UUID | NOT NULL, FK → partner_ops(id) | |
| `agreement_id` | UUID | FK → federation_agreements(id) | Governing agreement |
| `direction` | VARCHAR(10) | NOT NULL | `inbound` or `outbound` |
| `federation_operation_id` | UUID | | Opaque handle returned to partner as EWBI `operationId`; minted independently of `operation_id`. NULL for sync queries |
| `operation_id` | UUID | | SRM correlation when an async command was dispatched; NULL for sync queries. Never returned to partners |
| `correlation_id` | UUID | | Internal trace ID; mapped to/from `federation_correlation_id` at the boundary. Never returned to partners |
| `federation_correlation_id` | VARCHAR(255) | NOT NULL | External EWBI correlation ID |
| `api_type` | VARCHAR(100) | NOT NULL | e.g. `edge-cloud-deploy`, `qod-session` |
| `status` | VARCHAR(20) | NOT NULL, DEFAULT 'pending' | lowercase: `pending`, `in_progress`, `completed`, `partially_completed`, `failed`. EWBI JSON exposes uppercase |
| `federation_context_row_id` | UUID | FK → federation_contexts(id) | Directed OPG.04 context for this interaction |
| `federation_operation_id` | UUID | | Optional federation-side handle, independent of internal `operation_id`; not a universal OPG.04 `operationId` |
| `operation_id` | UUID | | SRM correlation when an async command was dispatched; NULL for synchronous queries. Never returned to partners |
| `correlation_id` | UUID | | Internal trace ID; mapped to/from the optional `federation_correlation_id` at the boundary. Never returned to partners |
| `federation_correlation_id` | VARCHAR(255) | | Optional OOP correlation extension |
| `idempotency_key` | VARCHAR(255) | | Required OPG.04 `Idempotency-Key` where the operation declares it |
| `request_fingerprint` | CHAR(64) | | SHA-256 of canonical request data; detects idempotency-key reuse with different content |
| `external_resource_id` | VARCHAR(255) | | Contract-defined external identifier, such as `appInstIdentifier` |
| `api_type` | VARCHAR(100) | NOT NULL | Internal normalized operation key, e.g. `install-app`, `device-location` |
| `status` | VARCHAR(20) | NOT NULL, DEFAULT 'pending' | Internal lowercase audit state: `pending`, `in_progress`, `completed`, `partially_completed`, `failed` |
| `request_summary` | JSONB | NOT NULL | Sanitized request metadata |
| `response_summary` | JSONB | | Sanitized response metadata |
| `error_detail` | JSONB | | RFC 7807 error if failed |
| `callback_url` | TEXT | | Required operation callback URL, when applicable |
| `callback_status` | VARCHAR(20) | | `pending`, `delivered`, `failed`; NULL without callback |
| `callback_attempts` | INTEGER | NOT NULL, DEFAULT 0 | Durable callback delivery attempts |
| `started_at` | TIMESTAMPTZ | NOT NULL, DEFAULT now() | |
| `completed_at` | TIMESTAMPTZ | | |
| `created_at` | TIMESTAMPTZ | NOT NULL, DEFAULT now() | |

Indexes: `idx_fed_tx_partner`, `UNIQUE (federation_operation_id) WHERE federation_operation_id IS NOT NULL` (poll lookup key for `GET /ewbi/v1/service/operations/{operationId}`), `idx_fed_tx_operation`, `idx_fed_tx_status`, `idx_fed_tx_started`, `idx_fed_tx_fed_correlation`.
Indexes: `idx_fed_tx_partner`, `UNIQUE (federation_operation_id) WHERE federation_operation_id IS NOT NULL`, `UNIQUE (partner_op_id, api_type, idempotency_key) WHERE idempotency_key IS NOT NULL`, `idx_fed_tx_context`, `idx_fed_tx_operation`, `idx_fed_tx_status`, `idx_fed_tx_started`, `idx_fed_tx_fed_correlation`.

Retention: configurable per operator policy; consider partitioning by `started_at` quarter.

### J.2 NATS (DataBus)

FM publishes commands and consumes SRM events; the client-visible source of truth for federation
FM publishes SRM commands, consumes FM federation commands and SRM events; the client-visible source of truth for federation
status remains `federation_transactions` in PostgreSQL.

| Stream | Subjects | Role |
@@ -232,6 +264,11 @@ status remains `federation_transactions` in PostgreSQL.
| `OOP_TASKS` | `command.srm.service.{deploy,scale,terminate}`, `command.srm.network.capability.{activate,update,deactivate}` | FM publishes; SRM consumes. Every command carries `CommandEnvelopeV1` with `source=federation`, `federation_partner_ref`, `app_provider_id=partner.id` (ADR-0001, ADR-0018) |
| `OOP_EVENTS` | `event.srm.operation.status`, `event.srm.operation.completed`, `event.srm.catalog.updated` | FM consumes via durable `fm-event-worker` (explicit ack, max-deliver 3) |

OEG-proxied federation establishment is a separate FM command input under
`command.fm.federation.*` (ADR-0043), including `command.fm.federation.create` and
`command.fm.federation.zone_subscribe`. The command schema/stream binding remains a platform
integration item; edge-cloud deployment API types are not dispatched through this hand-off.

**Delivery guarantee:** FM writes the `federation_transactions` row (`pending`) **before** publishing;
on successful publish it sets `in_progress` locally. Duplicates are absorbed by SRM's
`service_order.UNIQUE(operation_id)` idempotency anchor.
@@ -245,7 +282,7 @@ federation-manager/
├── src/federation_manager/
│   ├── core/                     # config, logging
│   ├── domain/
│   │   ├── models.py             # PartnerOP, Agreement, RoutingRule, Transaction entities + rules
│   │   ├── models.py             # PartnerOP, Context, Agreement, RoutingRule, Transaction entities + rules
│   │   ├── ports.py              # Protocols for repos, SRM client, DataBus publisher
│   │   ├── errors.py             # federation errors (no HTTP detail)
│   │   ├── topology_hiding.py    # sanitize rules
@@ -278,15 +315,15 @@ federation-manager/

| Topic | Approach |
|---|---|
| Inbound AuthN/Z | OAuth2 Client-Credentials + signed JWT via Keycloak `federation` realm (`fed-mgmt` scope); FM validates against Keycloak's JWKS (ADR-0038) |
| Inbound AuthN/Z | OAuth2 Client-Credentials + signed JWT via Keycloak `federation` realm (`fed-mgmt` scope); FM validates against Keycloak's JWKS (ADR-0042) |
| Admin AuthN/Z | Keycloak JWT, `operator` role, on `/internal/*` admin routes only — same centralized Keycloak, `oop` realm |
| Outbound AuthN | OAuth2 Client-Credentials — FM fetches a token from the partner's realm using our registered client credentials (ADR-0038) |
| Future: mTLS defense-in-depth | Deferred until real Nginx mTLS termination exists (ADR-0038). No certificate columns are carried today; the trust material they would need is defined when that layer is actually built. |
| Outbound AuthN | OAuth2 Client-Credentials — FM fetches a token from the partner's realm using our registered client credentials (ADR-0042) |
| Future: mTLS defense-in-depth | Deferred until real Nginx mTLS termination exists (ADR-0042). `partner_ca_cert_path` supports ordinary HTTPS verification; it is not an mTLS identity field. |
| Topology hiding | mandatory `TopologyHider` on every EWBI response (REQ-FED-02); strips internal IPs, cluster/namespace/node names, pod UIDs, trace IDs |
| Correlation boundary | internal `correlation_id` ↔ external `X-Federation-Correlation-ID`; internal IDs never leave |
| Header trust | FM must be reachable only via Nginx. FM trusts no caller-supplied identity header — partner identity comes solely from the validated JWT. |

Reference: OPG.04 v6.0 §9 (E/WBI OAuth2 Client-Credentials authorization — ADR-0038); OPG.02 v10.0 §2.4 (topology hiding), §4.2.3 (E/WBI authentication + agreed-resource exposure; not §4.2.2.3, corrected), §3.3.4.5 (mTLS as a mutually-agreed mechanism, deferred).
Reference: OPG.04 v6.0 §9 (E/WBI OAuth2 Client-Credentials authorization — ADR-0042); OPG.02 v10.0 §2.4 (topology hiding), §4.2.3 (E/WBI authentication + agreed-resource exposure; not §4.2.2.3, corrected), §3.3.4.5 (mTLS as a mutually-agreed mechanism, deferred).

## Implementation Status

@@ -294,14 +331,16 @@ Tracks the FM rewrite (branch `feature/r2-fastapi-rewrite`). Update as tables la

| Table | Schema in code | Repo adapter | Notes |
|---|---|---|---|
| `partner_ops` | Partial (id, mcc_mnc, oauth2_client_id, base_url, our_client_id, our_client_secret_ref, token_endpoint, status) | `PostgresPartnerRepo.find_by_oauth2_client_id`, `.find_by_id` | `name`, `supported_apis`, `sla_config`, `last_heartbeat_at`, `partner_ca_cert_path` pending |
| `partner_ops` | Partial (id, mcc_mnc, oauth2_client_id, base_url, our_client_id, our_client_secret_ref, token_endpoint, status) | `PostgresPartnerRepo.find_by_oauth2_client_id`, `.find_by_id` | `name`, `supported_apis`, `sla_config`, `last_health_check_at`, `partner_ca_cert_path` pending |
| `federation_contexts` | Not yet | — | Required to resolve opaque OPG.04 `federationContextId` by partner and direction |
| `federation_agreements` | Full | `PostgresAgreementRepo.find_active_for_partner` (prefers the agreement valid now, else newest) | Used by inbound authorizer and outbound hand-off (`AgreementChecker`) |
| `routing_rules` | Full (incl. `UNIQUE(identifier_type, value_range, priority)`, partial active index) | `PostgresRoutingRuleRepo.list_active` | Matching lives in `domain/routing.py` (`RoutingResolver`): most specific match wins, `priority` breaks ties |
| `shared_resource_catalogues` | Not yet | — | Catalogue sync (Phase 2) |
| `federation_transactions` | Full (incl. partial unique index on `federation_operation_id`) | `PostgresTransactionRepo.add`, `.record_outcome` | Written `pending` before the outbound EWBI call by `POST /internal/federation/outbound`; inbound `command.srm.*` path not wired yet |
| `federation_transactions` | Partial (current code lacks context, idempotency, request fingerprint, external resource, and callback columns) | `PostgresTransactionRepo.add`, `.record_outcome` | Written `pending` before the outbound hand-off; inbound `command.srm.*` path not wired yet |

## Revision History

| Version | Date | Summary |
|---------|------|---------|
| 2.0-draft | 2026-08-06 | Initial FM persistence model, promoted from requirements-and-design §J–§K. |
| 2.1-draft | 2026-09-08 | Align entities, context routing, transaction audit fields, ADR references, and DataBus roles with current FM architecture. |