Merge branch 'feature/r2-fastapi-rewrite' into 'develop'

# Folders

.idea

venv

.venv

__pycache__/

*.egg-info/

.mypy_cache/

.ruff_cache/

.pytest_cache/

src/__pycache__/

src/models/__pycache__/

src/controllers/__pycache__/

# Agent Guidelines for Federation Manager

This document provides coding agents with essential information for working in the Federation Manager codebase.

## Project Overview

**Federation Manager** is a Python-based component implementing Federation Management functionality for the GSMA Operator Platform (OP). It supports dual-role architecture:

- **Partner OP Mode**: Handles external federation partner requests (without `X-Internal` header)

- **Originating OP Mode**: Handles internal service requests (with `X-Internal` and `X-Partner-API-Root` headers)

## Build/Run Commands

### Setup

```bash

# Create virtual environment

python -m venv venv

source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies

pip install -r requirements.txt

# Configure application

cp src/conf/config.cfg.sample src/conf/config.cfg

# Edit config.cfg with your specific settings

```

### Run Application

```bash

# Local development

cd src/

python main.py

# Using Docker Compose (recommended)

docker compose up -d

# Build Docker image

docker build -t federation-manager .

```

### Testing

**Run all tests:**

```bash

cd src/

python test/run_all_tests.py --verbose --coverage

```

**Run specific test module:**

```bash

cd src/

python -m unittest test.test_federation_management -v

```

**Run single test case:**

```bash

cd src/

python -m unittest test.test_federation_management.TestFederationManagementController.test_01_create_federation -v

```

**Test execution order** (tests run sequentially):

1. `test_federation_management`

2. `test_availability_zone_info_synchronization`

3. `test_artefact_management`

4. `test_application_onboarding_management`

5. `test_application_deployment_management`

**Coverage reports:**

- Terminal output with coverage percentage

- HTML report in `src/htmlcov/` directory

### Linting

```bash

# No formal linting configured, but PEP8 style is followed

# Use flake8 or pylint manually if needed

flake8 src/ --max-line-length=120

```

## Architecture

```

API Layer (src/api/) 

  ↓

Adapter Layer (src/adapters/)

  ├── tf_adapter/ (Partner OP mode - external requests)

  └── fm_adapter/ (Originating OP mode - internal requests)

  ↓

Client Layer (src/clients/)

  ├── fed_manager.py (Federation Manager client)

  └── tf_sdk.py (Edge Cloud Platform SDK)

```

**Request routing** via `adapters.injector.resolve_adapter()`:

- Without `X-Internal` header → `tf_adapter` (Partner OP)

- With `X-Internal` header → `fm_adapter` (Originating OP)

## Code Style Guidelines

### General Python Standards

- Follow **PEP8** coding conventions

- Python **3.12+** required

- Use **snake_case** for functions and variables

- Use **PascalCase** for class names

- Maximum line length: **120 characters** (flexible)

### Imports

```python

# Standard library imports (absolute)

from __future__ import absolute_import

import os

import sys

from configparser import ConfigParser

from datetime import date, datetime  # noqa: F401

# Third-party imports

import connexion

from flask import abort, render_template

from flask_mongoengine import MongoEngine

# Local application imports

from adapters.injector import resolve_adapter

from adapters.error import APIError

from models.federation_request_data import FederationRequestData  # noqa: E501

import util

```

**Import ordering:**

1. `from __future__` imports (if needed for compatibility)

2. Standard library imports

3. Third-party library imports

4. Local application imports

**Import conventions:**

- Use explicit imports from packages

- Long model imports are acceptable with `# noqa: E501` to suppress line length warnings

- Use `# noqa: F401` for imports needed only for type hints

### File Headers

All Python files must include the Apache 2.0 license header:

```python

# -------------------------------------------------------------------------- #

# Copyright 2025-present, Federation Manager, by Software Networks, i2CAT    #

#                                                                            #

# Licensed under the Apache License, Version 2.0 (the "License"); you may    #

# not use this file except in compliance with the License. You may obtain    #

# a copy of the License at                                                   #

#                                                                            #

# http://www.apache.org/licenses/LICENSE-2.0                                 #

#                                                                            #

# Unless required by applicable law or agreed to in writing, software        #

# distributed under the License is distributed on an "AS IS" BASIS,          #

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.   #

# See the License for the specific language governing permissions and        #

# limitations under the License.                                             #

# -------------------------------------------------------------------------- #

```

### Type Hints

```python

# Function signatures with type hints

def create_federation(body: FederationRequestData) -> FederationResponseData:

    """Creates one direction federation with partner operator platform."""

    pass

# Model classes use swagger_types for type information

class CountryCode(Model):

    def __init__(self):

        self.swagger_types = {}

        self.attribute_map = {}

```

### Docstrings

```python

def create_federation(body):  # noqa: E501

    """Creates one direction federation with partner operator platform.

     # noqa: E501

    :param body:

    :type body: dict | bytes

    :rtype: FederationResponseData

    """

    pass

```

### Configuration Management

```python

# Use ConfigParser for configuration

from configparser import ConfigParser

CONFIG = ConfigParser()

config_file = os.environ.get("FM_CONFIG_FILE", "conf/config.cfg")

CONFIG.read(config_file)

HOST = CONFIG.get("server", "host")

PORT = int(CONFIG.get("server", "port"))

```

### Error Handling

```python

# Use custom APIError for API errors

from adapters.error import APIError

try:

    adapter = resolve_adapter(headers)

    return adapter.federation_management.create_federation(body, bearer_token, partner_api_root)

except APIError as error:

    abort(error.status_code, error.detail_error)

# Use abort() for HTTP errors

from flask import abort

try:

    body = FederationRequestData.from_dict(connexion.request.get_json())

except Exception as error:

    abort(422, f"Federation Validation Error. Message: {error}")

```

### Testing Conventions

```python

# Test classes inherit from BaseTestCase

from test import BaseTestCase

class TestFederationManagementController(BaseTestCase):

    """FederationManagementController integration test stubs"""

    BaseTestCase.federation_context_id_partner = ""

    BaseTestCase.token = ""

    def test_01_create_federation(self):

        """Test case for create_federation in both Partner OP and Originating OP modes"""

        # Test both roles

        pass

```

**Dual role testing:**

- All integration tests must validate both Partner OP and Originating OP modes

- Use `make_request_partner_op()` for external requests (no `X-Internal` header)

- Use `make_request_originating_op()` for internal requests (with `X-Internal` header)

### Naming Conventions

- **Functions/Methods**: `snake_case` (e.g., `create_federation`, `get_federation_details`)

- **Classes**: `PascalCase` (e.g., `FederationRequestData`, `BaseTestCase`)

- **Constants**: `UPPER_CASE` (e.g., `CONFIG`, `HOST`, `PORT`)

- **Variables**: `snake_case` (e.g., `bearer_token`, `partner_api_root`)

- **Test methods**: Prefix with `test_` and optionally number for execution order (e.g., `test_01_create_federation`)

## Key Files and Locations

- **Application entry**: `src/main.py`

- **API endpoints**: `src/api/`

- **Business logic**: `src/adapters/fm_adapter/` and `src/adapters/tf_adapter/`

- **Client SDKs**: `src/clients/`

- **Data models**: `src/models/`

- **Tests**: `src/test/`

- **Configuration**: `src/conf/config.cfg`

- **OpenAPI spec**: `src/swagger/swagger.yaml`

## Development Workflow

1. **Before making changes**: Review the dual-role architecture to understand request routing

2. **Configuration**: Always use `ConfigParser` to read from `conf/config.cfg`

3. **Adding endpoints**: Update `src/swagger/swagger.yaml` first, then implement in `src/api/`

4. **Adding business logic**: Implement in both adapters if applicable (fm_adapter and tf_adapter)

5. **Testing**: Write integration tests that validate both Partner OP and Originating OP modes

6. **Documentation**: Update docstrings and inline comments; avoid creating unnecessary markdown files

## Common Patterns

### Request Header Extraction

```python

bearer_token = util.get_token_from_request(connexion)

headers = dict(connexion.request.headers)

partner_api_root = headers.get("X-Partner-Api-Root")

```

### Adapter Resolution

```python

from adapters.injector import resolve_adapter

adapter = resolve_adapter(headers)  # Returns tf_adapter or fm_adapter based on X-Internal header

return adapter.federation_management.create_federation(body, bearer_token, partner_api_root)

```

### Model Deserialization

```python

body = FederationRequestData.from_dict(connexion.request.get_json())

```

## Notes for AI Agents

- **Never skip the Apache 2.0 license header** when creating new files

- **Always test both dual roles** when modifying API or adapter code

- **Configuration changes** should be documented and may require updates to `config.cfg.sample`

- **Docker builds** use `gunicorn` as the WSGI server (see `Dockerfile`)

- **Authentication** via OAuth 2.0 with Keycloak integration

- **Database**: MongoDB with MongoEngine ORM

- **API Framework**: Connexion (Flask-based) with OpenAPI/Swagger specifications

# GitLab CI/CD Setup for Federation Manager

This document explains how to set up and use the GitLab CI/CD pipeline for the Federation Manager project.

## Overview

The CI/CD pipeline includes a single stage with two different build jobs:

1. **Build (MR Validation)** - Builds Docker images for merge requests without pushing to registry

2. **Build and Push** - Builds Docker images and pushes them to the GitLab Container Registry for main branches

## Prerequisites

### GitLab Configuration

1. **Container Registry**: Ensure GitLab Container Registry is enabled for your project

2. **CI/CD Variables**: Configure the following variables in GitLab (Settings > CI/CD > Variables):

   | Variable | Description | Required |

   |----------|-------------|----------|

   | `CI_REGISTRY` | GitLab Container Registry URL | Auto-provided |

   | `CI_REGISTRY_USER` | Registry username | Auto-provided |

   | `CI_REGISTRY_PASSWORD` | Registry password | Auto-provided |

### Repository Setup

1. Ensure your GitLab project has a Container Registry enabled

2. The pipeline uses the project's Container Registry by default

3. Docker images will be tagged with commit SHA and 'latest'

## Pipeline Stages

### Build Stage

The build stage contains two different jobs depending on the trigger:

#### MR Build (build-mr)

- **Trigger**: On merge requests

- **Actions**:

  - Builds Docker image for validation

  - Tags with local name: `federation-manager:$CI_COMMIT_SHORT_SHA`

  - **Does NOT push to registry** (validation only)

  - No registry authentication required

#### Build and Push (build-and-push)

- **Trigger**: On main, develop branches, and tags

- **Actions**:

  - Builds Docker image

  - Tags with commit SHA and 'latest'

  - Pushes to GitLab Container Registry

  - Requires registry authentication

## Docker Image Information

### Image Naming Convention

#### Registry Images (build-and-push job)

- Registry: `$CI_REGISTRY_IMAGE/federation-manager`

- Tags:

  - `latest` - Latest build from main/develop branches

  - `$CI_COMMIT_SHORT_SHA` - Specific commit hash

#### Local Images (build-mr job)

- Local name: `federation-manager:$CI_COMMIT_SHORT_SHA`

- **Note**: These images are built locally for validation only and are NOT pushed to the registry

### Base Image

- Python 3.12 official image

- Includes system dependencies: bash, build-essential, git, wget, iptables, etc.

- Exposes port 8989

- Runs with Gunicorn (4 workers)

## Usage

### Triggering Builds

1. **Automatic triggers**:

   - **MR Build**: Create or update merge requests (builds for validation only)

   - **Build and Push**: Push to main or develop branches, create tags

2. **Manual triggers**:

   - Go to CI/CD > Pipelines in GitLab

   - Click "Run Pipeline"

   - Select branch/tag

### Build Types

#### Merge Request Validation

- **Purpose**: Validate that Docker builds work correctly before merging

- **Benefit**: Early feedback without affecting registry or consuming unnecessary resources

- **Output**: Local Docker image (not stored in registry)

- **Time**: Faster execution (no registry operations)

#### Production Builds

- **Purpose**: Create deployable images for main branches and releases

- **Output**: Images pushed to GitLab Container Registry

- **Accessibility**: Available for deployment and distribution

### Accessing Docker Images

**Note**: Only images from the `build-and-push` job are available in the registry. MR validation builds are local only.

1. **Pull from registry** (production images only):

   ```bash

   docker login registry.gitlab.com

   docker pull $CI_REGISTRY_IMAGE/federation-manager:latest

   ```

2. **Use in docker-compose**:

   ```yaml

   services:

     federation-manager:

       image: registry.gitlab.com/your-group/federation-manager/federation-manager:latest

   ```

### Monitoring Pipeline

- **Pipeline status**: GitLab project > CI/CD > Pipelines

- **Job logs**: Click on individual jobs to view logs

- **Coverage reports**: Available in merge request widgets

- **Container registry**: GitLab project > Packages & Registries > Container Registry

## Troubleshooting

### Common Issues

1. **Docker build fails**:

   - Check Dockerfile syntax

   - Verify all required files are present

   - Check .dockerignore doesn't exclude necessary files

2. **Registry push fails**:

   - Verify Container Registry is enabled

   - Check CI/CD variables are set correctly

   - Ensure sufficient permissions

3. **Build fails**:

   - Check Dockerfile syntax and build context

   - Verify all required files are present and not excluded by .dockerignore

   - Check Python 3.12 compatibility in requirements.txt

   - Ensure base image availability

services:

  postgres:

    image: postgres:16-alpine

    container_name: fm-postgres

    environment:

      POSTGRES_USER: fm

      POSTGRES_PASSWORD: fm

      POSTGRES_DB: fm_db

    ports:

      - "5433:5432"

    healthcheck:

      test: ["CMD-SHELL", "pg_isready -U fm -d fm_db"]

      interval: 3s

      timeout: 3s

      retries: 20

    volumes:

      - fm-pgdata:/var/lib/postgresql/data

  nats:

    image: nats:2.10-alpine

    container_name: fm-nats

    command: ["-js"]

    ports:

      - "4222:4222"

  keycloak:

    image: quay.io/keycloak/keycloak:26.1.4

    container_name: fm-keycloak

    command: ["start-dev", "--import-realm"]

    environment:

      KC_BOOTSTRAP_ADMIN_USERNAME: admin

      KC_BOOTSTRAP_ADMIN_PASSWORD: admin

    ports:

      - "8090:8080"

    volumes:

      - ./keycloak:/opt/keycloak/data/import

    healthcheck:

      test: ["CMD-SHELL", "exec 3<>/dev/tcp/127.0.0.1/8080"]

      interval: 3s

      timeout: 3s

      retries: 30

volumes:

  fm-pgdata: