Align control-path bootstrap with TF-SDK for Release 2.0.0

Konstantinos Togiasrequested to merge
fix/single-bootstrap-control-paths into develop

Align control-path bootstrap with TF-SDK for Release 2.0.0

Target: refactor/tf-sdk-srm-bug-fixes

Based on 2c083de, with the published SDK through 1b778a3.

Builds topology discovery and the execution registry from one SDK-loaded bootstrap. The registry key supplies each adapter's control_path_ref; the SDK checks incoming requests against that identity before backend work.

Replaces independent CONTROL_PATH_SETTINGS__ENTRIES input with the complete file selected by BOOTSTRAP_TOPOLOGY_PATH. Adds duplicate/conflicting identity checks and reuses existing SDK error mapping. Empty health startup and create-missing topology seeding remain available. Placement, database schema and termination use cases retain their existing behavior.

Requires TF-SDK 2.0.0rc3 and the matching deployment configuration: explicit kubeconfig/context/namespace or explicit in-cluster mode, protected credential mounts and a writable SDK runtime directory. Remove the chart's old controlPaths list. Upgrade or roll back the compatible component set together.

Local validation in a fresh locked environment with the registry-published SDK wheel:

  • Full SRM suite, including PostgreSQL/NATS integration: 535 passed.
  • Full mypy, Ruff lint/format and both import-boundary contracts passed.
  • New cases cover one bootstrap load, wrong-instance rejection, concurrent routes and invalid configuration. Observation and termination semantics remain unchanged.
  • Registry wheel and sdist match the tested tag-build hashes. The lock uses their actual published URLs; other package versions remain unchanged.
  • Live OEG/two-cluster acceptance remains separate.

Run MR CI with the locked dependencies before merge. Integrate the matching SDK/SRM/chart/configuration migration together.

Merge request reports

src/srm/adapters/transformation_functions/registry.py

+16 −2
Changes for src/srm/adapters/transformation_functions/registry.py: 16 added lines, 2 removed lines.
Original line number Diff line number Diff line 
from collections.abc import Mapping

from copy import deepcopy



import structlog
@@ -35,7 +36,7 @@ def build_control_path_registry( 
    settings: ControlPathSettings,

    factory: TransformationFunctionAdapterFactory | None,

) -> ConfiguredControlPathRegistry:

    logger.info("building_cpr", entries=settings.entries)

    logger.info("building_cpr", entry_count=len(settings.entries))

    if not settings.entries:

        return ConfiguredControlPathRegistry({})

    if factory is None:
@@ -43,9 +44,22 @@ def build_control_path_registry( 
            "control paths are configured but no TF-SDK adapter factory is installed: "

            + ", ".join(entry.control_path_ref for entry in settings.entries)

        )

    refs: set[str] = set()

    for entry in settings.entries:

        if not entry.control_path_ref.strip() or entry.control_path_ref in refs:

            raise ValueError("Control-path refs must be nonempty and unique")

        refs.add(entry.control_path_ref)

        if (

            "control_path_ref" in entry.config

            and entry.config["control_path_ref"] != entry.control_path_ref

        ):

            raise ValueError("Control-path config identity conflicts with its registry key")

    return ConfiguredControlPathRegistry(

        {

            entry.control_path_ref: factory.create_adapter(entry.adapter, entry.config)

            entry.control_path_ref: factory.create_adapter(

                entry.adapter,

                {**deepcopy(entry.config), "control_path_ref": entry.control_path_ref},

            )

            for entry in settings.entries

        }

    )

src/srm/adapters/transformation_functions/tfsdk_bootstrap.py

0 → 100644
+34 −0
Changes for src/srm/adapters/transformation_functions/tfsdk_bootstrap.py: 34 added lines, 0 removed lines.
Original line number Diff line number Diff line 
"""Assemble runtime registry and discovery from one SDK-loaded declaration."""



from openop_tfsdk.contract.topology import BootstrapTopology



from srm.adapters.transformation_functions.registry import (

    ConfiguredControlPathRegistry,

    build_control_path_registry,

)

from srm.adapters.transformation_functions.tfsdk_topology import TfSdkTopologyDiscoveryBridge

from srm.config import ControlPathEntry, ControlPathSettings

from srm.domain.ports.transformation_functions import (

    TopologyDiscoveryPort,

    TransformationFunctionAdapterFactory,

)





def build_tfsdk_bootstrap(

    bootstrap_path: str | None,

    factory: TransformationFunctionAdapterFactory | None,

) -> tuple[ConfiguredControlPathRegistry, TopologyDiscoveryPort | None]:

    if not bootstrap_path:

        return build_control_path_registry(ControlPathSettings(), factory), None

    bootstrap = BootstrapTopology.from_file(bootstrap_path)

    settings = ControlPathSettings(

        entries=[

            ControlPathEntry(

                control_path_ref=entry["ref"],

                adapter=entry["adapter_type"],

                config=entry["infrastructure"],

            )

            for entry in bootstrap.get_control_path_configurations()

        ]

    )

    return build_control_path_registry(settings, factory), TfSdkTopologyDiscoveryBridge(bootstrap)

src/srm/adapters/transformation_functions/tfsdk_execution.py

+2 −0
Changes for src/srm/adapters/transformation_functions/tfsdk_execution.py: 2 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -266,4 +266,6 @@ class OpenOpTfSdkAdapterFactory(TransformationFunctionAdapterFactory): 
            adapter = build_adapter(kind, dict(config))

        except TfSdkUnknownAdapterKind as exc:

            raise UnknownAdapterKind(kind) from exc

        except TfSdkTFSDKError as exc:

            raise _translate_tfsdk_error(exc) from exc

        return TfSdkExecutionBridge(adapter)

src/srm/config.py

+0 −1
Changes for src/srm/config.py: 0 added lines, 1 removed line.
Original line number Diff line number Diff line
@@ -70,7 +70,6 @@ class Settings(BaseSettings): 
    nats_settings: NatsSettings

    zone_provider_name: str = Field(default="oop-provider", min_length=1)

    bootstrap_settings: BootstrapSettings = BootstrapSettings()

    control_path_settings: ControlPathSettings = ControlPathSettings()

    federation_manager_settings: FederationManagerSettings

    bootstrap_topology_path: str | None = None

src/srm/main.py

+3 −5
Changes for src/srm/main.py: 3 added lines, 5 removed lines.
Original line number Diff line number Diff line
@@ -35,9 +35,8 @@ from srm.adapters.databus.nats_connection_manager import ( 
)

from srm.adapters.databus.nats_publisher import NatsPublisher

from srm.adapters.federation_manager.fm_client import FederationManager

from srm.adapters.transformation_functions.registry import build_control_path_registry

from srm.adapters.transformation_functions.tfsdk_bootstrap import build_tfsdk_bootstrap

from srm.adapters.transformation_functions.tfsdk_execution import OpenOpTfSdkAdapterFactory

from srm.adapters.transformation_functions.tfsdk_topology import build_topology_discovery

from srm.api.databus.nats_subscriber import (

    NatsEventSubscriber,

    NatsSubscriber,
@@ -65,11 +64,10 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]: 
    logger = structlog.get_logger()

    logger.info("Starting", app=settings.app_name, version=settings.app_version)



    control_path_registry = build_control_path_registry(

        settings.control_path_settings,

    control_path_registry, topology_discovery = build_tfsdk_bootstrap(

        settings.bootstrap_topology_path,

        factory=OpenOpTfSdkAdapterFactory(),

    )

    topology_discovery = build_topology_discovery(settings.bootstrap_topology_path)



    engine = None

    try: