Merge branch 'release/tfsdk-2.0.0-preparation' into 'develop'

  script:

  - python ci/security.py dependencies

  rules:

  - if: $CI_COMMIT_TAG || $CI_PIPELINE_SOURCE == "schedule" || $RUN_EXTENDED == "1"

  - if: $CI_COMMIT_TAG || ($CI_COMMIT_BRANCH == "main" && $CI_COMMIT_REF_PROTECTED == "true" && $CI_PIPELINE_SOURCE == "push") || $CI_PIPELINE_SOURCE == "schedule" || $RUN_EXTENDED == "1"

  - when: manual

    allow_failure: true

  artifacts:

  - job: build-package

    artifacts: true

  rules:

  - if: $CI_COMMIT_TAG || $CI_PIPELINE_SOURCE == "schedule" || $RUN_EXTENDED == "1"

  - if: $CI_COMMIT_TAG || ($CI_COMMIT_BRANCH == "main" && $CI_COMMIT_REF_PROTECTED == "true" && $CI_PIPELINE_SOURCE == "push") || $CI_PIPELINE_SOURCE == "schedule" || $RUN_EXTENDED == "1"

  script:

  - python ci/verify.py tests

  artifacts:

    - job: python-compatibility

    - job: srm-consumer

  rules:

    - if: '$CI_COMMIT_BRANCH == "main" && $CI_COMMIT_REF_PROTECTED == "true" && $CI_PIPELINE_SOURCE == "push"'

      when: on_success

      allow_failure: false

    - if: '$CI_COMMIT_TAG =~ /^v2[.]0[.][0-9]+(rc[1-9][0-9]*)?$/ && $CI_COMMIT_REF_PROTECTED == "true"'

      when: on_success

      allow_failure: false

# Release an SDK package

1. Review the release scope and applicable OEG-driven acceptance results, and

   merge the intended source/version through the release MR.

2. Create its matching protected tag, for example v2.0.0.

   Creating the tag approves publication after successful checks.

   CI builds the wheel and source distribution once, tests those bytes and runs

   the linked SRM consumer. The release matrix and dependency audit also run.

3. After required CI jobs pass, **publish-package** automatically uploads the

   tested artifacts to the project's GitLab Python package registry.

1. Review the release scope and merge the stable source/version into protected

   main through its release MR. The merge is the publication approval.

2. The main pipeline builds the wheel and source distribution once, verifies

   those bytes, runs the Python matrix, dependency audit and linked SRM consumer.

3. After required checks pass, publish-package creates v2.0.0 for package

   2.0.0 and uploads those same artifacts to the existing GitLab Python registry.

   The job-token tag push starts no second pipeline. GitLab Release pages remain

   with the release owner. Explicit protected candidate/tag pipelines remain

   available for trials and the existing publication path.

No manual job, qualification file, pipeline-ID input, extra approval form or

exception flag is required. Merging the release MR integrates source; branch

pipelines continue to avoid package uploads.

exception flag is required. Develop and MR pipelines do not publish packages.

## One-time project setting

Enable Settings → CI/CD → Job token permissions → Allow Git push requests to

the repository for this SDK project. GitLab calls the setting

ci_push_repository_for_job_token_allowed. This allows same-project job tokens

to push with their initiating user's repository permissions; it applies to

the project's jobs, not only this publisher. No new long-lived token is needed.

Keep branch/tag protections and cross-project pushes unchanged. The main merge

must be performed by a Maintainer allowed to create the protected release tag.

The publisher automatically saves release-qualification.json with artifact

identity, the GitLab publication job URL, actual SDK JUnit summary and linked

The existing required jobs and protected tag/version/source/checksum checks

remain enforced. Report collection changes no test outcome. If evidence is

incomplete, the release owner decides whether to wait or proceed before creating

the tag and can explain the decision in the existing MR. No separate exception

incomplete, the release owner decides whether to wait or merge and can explain

the decision in the existing MR. No separate exception

workflow is needed.

If a required job fails, fix/retry it or use an explicitly approved targeted CI

change through the normal review process.

For upload failures, retry the same job with the original artifacts. Inspect the

registry first if a partial upload occurred; never replace published bytes.

For upload failures, retry the same job with the original artifacts. The publisher

checks registry hashes, skips identical files and uploads only missing files.

A different checksum fails. A tag at another commit is never moved or replaced;

later main work at that already-tagged version skips publication. A new source

release needs a new package version.

A source, version or rebuilt-artifact change needs a fresh version/tag and checks.

## Order across repositories

Verify the candidate wheel directly in SRM before registry publication. Keep

the working protected consumer ref during the first final SDK publication;

do not switch that trigger to a consumer whose setup requires the unpublished

package. Applicable E2E evidence is reviewed here, while full final-platform

acceptance follows the normal SRM image built with the published final package.

After the SDK main pipeline creates the tag and publishes the package, verify

the registry files against that pipeline's build checksums. SRM maintainers then

update pyproject.toml and uv.lock together from the actual registry, run normal

locked checks and publish their image. Update chart/profile references, run

the applicable OEG-driven E2E and review the platform release MRs. The release

owner creates the agreed final platform tags and GitLab Releases. SDK package

availability must precede SRM's dependency installation, even when release

pages and the coordinated platform announcement are created later.

import sys

import zipfile

from email.parser import BytesParser

from urllib.parse import urlencode

from urllib.request import Request, urlopen

from packaging.version import Version

from reports import collect_reports

        raise ValueError("Publication requires a protected release tag")

def release_context(version):

    tag = os.getenv("CI_COMMIT_TAG", "")

    if tag:

        return tag, False

    if (

        os.getenv("CI_COMMIT_BRANCH") != "main"

        or os.getenv("CI_COMMIT_REF_PROTECTED") != "true"

        or os.getenv("CI_PIPELINE_SOURCE") != "push"

    ):

        raise ValueError("Automatic publication requires a protected main push")

    if Version(version).is_prerelease:

        raise ValueError("Use a stable package version for automatic main publication")

    return f"v{version}", True

def ensure_main_tag(tag, revision, *, create=True):

    ref = f"refs/tags/{tag}"

    raw = subprocess.check_output(

        ["git", "ls-remote", "--tags", "origin", ref, ref + "^{}"], text=True

    )

    refs = {name: sha for sha, name in (line.split() for line in raw.splitlines())}

    existing = refs.get(ref + "^{}", refs.get(ref))

    if existing:

        return existing == revision  # A later main merge cannot move a released version.

    if not create:

        return True

    try:

        subprocess.run(

            ["git", "push", "origin", f"{revision}:{ref}"], check=True, capture_output=True

        )

    except subprocess.CalledProcessError:

        raise ValueError(

            "Tag creation failed; enable same-project job-token Git pushes "

            "and use a Maintainer-triggered pipeline"

        ) from None

    return True

def registry_json(path):

    base = f"{os.environ['CI_API_V4_URL']}/projects/{os.environ['CI_PROJECT_ID']}"

    request = Request(base + path, headers={"JOB-TOKEN": os.environ["CI_JOB_TOKEN"]})

    with urlopen(request, timeout=30) as response:

        return json.load(response)

def pending_artifacts(version):

    query = urlencode(

        {

            "package_type": "pypi",

            "package_name": "openop-tfsdk",

            "package_version": version,

            "per_page": 100,

        }

    )

    existing = {}

    for package in registry_json("/packages?" + query):

        if package["name"] != "openop-tfsdk" or package["version"] != version:

            continue

        for item in registry_json(f"/packages/{package['id']}/package_files"):

            name, checksum = item["file_name"], item["file_sha256"]

            if name in existing and existing[name] != checksum:

                raise ValueError("Registry contains conflicting artifact checksums")

            existing[name] = checksum

    pending = []

    for path in sorted(DIST.iterdir()):

        if path.name not in existing:

            pending.append(path)

        elif existing[path.name] != sha256(path):

            raise ValueError("Published artifact differs; never overwrite a released version")

    return pending

def record_publication(receipt, status):

    receipt["publication_status"] = status

    write_json(OUT / "release-check.json", receipt)

    qualification = OUT / "release-qualification.json"

    write_json(qualification, {**json.loads(qualification.read_text()), **receipt})

def verify_release():

    manifest = json.loads((OUT / "manifest.json").read_text())

    for item in manifest["artifacts"]:

    if metadata["Name"] != "openop-tfsdk":

        raise ValueError("Unexpected distribution name")

    version = metadata["Version"]

    check_tag(os.getenv("CI_COMMIT_TAG", ""), version, os.getenv("CI_COMMIT_REF_PROTECTED"))

    tag, automatic = release_context(version)

    check_tag(tag, version, os.getenv("CI_COMMIT_REF_PROTECTED"))

    if manifest["source_sha"] != os.environ["CI_COMMIT_SHA"]:

        raise ValueError("Build and tag source revisions differ")

    # Tag pipelines may fetch only a detached tag; refresh the allowed ancestry refs.

    subprocess.run(["git", "fetch", "origin", "+refs/heads/*:refs/remotes/origin/*"], check=True)

    branches = ["origin/develop", "origin/main", "origin/refactor/tfsdk-architecture-alignment"]

    branches = (

        ["origin/main"]

        if automatic

        else ["origin/develop", "origin/main", "origin/refactor/tfsdk-architecture-alignment"]

    )

    if not any(

        subprocess.run(

            ["git", "merge-base", "--is-ancestor", "HEAD", ref], capture_output=True

    write_json(

        OUT / "release-check.json",

        {

            "tag": os.environ["CI_COMMIT_TAG"],

            "tag": tag,

            "version": version,

            "source_sha": manifest["source_sha"],

            "wheel_sha256": sha256(wheel),

            "release_type": "candidate" if Version(version).is_prerelease else "final",

            "publication_job_url": os.getenv("CI_JOB_URL"),

            "approval_source": "protected_main_merge" if automatic else "protected_tag",

        },

    )

    receipt = json.loads((OUT / "release-check.json").read_text())

    write_json(OUT / "release-qualification.json", {**receipt, **collect_reports()})

    return receipt

def main():

    parser.add_argument("action", choices=["check", "publish"])

    args = parser.parse_args()

    os.chdir(ROOT)

    verify_release()

    receipt = verify_release()

    if args.action == "publish":

        if receipt["approval_source"] == "protected_main_merge" and not ensure_main_tag(

            receipt["tag"], receipt["source_sha"], create=False

        ):

            record_publication(receipt, "skipped_existing_release_tag")

            print("This version already has a release tag; no tag or package was replaced")

            return

        pending = pending_artifacts(receipt["version"])

        if receipt["approval_source"] == "protected_main_merge" and not ensure_main_tag(

            receipt["tag"], receipt["source_sha"]

        ):

            raise ValueError("Release tag changed during publication; no package was uploaded")

        if not pending:

            record_publication(receipt, "already_published_matching_checksums")

            print("Verified artifacts are already published")

            return

        environment = dict(

            os.environ, TWINE_USERNAME="gitlab-ci-token", TWINE_PASSWORD=os.environ["CI_JOB_TOKEN"]

        )

                "upload",

                "--repository-url",

                url,

                *[str(p) for p in DIST.iterdir()],

                *[str(p) for p in pending],

            ],

            env=environment,

            check=True,

        )

        record_publication(receipt, "published")

if __name__ == "__main__":

[project]

name = "openop-tfsdk"

version = "2.0.0rc5"

version = "2.0.0"

description = "Open source SDK to abstract CAMARA/GSMA Transformation Functions (TFs) for Edge Cloud platforms, 5G network cores and Open RAN solutions."

keywords = [

  "Federation",

"""A reviewed main merge publishes one immutable version without another token."""

import subprocess

import sys

from pathlib import Path

import pytest

CI = Path(__file__).resolve().parents[2] / "ci"

sys.path.insert(0, str(CI))

import release  # noqa: E402

@pytest.mark.parametrize(

    "branch,protected,source,allowed",

    [

        ("main", "true", "push", True),

        ("develop", "true", "push", False),

        ("main", "false", "push", False),

        ("main", "true", "merge_request_event", False),

        ("main", "true", "web", False),

    ],

)

def test_only_a_protected_main_push_supplies_automatic_release_approval(

    monkeypatch, branch, protected, source, allowed

):

    for key, value in {

        "CI_COMMIT_TAG": "",

        "CI_COMMIT_BRANCH": branch,

        "CI_COMMIT_REF_PROTECTED": protected,

        "CI_PIPELINE_SOURCE": source,

    }.items():

        monkeypatch.setenv(key, value)

    if allowed:

        assert release.release_context("2.0.0") == ("v2.0.0", True)

        with pytest.raises(ValueError):

            release.release_context("2.0.0rc6")

    else:

        with pytest.raises(ValueError):

            release.release_context("2.0.0")

def test_explicit_candidate_tags_keep_the_existing_publication_path(monkeypatch):

    monkeypatch.setenv("CI_COMMIT_TAG", "v2.0.0rc6")

    assert release.release_context("2.0.0rc6") == ("v2.0.0rc6", False)

def test_automatic_tag_is_created_once_and_never_moves(tmp_path, monkeypatch):

    def git(*args, cwd=None):

        return subprocess.check_output(

            ["git", *args], cwd=cwd, stderr=subprocess.DEVNULL, text=True

        ).strip()

    remote, work = tmp_path / "remote.git", tmp_path / "work"

    git("init", "--bare", str(remote))

    git("init", "-b", "main", str(work))

    def commit(message):

        git(

            "-c",

            "user.name=Release test",

            "-c",

            "user.email=test@example.invalid",

            "commit",

            "--allow-empty",

            "-m",

            message,

            cwd=work,

        )

        return git("rev-parse", "HEAD", cwd=work)

    first = commit("Reviewed source")

    git("remote", "add", "origin", str(remote), cwd=work)

    git("push", "origin", "main", cwd=work)

    monkeypatch.chdir(work)

    tag = "v2.0.0"

    assert release.ensure_main_tag(tag, first, create=False)

    assert git("--git-dir", str(remote), "tag", "--list") == ""

    assert release.ensure_main_tag(tag, first)

    assert release.ensure_main_tag(tag, first)

    later = commit("Later main work")

    assert not release.ensure_main_tag(tag, later)

    assert git("--git-dir", str(remote), "rev-parse", "refs/tags/" + tag) == first

    assert git("--git-dir", str(remote), "rev-parse", "refs/heads/main") == first

@pytest.mark.parametrize("state", ["absent", "partial", "complete", "different"])

def test_registry_retry_uploads_only_missing_identical_artifacts(tmp_path, monkeypatch, state):

    dist = tmp_path / "dist"

    dist.mkdir()

    wheel, sdist = dist / "sdk.whl", dist / "sdk.tar.gz"

    wheel.write_bytes(b"tested wheel")

    sdist.write_bytes(b"tested source archive")

    monkeypatch.setattr(release, "DIST", dist)

    files = (

        []

        if state == "absent"

        else [

            {

                "file_name": wheel.name,

                "file_sha256": "wrong" if state == "different" else release.sha256(wheel),

            }

        ]

    )

    if state == "complete":

        files.append({"file_name": sdist.name, "file_sha256": release.sha256(sdist)})

    monkeypatch.setattr(

        release,

        "registry_json",

        lambda path: (

            files

            if path.endswith("package_files")

            else [{"id": 1, "name": "openop-tfsdk", "version": "2.0.0"}]

        ),

    )

    if state == "different":

        with pytest.raises(ValueError, match="never overwrite"):

            release.pending_artifacts("2.0.0")

    else:

        assert set(release.pending_artifacts("2.0.0")) == (

            {wheel, sdist} if state == "absent" else {sdist} if state == "partial" else set()

        )