Loading .gitlab-ci.yml +1 −0 Changes for .gitlab-ci.yml: 1 added line, 0 removed lines. Original line number Diff line number Diff line Loading @@ -130,6 +130,7 @@ sdk-consumer: when: always paths: - .cache/sdk-consumer/qualification.json - .cache/sdk-consumer/failure.json - .cache/sdk-consumer/project/pyproject.toml - .cache/sdk-consumer/project/uv.lock - .cache/sdk-consumer/project/wheels/ Loading Dockerfile +8 −1 Changes for Dockerfile: 8 added lines, 1 removed line. Original line number Diff line number Diff line Loading @@ -12,6 +12,8 @@ ENV PYTHONDONTWRITEBYTECODE=1 \ PIP_DEFAULT_TIMEOUT=120 ARG DEPENDENCY_PROJECT=. ARG SDK_CONSUMER_MODE=adoption ARG SDK_WHEEL_NAME COPY ${DEPENDENCY_PROJECT}/pyproject.toml ./ COPY ${DEPENDENCY_PROJECT}/uv.lock ./ Loading @@ -20,7 +22,12 @@ RUN --mount=type=bind,target=/build-context \ uv sync --locked; \ else \ cp -r "/build-context/$DEPENDENCY_PROJECT/wheels" ./wheels && \ uv sync --locked --no-install-project; \ uv sync --locked --no-install-project && \ case "$SDK_CONSUMER_MODE" in \ adoption) ;; \ compatibility) uv pip install --python /app/.venv/bin/python --no-deps --no-index --reinstall "/app/wheels/$SDK_WHEEL_NAME" ;; \ *) exit 1 ;; \ esac; \ fi FROM ${PYTHON_IMAGE} AS runtime Loading ci/sdk-consumer.md +35 −7 Changes for ci/sdk-consumer.md: 35 added lines, 7 removed lines. Original line number Diff line number Diff line Loading @@ -5,7 +5,27 @@ merges. It sends one wheel's build job, source revision and checksum. SRM checks its own checkout against `EXPECTED_SRM_SHA` before installing anything. A changed branch head requires a new run. Before selection, the SRM MR must explicitly adopt the candidate. Use the SDK ## Compatibility without adoption SDK target-branch checks and branch experiments set `SDK_CONSUMER_MODE=compatibility`. No SRM MR or SDK pin update is required. The job copies the committed dependency files unchanged and installs their lock. It then replaces only SDK with the exact candidate wheel, using no index and no dependency installation. Every other installed package/version must remain unchanged. Installed SDK contents and all package requirements are checked. An incompatible dependency baseline produces `failure.json` with status `dependency-update-required`; no successful qualification or image is produced. Prepare explicit adoption rather than resolving different dependencies in CI. The experiment retains its baseline project/lock hashes, exact wheel and `locked-then-sdk-wheel-v1` recipe, plus the disclosed SDK-only replacement. It establishes compatibility of those inputs, not adoption by the unchanged SRM. ## Explicit adoption An open SRM MR selection uses `SDK_CONSUMER_MODE=adoption`, also the fail-closed default when the mode is omitted. The SRM MR must explicitly adopt the candidate. Use the SDK pipeline's `candidate-package.json`: set the exact SDK version in SRM's project requirements and its content-addressed URL under `[tool.uv.sources]`. Run `uv lock` as a developer, inspect the changes and commit both dependency files. Loading @@ -17,7 +37,7 @@ normal explicit SDK PyPI index and commit its regenerated lock. Qualification also accepts that form when the declared version and locked wheel hash match the exact SDK under test. A different wheel or index remains an error. The `sdk-consumer` job downloads the checked artifact and copies the committed In adoption mode, the `sdk-consumer` job downloads the checked artifact and copies the committed `pyproject.toml` and `uv.lock` unchanged into `.cache/sdk-consumer/project`. It rejects an undeclared candidate before installation, then uses `uv sync --locked --extra dev --no-install-project`. No `uv add`, replacement Loading @@ -27,15 +47,21 @@ copied dependency inputs must retain their original hashes after installation. The job checks installed SDK files against the wheel, runs dependency checks, and executes the offline unit/bridge and CI tests without skips. Its artifacts retain the wheel, committed project/lock hashes and qualification. Dependency changes belong in the MR; this job records no implicit dependency changes. changes belong in the MR; this mode records no implicit dependency changes and uses the `committed-lock-v1` recipe. These checks alone do not establish cluster, NEF or OEG acceptance. ## Verified image and replay inputs With `SDK_BUILD_IMAGE=1`, `sdk-consumer-image` builds through the existing Dockerfile using that tested project/lock and the pinned test Python image. Dockerfile using that tested project/lock, the same explicit mode/recipe and the pinned test Python image. Compatibility repeats the SDK-only wheel replacement; adoption installs its committed SDK unchanged. It checks the installed wheel, runtime dependency versions and package metadata before pushing a unique `sdk-candidate-p<PIPELINE>-<SDK-SHA>` image. The image receipt records its immutable digest, both source identities, wheel checksum, project/lock hashes, unchanged-input marker and actual dependencies. Ansible deploys that finished image; it project/lock hashes, mode, recipe, disclosed replacement, unchanged-source and non-SDK dependency markers, and actual dependencies. Ansible deploys that finished image; it performs no further package installation. Normal MR pipelines also build images after their usual checks, using unique Loading @@ -47,8 +73,10 @@ For local offline consumer verification, use Python 3.12 and uv 0.11.26. Set `SDK_WHEEL_FILE`, `SDK_MANIFEST_FILE`, `SDK_SOURCE_SHA`, `SDK_WHEEL_NAME` and `SDK_WHEEL_SHA256`, then run `python ci/sdk_consumer.py` in a clean checkout. Use a disk-backed `TMPDIR` and a fresh `.cache/sdk-consumer` directory. The locked installation creates its own isolated environment from the committed candidate URL; stable SDK publication is not needed first. Supply the matching pinned `CI_JOB_IMAGE` if preparing locked installation creates its own isolated environment. Set `SDK_CONSUMER_MODE=compatibility` for an SDK-only experiment, or use adoption with an already committed candidate URL/index and matching lock. Stable SDK publication is not needed first. Supply the matching pinned `CI_JOB_IMAGE` if preparing inputs for the CI image build. SDK must admit SRM job tokens for exact build-artifact downloads. SRM admits the Loading ci/sdk_consumer.py +134 −8 Changes for ci/sdk_consumer.py: 134 added lines, 8 removed lines. Original line number Diff line number Diff line """Qualify an exact SDK wheel and build its SRM consumer with the same lock.""" """Test SDK compatibility or committed adoption, preserving exact image inputs.""" import argparse import hashlib Loading Loading @@ -114,13 +114,15 @@ def candidate(out, environment, fetch=urlopen): def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=urlopen): environment = dict(os.environ if environment is None else environment) source = revision(root) mode = environment.get("SDK_CONSUMER_MODE", "adoption") require(mode in {"compatibility", "adoption"}, "Unknown SDK consumer mode") require(not (out / "project").exists(), "Use a fresh SDK consumer output directory") out.mkdir(parents=True, exist_ok=True) sdk = candidate(out, environment, fetch) project = out / "project" for name in ("pyproject.toml", "uv.lock", "README.md"): shutil.copyfile(root / name, project / name) # The SRM MR owns adoption. CI only materializes its committed inputs. # Both recipes start with the committed lock; only compatibility replaces SDK. declaration = tomllib.loads((root / "pyproject.toml").read_text()) lock = tomllib.loads((root / "uv.lock").read_text()) packages = [p for p in lock["package"] if p["name"] == "openop-tfsdk"] Loading @@ -146,7 +148,7 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url ) and any(isinstance(p.get("url"), str) for p in locked_wheels) ) require( adopted = ( package.get("version") == sdk["version"] and any( re.fullmatch( Loading @@ -155,7 +157,11 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url for item in declaration.get("project", {}).get("dependencies", []) ) and (candidate_pin or stable_pin) and locked_wheels, and bool(locked_wheels) ) if mode == "adoption": require( adopted, "SDK candidate is not pinned by the committed SRM dependency and lock; " "update them in an SRM MR before qualification", ) Loading @@ -180,8 +186,15 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url "python_image": environment.get("CI_JOB_IMAGE"), "lock_sha256": digest(project / "uv.lock"), "project_sha256": digest(project / "pyproject.toml"), "qualification_mode": mode, "install_recipe": "committed-lock-v1" if mode == "adoption" else "locked-then-sdk-wheel-v1", "baseline_sdk_version": package["version"], } changes = ( replace_sdk(project, sdk, package["version"], call, out, proof) if mode == "compatibility" else [] ) require( all( digest(project / name) == value == digest(root / name) Loading @@ -189,10 +202,19 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url ), "Strict consumer installation changed committed dependency inputs", ) proof["dependency_changes"] = [] proof["dependency_changes"] = changes proof["source_inputs_unchanged"] = True proof["sdk_dependency_kind"] = "candidate-url" if candidate_pin else "pypi" proof["sdk_dependency_url"] = expected_url if candidate_pin else locked_wheels[0]["url"] proof["non_sdk_dependencies_unchanged"] = True proof["sdk_dependency_kind"] = ( "wheel-overlay" if mode == "compatibility" else "candidate-url" if candidate_pin else "pypi" ) proof["sdk_dependency_url"] = ( "wheels/" + sdk["filename"] if mode == "compatibility" else expected_url if candidate_pin else locked_wheels[0]["url"] ) write(out / "candidate.json", proof) return proof Loading @@ -211,6 +233,101 @@ print(json.dumps({'version':sdk.version,'files':files, """ def replace_sdk(project, sdk, baseline_version, call, out, proof): python = str(project / ".venv/bin/python") def inventory(): return json.loads( call([python, "-c", INVENTORY], check=True, capture_output=True, text=True).stdout ) before = inventory() require( before["version"] == baseline_version, "Installed baseline differs from the committed lock" ) call( [ "uv", "pip", "install", "--python", python, "--no-deps", "--no-index", "--reinstall", str(project / "wheels" / sdk["filename"]), ], check=True, ) after = inventory() check_package(after, project / "wheels" / sdk["filename"]) require(after["version"] == sdk["version"], "Installed SDK version differs") def remaining(value): return sorted(tuple(x) for x in value["dependencies"] if x[0] != "openop-tfsdk") require( remaining(before) == remaining(after), "Other SRM dependencies changed during compatibility check", ) checked = call( ["uv", "pip", "check", "--python", python], capture_output=True, text=True, check=False ) if checked.returncode: message = ( "SDK compatibility requires an explicit SRM dependency update; " "no other dependencies were changed" ) write( out / "failure.json", { **proof, "status": "dependency-update-required", "reason": message, "details": checked.stdout + checked.stderr, }, ) raise ValueError(message) return [ { "name": "openop-tfsdk", "from_version": baseline_version, "to_version": sdk["version"], "wheel_sha256": sdk["sha256"], } ] def valid_recipe(proof): if ( proof.get("non_sdk_dependencies_unchanged") is not True or proof.get("source_inputs_unchanged") is not True or not isinstance(proof.get("baseline_sdk_version"), str) ): return False mode = proof.get("qualification_mode") if mode == "adoption": return ( proof.get("install_recipe") == "committed-lock-v1" and proof.get("dependency_changes") == [] ) return ( mode == "compatibility" and proof.get("install_recipe") == "locked-then-sdk-wheel-v1" and proof.get("non_sdk_dependencies_unchanged") is True and isinstance(proof.get("baseline_sdk_version"), str) and proof.get("dependency_changes") == [ { "name": "openop-tfsdk", "from_version": proof["baseline_sdk_version"], "to_version": proof["sdk"]["version"], "wheel_sha256": proof["sdk"]["sha256"], } ] ) def check_package(proof, wheel): with zipfile.ZipFile(wheel) as contents: expected = { Loading Loading @@ -282,6 +399,7 @@ def verify_inputs(root, out): and digest(project / "wheels" / proof["sdk"]["filename"]) == proof["sdk"]["sha256"], "Candidate inputs changed after consumer tests", ) require(valid_recipe(proof), "Consumer installation recipe differs from its qualified mode") return proof Loading Loading @@ -311,6 +429,10 @@ def build_image(root=ROOT, out=OUT, call=subprocess.run): "DEPENDENCY_PROJECT=" + context, "--build-arg", "PYTHON_IMAGE=" + proof["python_image"], "--build-arg", "SDK_CONSUMER_MODE=" + proof["qualification_mode"], "--build-arg", "SDK_WHEEL_NAME=" + proof["sdk"]["filename"], "-t", tag, ".", Loading Loading @@ -396,6 +518,10 @@ def build_image(root=ROOT, out=OUT, call=subprocess.run): "sdk_dependency_url": proof["sdk_dependency_url"], "sdk_dependency_kind": proof["sdk_dependency_kind"], "project_sha256": proof["project_sha256"], "qualification_mode": proof["qualification_mode"], "install_recipe": proof["install_recipe"], "baseline_sdk_version": proof["baseline_sdk_version"], "non_sdk_dependencies_unchanged": proof["non_sdk_dependencies_unchanged"], "runtime_dependencies": inventory["dependencies"], "consumer_checks": {"job_id": proof["qualification_job_id"], "status": "passed"}, } Loading ci/tests/test_sdk_consumer.py +75 −0 Changes for ci/tests/test_sdk_consumer.py: 75 added lines, 0 removed lines. Original line number Diff line number Diff line Loading @@ -5,6 +5,7 @@ import json import sys import zipfile from pathlib import Path from types import SimpleNamespace import pytest Loading Loading @@ -189,6 +190,80 @@ def test_install_cannot_rewrite_the_copied_lock(wheel, tmp_path, monkeypatch): consumer.prepare(root, root / "out", env, call=changing_install) @pytest.mark.parametrize("fault", [None, "dependency-check", "other-version"]) def test_compatibility_replaces_only_sdk_without_requiring_adoption( wheel, tmp_path, monkeypatch, fault ): root = tmp_path / "srm" root.mkdir() env = {**committed_project(root, wheel, old=True), "SDK_CONSUMER_MODE": "compatibility"} original = {n: (root / n).read_bytes() for n in ("pyproject.toml", "uv.lock")} monkeypatch.setattr(consumer, "revision", lambda *_: "a" * 40) calls = [] inventories = iter( [ {"version": "1.0.0", "dependencies": [["openop-tfsdk", "1.0.0"], ["other", "3.0"]]}, { "version": "2.0.0", "files": {"__init__.py": hashlib.sha256(b"value=1\n").hexdigest()}, "dependencies": [ ["openop-tfsdk", "2.0.0"], ["other", "4.0" if fault == "other-version" else "3.0"], ], }, ] ) def call(args, **kwargs): calls.append(args) if "-c" in args: return SimpleNamespace(stdout=json.dumps(next(inventories))) if args[:3] == ["uv", "pip", "check"]: return SimpleNamespace( returncode=1 if fault == "dependency-check" else 0, stdout="SDK requires a newer dependency" if fault else "", stderr="", ) return SimpleNamespace(returncode=0) if fault: with pytest.raises(ValueError, match="dependency update|Other SRM dependencies changed"): consumer.prepare(root, root / "out", env, call=call) assert not (root / "out/qualification.json").exists() if fault == "dependency-check": assert ( json.loads((root / "out/failure.json").read_text())["status"] == "dependency-update-required" ) else: proof = consumer.prepare(root, root / "out", env, call=call) assert proof["qualification_mode"] == "compatibility" assert proof["non_sdk_dependencies_unchanged"] is True assert proof["install_recipe"] == "locked-then-sdk-wheel-v1" assert proof["dependency_changes"] == [ { "name": "openop-tfsdk", "from_version": "1.0.0", "to_version": "2.0.0", "wheel_sha256": env["SDK_WHEEL_SHA256"], } ] replacement = next(args for args in calls if args[:3] == ["uv", "pip", "install"]) assert "--no-deps" in replacement and "--no-index" in replacement assert all((root / n).read_bytes() == value for n, value in original.items()) assert all((root / "out/project" / n).read_bytes() == value for n, value in original.items()) assert not any("add" in args or "lock" in args for args in calls) def test_unknown_consumer_mode_is_rejected_before_install(wheel, tmp_path, monkeypatch): root = tmp_path / "srm" root.mkdir() env = {**committed_project(root, wheel), "SDK_CONSUMER_MODE": "anything"} monkeypatch.setattr(consumer, "revision", lambda *_: "a" * 40) with pytest.raises(ValueError, match="consumer mode"): consumer.prepare(root, root / "out", env, call=lambda *_a, **_kw: pytest.fail("No install")) @pytest.mark.parametrize("fault", [None, "wheel", "index"]) def test_stable_index_adoption_accepts_only_the_same_locked_wheel( wheel, tmp_path, monkeypatch, fault Loading Loading
.gitlab-ci.yml +1 −0 Changes for .gitlab-ci.yml: 1 added line, 0 removed lines. Original line number Diff line number Diff line Loading @@ -130,6 +130,7 @@ sdk-consumer: when: always paths: - .cache/sdk-consumer/qualification.json - .cache/sdk-consumer/failure.json - .cache/sdk-consumer/project/pyproject.toml - .cache/sdk-consumer/project/uv.lock - .cache/sdk-consumer/project/wheels/ Loading
Dockerfile +8 −1 Changes for Dockerfile: 8 added lines, 1 removed line. Original line number Diff line number Diff line Loading @@ -12,6 +12,8 @@ ENV PYTHONDONTWRITEBYTECODE=1 \ PIP_DEFAULT_TIMEOUT=120 ARG DEPENDENCY_PROJECT=. ARG SDK_CONSUMER_MODE=adoption ARG SDK_WHEEL_NAME COPY ${DEPENDENCY_PROJECT}/pyproject.toml ./ COPY ${DEPENDENCY_PROJECT}/uv.lock ./ Loading @@ -20,7 +22,12 @@ RUN --mount=type=bind,target=/build-context \ uv sync --locked; \ else \ cp -r "/build-context/$DEPENDENCY_PROJECT/wheels" ./wheels && \ uv sync --locked --no-install-project; \ uv sync --locked --no-install-project && \ case "$SDK_CONSUMER_MODE" in \ adoption) ;; \ compatibility) uv pip install --python /app/.venv/bin/python --no-deps --no-index --reinstall "/app/wheels/$SDK_WHEEL_NAME" ;; \ *) exit 1 ;; \ esac; \ fi FROM ${PYTHON_IMAGE} AS runtime Loading
ci/sdk-consumer.md +35 −7 Changes for ci/sdk-consumer.md: 35 added lines, 7 removed lines. Original line number Diff line number Diff line Loading @@ -5,7 +5,27 @@ merges. It sends one wheel's build job, source revision and checksum. SRM checks its own checkout against `EXPECTED_SRM_SHA` before installing anything. A changed branch head requires a new run. Before selection, the SRM MR must explicitly adopt the candidate. Use the SDK ## Compatibility without adoption SDK target-branch checks and branch experiments set `SDK_CONSUMER_MODE=compatibility`. No SRM MR or SDK pin update is required. The job copies the committed dependency files unchanged and installs their lock. It then replaces only SDK with the exact candidate wheel, using no index and no dependency installation. Every other installed package/version must remain unchanged. Installed SDK contents and all package requirements are checked. An incompatible dependency baseline produces `failure.json` with status `dependency-update-required`; no successful qualification or image is produced. Prepare explicit adoption rather than resolving different dependencies in CI. The experiment retains its baseline project/lock hashes, exact wheel and `locked-then-sdk-wheel-v1` recipe, plus the disclosed SDK-only replacement. It establishes compatibility of those inputs, not adoption by the unchanged SRM. ## Explicit adoption An open SRM MR selection uses `SDK_CONSUMER_MODE=adoption`, also the fail-closed default when the mode is omitted. The SRM MR must explicitly adopt the candidate. Use the SDK pipeline's `candidate-package.json`: set the exact SDK version in SRM's project requirements and its content-addressed URL under `[tool.uv.sources]`. Run `uv lock` as a developer, inspect the changes and commit both dependency files. Loading @@ -17,7 +37,7 @@ normal explicit SDK PyPI index and commit its regenerated lock. Qualification also accepts that form when the declared version and locked wheel hash match the exact SDK under test. A different wheel or index remains an error. The `sdk-consumer` job downloads the checked artifact and copies the committed In adoption mode, the `sdk-consumer` job downloads the checked artifact and copies the committed `pyproject.toml` and `uv.lock` unchanged into `.cache/sdk-consumer/project`. It rejects an undeclared candidate before installation, then uses `uv sync --locked --extra dev --no-install-project`. No `uv add`, replacement Loading @@ -27,15 +47,21 @@ copied dependency inputs must retain their original hashes after installation. The job checks installed SDK files against the wheel, runs dependency checks, and executes the offline unit/bridge and CI tests without skips. Its artifacts retain the wheel, committed project/lock hashes and qualification. Dependency changes belong in the MR; this job records no implicit dependency changes. changes belong in the MR; this mode records no implicit dependency changes and uses the `committed-lock-v1` recipe. These checks alone do not establish cluster, NEF or OEG acceptance. ## Verified image and replay inputs With `SDK_BUILD_IMAGE=1`, `sdk-consumer-image` builds through the existing Dockerfile using that tested project/lock and the pinned test Python image. Dockerfile using that tested project/lock, the same explicit mode/recipe and the pinned test Python image. Compatibility repeats the SDK-only wheel replacement; adoption installs its committed SDK unchanged. It checks the installed wheel, runtime dependency versions and package metadata before pushing a unique `sdk-candidate-p<PIPELINE>-<SDK-SHA>` image. The image receipt records its immutable digest, both source identities, wheel checksum, project/lock hashes, unchanged-input marker and actual dependencies. Ansible deploys that finished image; it project/lock hashes, mode, recipe, disclosed replacement, unchanged-source and non-SDK dependency markers, and actual dependencies. Ansible deploys that finished image; it performs no further package installation. Normal MR pipelines also build images after their usual checks, using unique Loading @@ -47,8 +73,10 @@ For local offline consumer verification, use Python 3.12 and uv 0.11.26. Set `SDK_WHEEL_FILE`, `SDK_MANIFEST_FILE`, `SDK_SOURCE_SHA`, `SDK_WHEEL_NAME` and `SDK_WHEEL_SHA256`, then run `python ci/sdk_consumer.py` in a clean checkout. Use a disk-backed `TMPDIR` and a fresh `.cache/sdk-consumer` directory. The locked installation creates its own isolated environment from the committed candidate URL; stable SDK publication is not needed first. Supply the matching pinned `CI_JOB_IMAGE` if preparing locked installation creates its own isolated environment. Set `SDK_CONSUMER_MODE=compatibility` for an SDK-only experiment, or use adoption with an already committed candidate URL/index and matching lock. Stable SDK publication is not needed first. Supply the matching pinned `CI_JOB_IMAGE` if preparing inputs for the CI image build. SDK must admit SRM job tokens for exact build-artifact downloads. SRM admits the Loading
ci/sdk_consumer.py +134 −8 Changes for ci/sdk_consumer.py: 134 added lines, 8 removed lines. Original line number Diff line number Diff line """Qualify an exact SDK wheel and build its SRM consumer with the same lock.""" """Test SDK compatibility or committed adoption, preserving exact image inputs.""" import argparse import hashlib Loading Loading @@ -114,13 +114,15 @@ def candidate(out, environment, fetch=urlopen): def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=urlopen): environment = dict(os.environ if environment is None else environment) source = revision(root) mode = environment.get("SDK_CONSUMER_MODE", "adoption") require(mode in {"compatibility", "adoption"}, "Unknown SDK consumer mode") require(not (out / "project").exists(), "Use a fresh SDK consumer output directory") out.mkdir(parents=True, exist_ok=True) sdk = candidate(out, environment, fetch) project = out / "project" for name in ("pyproject.toml", "uv.lock", "README.md"): shutil.copyfile(root / name, project / name) # The SRM MR owns adoption. CI only materializes its committed inputs. # Both recipes start with the committed lock; only compatibility replaces SDK. declaration = tomllib.loads((root / "pyproject.toml").read_text()) lock = tomllib.loads((root / "uv.lock").read_text()) packages = [p for p in lock["package"] if p["name"] == "openop-tfsdk"] Loading @@ -146,7 +148,7 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url ) and any(isinstance(p.get("url"), str) for p in locked_wheels) ) require( adopted = ( package.get("version") == sdk["version"] and any( re.fullmatch( Loading @@ -155,7 +157,11 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url for item in declaration.get("project", {}).get("dependencies", []) ) and (candidate_pin or stable_pin) and locked_wheels, and bool(locked_wheels) ) if mode == "adoption": require( adopted, "SDK candidate is not pinned by the committed SRM dependency and lock; " "update them in an SRM MR before qualification", ) Loading @@ -180,8 +186,15 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url "python_image": environment.get("CI_JOB_IMAGE"), "lock_sha256": digest(project / "uv.lock"), "project_sha256": digest(project / "pyproject.toml"), "qualification_mode": mode, "install_recipe": "committed-lock-v1" if mode == "adoption" else "locked-then-sdk-wheel-v1", "baseline_sdk_version": package["version"], } changes = ( replace_sdk(project, sdk, package["version"], call, out, proof) if mode == "compatibility" else [] ) require( all( digest(project / name) == value == digest(root / name) Loading @@ -189,10 +202,19 @@ def prepare(root=ROOT, out=OUT, environment=None, call=subprocess.run, fetch=url ), "Strict consumer installation changed committed dependency inputs", ) proof["dependency_changes"] = [] proof["dependency_changes"] = changes proof["source_inputs_unchanged"] = True proof["sdk_dependency_kind"] = "candidate-url" if candidate_pin else "pypi" proof["sdk_dependency_url"] = expected_url if candidate_pin else locked_wheels[0]["url"] proof["non_sdk_dependencies_unchanged"] = True proof["sdk_dependency_kind"] = ( "wheel-overlay" if mode == "compatibility" else "candidate-url" if candidate_pin else "pypi" ) proof["sdk_dependency_url"] = ( "wheels/" + sdk["filename"] if mode == "compatibility" else expected_url if candidate_pin else locked_wheels[0]["url"] ) write(out / "candidate.json", proof) return proof Loading @@ -211,6 +233,101 @@ print(json.dumps({'version':sdk.version,'files':files, """ def replace_sdk(project, sdk, baseline_version, call, out, proof): python = str(project / ".venv/bin/python") def inventory(): return json.loads( call([python, "-c", INVENTORY], check=True, capture_output=True, text=True).stdout ) before = inventory() require( before["version"] == baseline_version, "Installed baseline differs from the committed lock" ) call( [ "uv", "pip", "install", "--python", python, "--no-deps", "--no-index", "--reinstall", str(project / "wheels" / sdk["filename"]), ], check=True, ) after = inventory() check_package(after, project / "wheels" / sdk["filename"]) require(after["version"] == sdk["version"], "Installed SDK version differs") def remaining(value): return sorted(tuple(x) for x in value["dependencies"] if x[0] != "openop-tfsdk") require( remaining(before) == remaining(after), "Other SRM dependencies changed during compatibility check", ) checked = call( ["uv", "pip", "check", "--python", python], capture_output=True, text=True, check=False ) if checked.returncode: message = ( "SDK compatibility requires an explicit SRM dependency update; " "no other dependencies were changed" ) write( out / "failure.json", { **proof, "status": "dependency-update-required", "reason": message, "details": checked.stdout + checked.stderr, }, ) raise ValueError(message) return [ { "name": "openop-tfsdk", "from_version": baseline_version, "to_version": sdk["version"], "wheel_sha256": sdk["sha256"], } ] def valid_recipe(proof): if ( proof.get("non_sdk_dependencies_unchanged") is not True or proof.get("source_inputs_unchanged") is not True or not isinstance(proof.get("baseline_sdk_version"), str) ): return False mode = proof.get("qualification_mode") if mode == "adoption": return ( proof.get("install_recipe") == "committed-lock-v1" and proof.get("dependency_changes") == [] ) return ( mode == "compatibility" and proof.get("install_recipe") == "locked-then-sdk-wheel-v1" and proof.get("non_sdk_dependencies_unchanged") is True and isinstance(proof.get("baseline_sdk_version"), str) and proof.get("dependency_changes") == [ { "name": "openop-tfsdk", "from_version": proof["baseline_sdk_version"], "to_version": proof["sdk"]["version"], "wheel_sha256": proof["sdk"]["sha256"], } ] ) def check_package(proof, wheel): with zipfile.ZipFile(wheel) as contents: expected = { Loading Loading @@ -282,6 +399,7 @@ def verify_inputs(root, out): and digest(project / "wheels" / proof["sdk"]["filename"]) == proof["sdk"]["sha256"], "Candidate inputs changed after consumer tests", ) require(valid_recipe(proof), "Consumer installation recipe differs from its qualified mode") return proof Loading Loading @@ -311,6 +429,10 @@ def build_image(root=ROOT, out=OUT, call=subprocess.run): "DEPENDENCY_PROJECT=" + context, "--build-arg", "PYTHON_IMAGE=" + proof["python_image"], "--build-arg", "SDK_CONSUMER_MODE=" + proof["qualification_mode"], "--build-arg", "SDK_WHEEL_NAME=" + proof["sdk"]["filename"], "-t", tag, ".", Loading Loading @@ -396,6 +518,10 @@ def build_image(root=ROOT, out=OUT, call=subprocess.run): "sdk_dependency_url": proof["sdk_dependency_url"], "sdk_dependency_kind": proof["sdk_dependency_kind"], "project_sha256": proof["project_sha256"], "qualification_mode": proof["qualification_mode"], "install_recipe": proof["install_recipe"], "baseline_sdk_version": proof["baseline_sdk_version"], "non_sdk_dependencies_unchanged": proof["non_sdk_dependencies_unchanged"], "runtime_dependencies": inventory["dependencies"], "consumer_checks": {"job_id": proof["qualification_job_id"], "status": "passed"}, } Loading
ci/tests/test_sdk_consumer.py +75 −0 Changes for ci/tests/test_sdk_consumer.py: 75 added lines, 0 removed lines. Original line number Diff line number Diff line Loading @@ -5,6 +5,7 @@ import json import sys import zipfile from pathlib import Path from types import SimpleNamespace import pytest Loading Loading @@ -189,6 +190,80 @@ def test_install_cannot_rewrite_the_copied_lock(wheel, tmp_path, monkeypatch): consumer.prepare(root, root / "out", env, call=changing_install) @pytest.mark.parametrize("fault", [None, "dependency-check", "other-version"]) def test_compatibility_replaces_only_sdk_without_requiring_adoption( wheel, tmp_path, monkeypatch, fault ): root = tmp_path / "srm" root.mkdir() env = {**committed_project(root, wheel, old=True), "SDK_CONSUMER_MODE": "compatibility"} original = {n: (root / n).read_bytes() for n in ("pyproject.toml", "uv.lock")} monkeypatch.setattr(consumer, "revision", lambda *_: "a" * 40) calls = [] inventories = iter( [ {"version": "1.0.0", "dependencies": [["openop-tfsdk", "1.0.0"], ["other", "3.0"]]}, { "version": "2.0.0", "files": {"__init__.py": hashlib.sha256(b"value=1\n").hexdigest()}, "dependencies": [ ["openop-tfsdk", "2.0.0"], ["other", "4.0" if fault == "other-version" else "3.0"], ], }, ] ) def call(args, **kwargs): calls.append(args) if "-c" in args: return SimpleNamespace(stdout=json.dumps(next(inventories))) if args[:3] == ["uv", "pip", "check"]: return SimpleNamespace( returncode=1 if fault == "dependency-check" else 0, stdout="SDK requires a newer dependency" if fault else "", stderr="", ) return SimpleNamespace(returncode=0) if fault: with pytest.raises(ValueError, match="dependency update|Other SRM dependencies changed"): consumer.prepare(root, root / "out", env, call=call) assert not (root / "out/qualification.json").exists() if fault == "dependency-check": assert ( json.loads((root / "out/failure.json").read_text())["status"] == "dependency-update-required" ) else: proof = consumer.prepare(root, root / "out", env, call=call) assert proof["qualification_mode"] == "compatibility" assert proof["non_sdk_dependencies_unchanged"] is True assert proof["install_recipe"] == "locked-then-sdk-wheel-v1" assert proof["dependency_changes"] == [ { "name": "openop-tfsdk", "from_version": "1.0.0", "to_version": "2.0.0", "wheel_sha256": env["SDK_WHEEL_SHA256"], } ] replacement = next(args for args in calls if args[:3] == ["uv", "pip", "install"]) assert "--no-deps" in replacement and "--no-index" in replacement assert all((root / n).read_bytes() == value for n, value in original.items()) assert all((root / "out/project" / n).read_bytes() == value for n, value in original.items()) assert not any("add" in args or "lock" in args for args in calls) def test_unknown_consumer_mode_is_rejected_before_install(wheel, tmp_path, monkeypatch): root = tmp_path / "srm" root.mkdir() env = {**committed_project(root, wheel), "SDK_CONSUMER_MODE": "anything"} monkeypatch.setattr(consumer, "revision", lambda *_: "a" * 40) with pytest.raises(ValueError, match="consumer mode"): consumer.prepare(root, root / "out", env, call=lambda *_a, **_kw: pytest.fail("No install")) @pytest.mark.parametrize("fault", [None, "wheel", "index"]) def test_stable_index_adoption_accepts_only_the_same_locked_wheel( wheel, tmp_path, monkeypatch, fault Loading
