Commit 6f191e7e authored by Lluis Gifre Renom's avatar Lluis Gifre Renom
Browse files

Deployment scripts:

- QuestDB and Grafana only deployed if Monitoring/Slice is deployed
- Grafana links only shown in WebUI if Grafana deployed
- Separated WebUI/Grafana manifests
parent 64ac30ce
Loading
Loading
Loading
Loading

deploy/README.md

0 → 100644
+40 −0
Changes for deploy/README.md: 40 added lines, 0 removed lines.
Original line number Diff line number Diff line
# Optional Monitoring Infrastructure

QuestDB and the bundled Grafana instance are deployed only when
`TFS_COMPONENTS` includes the complete token `monitoring` or `slice`.
Both are legacy components. Selecting only core, optical, MCP or Agentic
components does not deploy this infrastructure.
The fallback component list in `deploy/all.sh` still includes `slice` for
compatibility; select an explicit component list without `slice` or
`monitoring` for a lightweight deployment, as the ECOC demo specs do.

`manifests/webuiservice.yaml` contains only the WebUI.
`manifests/grafanaservice.yaml` contains a separate Grafana Deployment,
Service, PVC and ingress at `/grafana/`. The deployment script enables the
WebUI navigation link through `TFS_GRAFANA_ENABLED=YES` only in the legacy
monitoring mode, and configures datasources through the Grafana Service.

The `qdb-data` Secret and dashboard exposure/configuration are skipped when
neither legacy consumer is selected. Shared Kubernetes observability
installations are not automatically removed or disabled.
Explicit calls to `deploy/qdb.sh` still deploy QuestDB for operators who need
to manage that infrastructure separately.

Existing QuestDB namespaces/storage and previously deployed Grafana resources
are not pruned automatically. Back up any required data and clean up obsolete
resources explicitly when migrating. Direct manifest users must apply the
new Grafana manifest separately and enable its WebUI link if desired.

For the ECOC demo, the branch-only root `redeploy.sh` delegates to
`src/tests/ecoc26-hyr-agentic-optical/redeploy-all.sh`.
Run `./redeploy.sh --help` before the first reset. This convenience wrapper
is not intended for the upstream merge.

Offline deployment checks:

```bash
python -m unittest discover -s deploy/tests -v
```

These repository-level checks require PyYAML and stay separate from component
unit tests, whose container images do not contain deployment manifests.
+7 −0
Changes for deploy/all.sh: 7 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -257,6 +257,7 @@ export GRAF_EXT_PORT_HTTP=${GRAF_EXT_PORT_HTTP:-"3000"}
########################################################################################################################

bash scripts/dockerhub_login.sh
source deploy/monitoring_dependencies.sh

# Deploy CockroachDB
./deploy/crdb.sh
@@ -265,7 +266,11 @@ bash scripts/dockerhub_login.sh
./deploy/nats.sh

# Deploy QuestDB
if uses_legacy_monitoring; then
    ./deploy/qdb.sh
else
    echo "Skipping QuestDB: neither monitoring nor slice is selected."
fi

# Deploy Apache Kafka
./deploy/kafka.sh
@@ -274,7 +279,9 @@ bash scripts/dockerhub_login.sh
# ./deploy/monitoring.sh

# Expose Dashboard
if uses_legacy_monitoring; then
    ./deploy/expose_dashboard.sh
fi

# Deploy TeraFlowSDN
./deploy/tfs.sh
+5 −0
Changes for deploy/expose_dashboard.sh: 5 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -30,6 +30,11 @@ export GRAF_EXT_PORT_HTTP=${GRAF_EXT_PORT_HTTP:-"3000"}
########################################################################################################################

MONITORING_NAMESPACE="monitoring"
source "$(dirname "${BASH_SOURCE[0]}")/monitoring_dependencies.sh"
if ! uses_legacy_monitoring; then
    echo "Skipping dashboard exposure: monitoring/slice not selected."
    exit 0
fi

function expose_dashboard() {
    echo "Prometheus Port Mapping"
+25 −0
Changes for deploy/monitoring_dependencies.sh: 25 added lines, 0 removed lines.
Original line number Diff line number Diff line
#!/bin/bash
# Copyright 2022-2026 ETSI SDG TeraFlowSDN (TFS) (https://tfs.etsi.org/)
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Source from deployment scripts; match complete component names.
uses_legacy_monitoring() {
    local component
    for component in ${TFS_COMPONENTS:-}; do
        case "$component" in
            monitoring|slice) return 0 ;;
        esac
    done
    return 1
}
+125 −0
Changes for deploy/tests/test_monitoring_dependencies.py: 125 added lines, 0 removed lines.
Original line number Diff line number Diff line
# Copyright 2022-2026 ETSI SDG TeraFlowSDN (TFS) (https://tfs.etsi.org/)
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Offline checks for optional legacy monitoring infrastructure."""

from pathlib import Path
import shutil
import subprocess
import tempfile
import unittest

import yaml


ROOT = Path(__file__).resolve().parents[2]


class MonitoringDependenciesTests(unittest.TestCase):
    def test_all_only_invokes_monitoring_dependencies_when_selected(self):
        with tempfile.TemporaryDirectory() as directory:
            root = Path(directory)
            (root / "deploy").mkdir()
            (root / "scripts").mkdir()
            for filename in ("all.sh", "monitoring_dependencies.sh"):
                shutil.copy(ROOT / "deploy" / filename, root / "deploy")
            (root / "scripts/dockerhub_login.sh").write_text("exit 0\n")
            for name in ("crdb", "nats", "qdb", "kafka",
                         "expose_dashboard", "tfs", "show"):
                script = root / "deploy" / (name + ".sh")
                script.write_text('#!/bin/bash\necho ' + name + ' >> calls\n')
                script.chmod(0o755)
            for components, expected in [
                ("context device webui", False),
                ("context monitoring webui", True),
                ("context slice webui", True),
            ]:
                with self.subTest(components=components):
                    (root / "calls").write_text("")
                    result = subprocess.run(
                        ["bash", "deploy/all.sh"], cwd=root,
                        env={"PATH": "/usr/bin:/bin",
                             "TFS_COMPONENTS": components},
                        capture_output=True, text=True,
                    )
                    self.assertEqual(result.returncode, 0, result.stderr)
                    calls = (root / "calls").read_text().splitlines()
                    for dependency in ("qdb", "expose_dashboard"):
                        self.assertEqual(dependency in calls, expected)
                    self.assertIn("tfs", calls)

    def test_exact_component_membership(self):
        for components, enabled in [
            ("context device webui", False), ("monitoring", True),
            ("slice", True), ("context slice webui", True),
            ("monitoring slice", True), ("kpi_manager telemetry", False),
            ("monitoring_extra slice_extra", False), ("", False),
        ]:
            with self.subTest(components=components):
                result = subprocess.run(
                    ["bash", "-c",
                     'source deploy/monitoring_dependencies.sh; '
                     'TFS_COMPONENTS="$1"; uses_legacy_monitoring',
                     "_", components],
                    cwd=ROOT, check=False,
                )
                self.assertEqual(result.returncode == 0, enabled)

    def test_webui_has_no_grafana_resources(self):
        docs = list(yaml.safe_load_all(
            (ROOT / "manifests/webuiservice.yaml").read_text()
        ))
        self.assertFalse(any(d["kind"] == "PersistentVolumeClaim"
                             for d in docs))
        deployment = next(d for d in docs if d["kind"] == "Deployment")
        spec = deployment["spec"]["template"]["spec"]
        self.assertEqual([c["name"] for c in spec["containers"]], ["server"])
        self.assertNotIn("volumes", spec)
        service = next(d for d in docs if d["kind"] == "Service")
        self.assertEqual([p["port"] for p in service["spec"]["ports"]], [8004])

    def test_grafana_is_self_contained(self):
        docs = list(yaml.safe_load_all(
            (ROOT / "manifests/grafanaservice.yaml").read_text()
        ))
        self.assertEqual([d["kind"] for d in docs], [
            "PersistentVolumeClaim", "Deployment", "Service", "Ingress",
        ])
        pvc, deployment, service, ingress = docs
        spec = deployment["spec"]["template"]["spec"]
        self.assertEqual(
            spec["volumes"][0]["persistentVolumeClaim"]["claimName"],
            pvc["metadata"]["name"],
        )
        self.assertEqual(service["spec"]["selector"],
                         deployment["spec"]["selector"]["matchLabels"])
        backend = ingress["spec"]["rules"][0]["http"]["paths"][0]["backend"]
        self.assertEqual(backend["service"]["name"],
                         service["metadata"]["name"])
        self.assertEqual(backend["service"]["port"]["number"], 3000)
        base = (ROOT / "manifests/nginx_ingress_http.yaml").read_text()
        self.assertNotIn("/grafana", base)

    def test_dashboard_exposure_disabled_without_cluster_access(self):
        result = subprocess.run(
            ["bash", "deploy/expose_dashboard.sh"], cwd=ROOT,
            env={"PATH": "/usr/bin:/bin", "TFS_COMPONENTS": "context webui"},
            capture_output=True, text=True,
        )
        self.assertEqual(result.returncode, 0, result.stderr)
        self.assertIn("Skipping dashboard exposure", result.stdout)


if __name__ == "__main__":
    unittest.main()
Loading