test: add ordered ECOC optical demo redeployment helpers

    echo "Waiting for '$COMPONENT' component..."

    COMPONENT_OBJNAME=$(echo "${COMPONENT}" | sed "s/\_/-/g")

    kubectl wait --namespace $TFS_K8S_NAMESPACE \

        --for='condition=available' --timeout=90s deployment/${COMPONENT_OBJNAME}service

        --for='condition=available' \

        --timeout="${TFS_ROLLOUT_TIMEOUT_SECONDS:-90}s" \

        deployment/${COMPONENT_OBJNAME}service

    WAIT_EXIT_CODE=$?

    if [[ $WAIT_EXIT_CODE != 0 ]]; then

        echo "  Failed to deploy '${COMPONENT}' component, exit code '${WAIT_EXIT_CODE}', exiting..."

## Deploy

From the controller repository, launch the node agents before onboarding:

### Prerequisites

Use a dedicated demo machine with Docker, MicroK8s, and the normal TFS

deployment dependencies installed (including kubectl, Helm, jq and curl).

The scripts target the MicroK8s 1.29 nginx-ingress deployment used by this

demo; a different ingress addon requires adapting the readiness check.

The current user must be able to run Docker and query MicroK8s. Starting a

stopped MicroK8s installation may request sudo authentication; for unattended

execution, start it interactively first. Already-running instances are reused.

Activate the Python environment used for TFS development; it must contain

the generated TFS protobuf modules, scenario-loader dependencies and PyYAML.

For example, on a machine with a prepared pyenv environment:

```bash

pyenv shell tfs

```

Internet access is required for container images and the configured LLM.

Ports 80 and 32000 must be available for ingress and the local registry.

The Docker management subnet `172.254.253.0/24` must not overlap another

network. The descriptor addresses match this subnet.

Store the API key in `$HOME/openai.key`, **outside the repository**, containing

only the key (not a dotenv assignment). This is the default path in

`deploy_specs.sh`; no environment variable is needed. Restrict its permissions:

```bash

chmod 600 "$HOME/openai.key"

```

To use another private file, override `DEMO_API_KEY_FILE` with its absolute

path. Never commit the key file; transfer it separately through a secure channel.

The launcher defaults to `openai/gpt-4.1-mini`. For Anthropic, set

`DEMO_MODEL=anthropic/claude-haiku-4-5-20251001` and point

`DEMO_API_KEY_FILE` to the corresponding private key.

The selected key is injected through kubectl stdin, not command arguments or

a generated manifest file. Never run the helper with shell tracing, and never

commit keys or live transcripts. Directly reapplying the repository's Agentic

manifest replaces the key with its empty placeholder.

### One-Command Full Redeployment

From the controller repository:

```bash

bash src/tests/ecoc26-hyr-agentic-optical/redeploy-all.sh

```

The script prints a destructive-reset warning and asks you to type `RESET`.

Use `--yes` only when intentionally resetting a dedicated demo machine.

It runs the following ordered stages:

1. Remove only the four named node-agent containers and their `na-br`

   network, pull the demo images, and recreate them.

2. Wait for ConfD startup and a reachable NETCONF port on every node agent.

3. Start local MicroK8s and enable DNS, hostpath storage, registry and ingress.

4. Remove the old `tfs` namespace, then rebuild/deploy the controller,

   Optical Controller and MCP Server, resetting the demo Context database.

5. Onboard `descriptors/topology.json` with the standard scenario loader

   and validate the resulting inventory.

6. Build/deploy Agentic with its private key, enable its TFS WebUI link,

   wait for readiness, and check ADK Web access.

**This deletes the TFS demo inventory, services, reservations and Agentic

session storage.** CockroachDB reset follows the upstream deployment script

and drops databases with the `tfs_` prefix, not just a single service.

Do not use this launcher on shared or production infrastructure.

It does not prune unrelated Docker images/containers, reset all MicroK8s

storage, or touch a remote kubectl context. It creates a temporary private

kubeconfig for the local MicroK8s instance and removes it on exit.

The persistent database infrastructure is reused, rather than destroyed.

Initial builds can take several minutes. Stages are printed as `[1/6]`

through `[6/6]`; do not interrupt a build merely because output is quiet.

Underlying TFS build logs are written by the standard deployment scripts.

Errors stop the launcher; partial deployments are retained for diagnosis.

### TFS-Only Redeployment And Settings

To keep the existing node-agent containers and rebuild only TFS:

```bash

bash src/tests/ecoc26-hyr-agentic-optical/deploy-node-agents.sh

source my_deploy.sh

./deploy/all.sh

./src/tests/tools/load_scenario/run.sh \

  src/tests/ecoc26-hyr-agentic-optical/descriptors/topology.json

bash src/tests/ecoc26-hyr-agentic-optical/redeploy.sh

```

Review database reset settings in `my_deploy.sh` before redeploying an existing

controller. The node-agent launcher recreates its four named containers.

Run the scenario loader with the TFS Python dependencies and runtime variables.

This still resets the controller inventory and reloads the topology. It first

checks that all four node agents are ready; it does not reset their device

configuration. Prefer `redeploy-all.sh` for a completely fresh rehearsal.

`deploy_specs.sh` contains the self-contained demo deployment settings;

neither launcher sources `my_deploy.sh`.

`DEMO_PYTHON` selects the Python executable (default `python`).

`DEMO_WAIT_SECONDS` controls Kubernetes waits (default 600), including

the upstream TFS component rollout through `TFS_ROLLOUT_TIMEOUT_SECONDS`, and

`NODE_AGENT_WAIT_SECONDS` bounds node-agent initialization (default 300).

Set `TFS_SKIP_BUILD=YES` only when all required images already exist in

the local registry with `TFS_IMAGE_TAG` (default `dev`).

If a stage fails, check `docker logs na-t1` (or the named failing agent),

`microk8s kubectl -n tfs get pods`, and the relevant component logs.

For Agentic startup, verify the key file permissions and model configuration.

After fixing the cause, rerun the launcher; do not manually clear occupied

spectrum while services are active.

Set `ADK_MODEL` to `openai/gpt-4.1-mini` in the Agentic

ConfigMap, leave `ADK_PEERS` empty, and populate `OPENAI_API_KEY` in

`agentic-secrets` from a private environment file. Anthropic is also supported

with `anthropic/claude-haiku-4-5-20251001` and `ANTHROPIC_API_KEY`.

The current validation uses OpenAI.

Never put a real API key

in the manifest, topology, transcript, or this directory.

Reapplying the example manifest resets its empty secret placeholders; inject

the key afterwards and restart `deployment/agenticservice`.

Offline launcher regressions exercise command ordering, cancellation,

failure handling and private-key rendering with stubbed external commands:

```bash

python -m unittest discover \

  -s src/tests/ecoc26-hyr-agentic-optical -p test_deployment.py -v

```

These checks do not replace a live deployment rehearsal.

Open `/agentic/dev-ui/` on the controller and choose the `agentic` application.

Use a fresh session when validating prompt changes to avoid stale replies.

## Validation Notes (2026-09-15)

### Full Launcher Validation (2026-09-17)

The full launcher completed on the local MicroK8s demo with OpenAI

`gpt-4.1-mini`: four node agents ready, nine TFS deployments available,

and topology onboarding validated with eight devices and fourteen optical

links. Five read-only ADK Web prompts exercised device types, ROADM

classification, transponder endpoints, optical links, and empty service and

connection inventories. This deployment check did not repeat mutations.

Node readiness checks use the foreground NETCONF application's banner and

its listening port; the shell message after that application is not reached

while it runs. The launcher reuses running MicroK8s and permits longer TFS

rollout waits for cold initialization. The successful retry reused the images

built during the initial deployment attempt with `TFS_SKIP_BUILD=YES`.

### Controller And Conversation Validation

- Local deployment uses the supplied node agents and topology: 8 devices,

  including 2 ROADMs and 6 transponders, and 14 optical links.

- Corrected executable permissions on the mounted node-agent startup scripts;

# See the License for the specific language governing permissions and

# limitations under the License.

set -euo pipefail

SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"

cd "$SCRIPT_DIR/../../.."

TEST_NAME="ecoc26-hyr-agentic-optical"

echo

echo "Clean-up:"

echo "---------"

docker rm -f na-t1 na-t2 na-r1 na-r2

docker network rm na-br

bash "$SCRIPT_DIR/destroy-node-agents.sh"

echo

echo

echo "Waiting for initialization..."

echo "-----------------------------"

docker ps -a

while ! docker logs na-t1 2>&1 | grep -q '*** ConfD OpenConfig NETCONF agent ***'; do sleep 1; done

while ! docker logs na-t2 2>&1 | grep -q '*** ConfD OpenConfig NETCONF agent ***'; do sleep 1; done

while ! docker logs na-r1 2>&1 | grep -q '*** ConfD OpenConfig NETCONF agent ***'; do sleep 1; done

while ! docker logs na-r2 2>&1 | grep -q '*** ConfD OpenConfig NETCONF agent ***'; do sleep 1; done

sleep 3

docker ps -a

bash "$SCRIPT_DIR/wait-node-agents.sh"

echo

#!/bin/bash

# Copyright 2022-2026 ETSI SDG TeraFlowSDN (TFS) (https://tfs.etsi.org/)

#

# Licensed under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.

# You may obtain a copy of the License at

#

#      http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

# Demo-specific overrides; other settings use deploy/all.sh defaults.

# Optical Controller precedes Service; MCP and Agentic follow the core.

export TFS_COMPONENTS="context device pathcomp opticalcontroller service"

export TFS_COMPONENTS="$TFS_COMPONENTS nbi webui mcp_server agentic"

export TFS_K8S_NAMESPACE="tfs"

export TFS_REGISTRY_IMAGES="http://localhost:32000/tfs/"

export TFS_IMAGE_TAG="${TFS_IMAGE_TAG:-dev}"

export TFS_SKIP_BUILD="${TFS_SKIP_BUILD:-}"

export TFS_EXTRA_MANIFESTS="manifests/nginx_ingress_http.yaml"

export CRDB_DEPLOY_MODE="single"

export NATS_DEPLOY_MODE="single"

export KFK_DEPLOY_MODE="single"

export QDB_DEPLOY_MODE="single"

# Fresh demo, not a production upgrade: drops databases starting with tfs_.

export CRDB_DROP_DATABASE_IF_EXISTS="YES"

export QDB_DROP_TABLES_IF_EXIST=""

export CRDB_REDEPLOY=""

export NATS_REDEPLOY=""

export KFK_REDEPLOY=""

export QDB_REDEPLOY=""

export DEMO_MODEL="${DEMO_MODEL:-openai/gpt-4.1-mini}"

# Use an absolute path to a PRIVATE file containing only the API key.

export DEMO_API_KEY_FILE="${DEMO_API_KEY_FILE:-$HOME/openai.key}"

export DEMO_PYTHON="${DEMO_PYTHON:-python}"

export DEMO_WAIT_SECONDS="${DEMO_WAIT_SECONDS:-600}"

export TFS_ROLLOUT_TIMEOUT_SECONDS="$DEMO_WAIT_SECONDS"

export NODE_AGENT_WAIT_SECONDS="${NODE_AGENT_WAIT_SECONDS:-300}"

# See the License for the specific language governing permissions and

# limitations under the License.

set -euo pipefail

docker info >/dev/null

echo

echo "Clean-up:"

echo "---------"

docker rm -f na-t1 na-t2 na-r1 na-r2

for NODE in na-t1 na-t2 na-r1 na-r2; do

    if docker container inspect "$NODE" >/dev/null 2>&1; then

        docker rm -f "$NODE"

    fi

done

if docker network inspect na-br >/dev/null 2>&1; then

    docker network rm na-br

fi

echo "Done!"