Commit 45daec38 authored by Laura Dominguez's avatar Laura Dominguez
Browse files

Enable gRPC integration and reliable IPsec rekeying

parent 071c7e47
Loading
Loading
Loading
Loading

proto/ccips.proto

0 → 100644
+33 −0
Changes for proto/ccips.proto: 33 added lines, 0 removed lines.
Original line number Diff line number Diff line
syntax = "proto3";

package ccips;

service CcipsService {
  rpc CreateHandler (JsonRequest) returns (CcipsResponse);
  rpc DeleteHandler (HandlerRequest) returns (CcipsResponse);
  rpc GetHandlerConfig (HandlerRequest) returns (CcipsResponse);
  rpc GetAllConfigs (EmptyRequest) returns (CcipsResponse);
  rpc ProcessRekey (RekeyRequest) returns (CcipsResponse);
  rpc ProcessRekeyNotification (JsonRequest) returns (CcipsResponse);
}

message EmptyRequest {}

message JsonRequest {
  string payload_json = 1;
}

message HandlerRequest {
  string handler_id = 1;
}

message RekeyRequest {
  string handler_id = 1;
  string ipsec_name = 2;
}

message CcipsResponse {
  string payload_json = 1;
  string error = 2;
  string error_type = 3;
}
 No newline at end of file
+1 −1
Changes for src/ccips/Dockerfile: 1 added line, 1 removed line.
Original line number Diff line number Diff line
@@ -17,7 +17,7 @@ FROM python:3.9-slim
# ============================================================================
# Base/runtime configuration
# ============================================================================
ENV PYTHONUNBUFFERED=0 
ENV PYTHONUNBUFFERED=1
ENV OPENSSL_PREFIX=/usr/local 
ENV OPENSSL_CONF=/usr/local/ssl/openssl.cnf 
ENV OPENSSL_MODULES=/usr/local/lib64/ossl-modules 

src/ccips/README.md

0 → 100644
+135 −0
Changes for src/ccips/README.md: 135 added lines, 0 removed lines.
Original line number Diff line number Diff line
# CCIPS service

## Request flow

```text
HTTP client -> NBI REST routes -> CCIPS gRPC client -> ccipsservice -> StorageHandler -> agents
```

The NBI contains only the CCIPS client. Handler state, SQLite persistence,
NETCONF/HTTP/HTTPS sessions, notifications, certificates and rekey execution
belong to the CCIPS server. Each NBI worker reuses a gRPC channel. REST routes
run blocking RPCs through eventlet's native thread pool.

The existing HTTP URLs and success responses are retained:

| Method | Path | Operation |
| --- | --- | --- |
| POST | `/ccips` | Create a handler |
| GET | `/ccips/ccips-all` | List configurations |
| GET | `/ccips/<handler_id>` | Retrieve a configuration |
| DELETE | `/ccips/<handler_id>` | Stop and delete a handler |
| POST | `/ccips/process_rekey/<handler_id>/<ipsec_name>` | Process rekey |

Any configured NBI base URL still prefixes these paths. IPsec names may contain
slashes. Rekey retains its existing 200/400/404/409/502 responses. A gRPC
transport failure returns HTTP 502 rather than falling back to local execution.

The handler GET response additionally exposes `RekeyState`, with the base
`ipsec_name` and current `spi` for each direction. It reads active state under
the handler lock or uses persisted lifecycle SPIs without opening agent
sessions. Legacy records without lifecycle metadata do not invent SPI values.
A successful rekey returns both `old_spi` and `new_spi`.

The local `REKEY_TUNNEL.sh` caller must use the current SPI, not the next SPI,
in the URL suffix. It queries `GET /ccips/<handler_id>` before each cycle,
handles both directions independently and accepts advancement only after a
200 response with matching `old_spi` and a confirmed `new_spi`. On HTTP 409 it
refreshes state on the next cycle; other errors or incomplete success responses
stop the caller instead of blindly incrementing or retrying. The updated local
script accepts `BASE_URL`, `REKEY_INTERVAL` (default 25 seconds) and
`CCIPS_REQUEST_TIMEOUT` (default 310 seconds). It requires `jq` and `curl`.
For HTTPS, setting `CERT_DIR` supplies `NBI.crt`, `NBI.key` and `CA.crt`.

Mutating RPCs are not automatically retried: a lost response or deadline does
not prove that the server did not execute the operation. Check the handler's
state before repeating a request whose outcome is unknown.

`proto/ccips.proto` defines the RPCs. JSON payloads inside protobuf messages
preserve the existing request/response structures, including lists and integer
SPIs, without introducing floating-point conversion. Business errors travel
separately from transport errors. The generated Python files follow the normal
TeraFlow build process and are not committed.

`ProcessRekeyNotification` retains the previous client's acknowledgement-only
behavior. Actual rekey processing remains in `ProcessRekey` and the existing
SBI notification handlers; this change does not invent a new notification flow.

## Configuration

| Variable | Component | Default |
| --- | --- | --- |
| `CCIPSSERVICE_SERVICE_HOST` | NBI client | `ccipsservice` |
| `CCIPSSERVICE_SERVICE_PORT_GRPC` | Client and server | `10071` |
| `CCIPS_GRPC_TIMEOUT` | NBI client | `300` seconds per RPC |
| `CCIPS_STORAGE_DB` | CCIPS server | `/tmp/ccips_handlers.sqlite3` |
| `CCIPS_CERTS_DIR` | CCIPS server | `/etc/ccips/certs` |

The internal gRPC connection uses the same insecure-channel transport as
TeraFlow's `GenericGrpcService`, inside the cluster. Agent-facing HTTPS/PQC
configuration is unchanged. Only `ccipsservice` needs the `ccips-pqc-certs`
secret mounted; the NBI no longer needs the CCIPS certificate mount.

## Deploying the change

Rebuild and redeploy **both** `ccips` and `nbi`. Their Dockerfiles generate the
new protobuf modules automatically using `proto/*.proto`. Restarting existing
images alone does not install the new client/server implementation.

With the repository's registry, namespace and image-tag variables configured,
the component deployment entry point is:

```bash
bash deploy/component.sh "ccips nbi"
```

### Existing handlers

The old implementation stored handlers in the **NBI pod's** SQLite database.
The CCIPS pod has a separate filesystem and will not automatically see those
records. Do not remove or replace the old NBI pods before securing their state.

For an environment with active tunnels:

1. Quiesce CCIPS REST requests and agent notifications for the maintenance window.
2. Take a consistent SQLite backup from the old NBI instance using SQLite's
   backup API or `.backup` command, and retain it outside the pod. Do not copy
   only the database file while WAL writes are possible.
3. Restore the database at `CCIPS_STORAGE_DB` in the new CCIPS instance before
   it starts serving traffic. Preserve the complete schema and lifecycle
   metadata. If multiple NBI pods held independent state, reconcile their
   databases explicitly rather than overwriting one with another.
4. Deploy the new client and server, validate listing/retrieval of the existing
   handlers, then resume requests and notifications.

StorageHandler's existing restoration logic reconstructs sessions on demand.
Legacy records without lifecycle/teardown metadata retain their existing
limitations. The default SQLite location is ephemeral: use a persistent volume
and set `CCIPS_STORAGE_DB` for state that must survive pod replacement. SQLite
and live sessions are not a multi-replica shared-state design; keep a single
CCIPS replica unless that ownership is redesigned.

After deployment, operation and SBI logs appear here:

```bash
kubectl logs -n tfs deployment/ccipsservice -f
```

HTTP access logs and client-side transport errors still appear in `nbiservice`.

## Local tests

Generate the stubs using the same relative-import adjustment as the Dockerfiles:

```bash
python3 -m grpc_tools.protoc -I proto --python_out=src/common/proto --grpc_python_out=src/common/proto proto/ccips.proto
sed -i 's/^import ccips_pb2 as/from . import ccips_pb2 as/' src/common/proto/ccips_pb2_grpc.py
PYTHONPATH=src python3 -m pytest -q src/ccips/tests/test_grpc.py src/nbi/tests/test_ccips_routes.py
```

These tests require the existing common/CCIPS dependencies, Flask-RESTful,
eventlet and pytest. They use real local gRPC connections and temporary SQLite
databases, replacing only agent interactions. No Kubernetes cluster or network
equipment is needed. They also exercise a separate eventlet client process,
check that its event loop stays responsive, and verify that importing the
client does not load the CCIPS business layer.
 No newline at end of file
+7 −2
Changes for src/ccips/__init__.py: 7 added lines, 2 removed lines.
Original line number Diff line number Diff line
@@ -29,6 +29,11 @@ TeraFlow service layout:
- ccips/nbi: external REST/NBI entry points
"""

from .service.DeviceService import DeviceService

__all__ = ["DeviceService"]


def __getattr__(name):
	if name == "DeviceService":
		from .service.DeviceService import DeviceService
		return DeviceService
	raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+41 −97
Changes for src/ccips/client/DeviceClient.py: 41 added lines, 97 removed lines.
Original line number Diff line number Diff line
@@ -12,56 +12,42 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import json
import logging
import os
from typing import Any, Optional, Tuple
import uuid

import grpc

from common.Constants import ServiceNameEnum
from common.Settings import get_service_host, get_service_port_grpc
from common.tools.client.RetryDecorator import retry, delay_exponential
from common.tools.grpc.Tools import grpc_message_to_json_string
from ccips.service.handler import StorageHandler
from ccips.service.notification import SADBExpireNotification
from common.proto import ccips_pb2, ccips_pb2_grpc

LOGGER = logging.getLogger(__name__)

MAX_RETRIES = 15
DELAY_FUNCTION = delay_exponential(initial=0.01, increment=2.0, maximum=5.0)
_SHARED_STORAGE = StorageHandler()


class _NoopCcipsStub:
    """Compatibility stub used while the generated CCIPS proto is not present."""

    def __init__(self, endpoint: str):
        self.endpoint = endpoint

    def __getattr__(self, name: str):
        raise NotImplementedError(
            "CCIPS gRPC stub is not available yet. The in-memory migration keeps the NBI API functional until the proto is generated."
        )


class DeviceClient:
    """Internal client for the migrated CCIPS NBI service."""
    """CCIPS gRPC client; all handler state belongs to the CCIPS server."""

    def __init__(self, host: Optional[str] = None, port: Optional[int] = None):
        self.host = host or get_service_host(ServiceNameEnum.CCIPS)
        self.port = port or get_service_port_grpc(ServiceNameEnum.CCIPS)
        self.port = port if port is not None else get_service_port_grpc(ServiceNameEnum.CCIPS)
        self.endpoint = f"{self.host}:{self.port}"
        LOGGER.debug("Creating CCIPS internal client for endpoint %s", self.endpoint)
        self.timeout = float(os.environ.get("CCIPS_GRPC_TIMEOUT", "300"))
        self.channel = None
        self.stub = _NoopCcipsStub(self.endpoint)
        self.storage = _SHARED_STORAGE
        LOGGER.warning("DeviceClient using shared storage %s", hex(id(self.storage)))
        self.stub = None
        self.connect()

    def connect(self) -> None:
        """Keep the client API-compatible with the TeraFlow service model."""
        self.channel = object()
        self.stub = self.stub or _NoopCcipsStub(self.endpoint)
        LOGGER.debug("CCIPS client connected to local in-memory storage at %s", self.endpoint)
        if self.channel is not None:
            return
        self.channel = grpc.insecure_channel(self.endpoint)
        self.stub = ccips_pb2_grpc.CcipsServiceStub(self.channel)
        LOGGER.debug("CCIPS gRPC client configured for %s", self.endpoint)

    def close(self) -> None:
        if self.channel is not None:
            self.channel.close()
        self.channel = None
        self.stub = None

@@ -69,78 +55,36 @@ class DeviceClient:
        if self.stub is None:
            self.connect()

    @retry(max_retries=MAX_RETRIES, delay_function=DELAY_FUNCTION)
    def create_handler(self, request: Any) -> Tuple[Any, Optional[Exception]]:
    def _call(self, method: str, request: Any) -> Tuple[Any, Optional[Exception]]:
        self.check_connection()
        LOGGER.debug("create_handler request: %s", grpc_message_to_json_string(request))
        return self.storage.create_handler(request)
        response = getattr(self.stub, method)(request, timeout=self.timeout)
        if response.error:
            error_class = ValueError if response.error_type == "ValueError" else RuntimeError
            return None, error_class(response.error)
        return json.loads(response.payload_json) if response.payload_json else None, None

    def create_handler(self, request: Any) -> Tuple[Any, Optional[Exception]]:
        return self._call("CreateHandler", ccips_pb2.JsonRequest(
            payload_json=json.dumps(request, default=vars),
        ))

    @retry(max_retries=MAX_RETRIES, delay_function=DELAY_FUNCTION)
    def delete_handler(self, handler_id: str) -> Optional[Exception]:
        self.check_connection()
        LOGGER.debug("delete_handler handler_id=%s", handler_id)
        try:
            parsed_id = uuid.UUID(str(handler_id))
        except (TypeError, ValueError) as exc:
            return exc
        return self.storage.delete_handler(parsed_id)

    @retry(max_retries=MAX_RETRIES, delay_function=DELAY_FUNCTION)
        _, err = self._call("DeleteHandler", ccips_pb2.HandlerRequest(handler_id=str(handler_id)))
        return err

    def get_handler_config(self, handler_id: str) -> Tuple[Any, Optional[Exception]]:
        self.check_connection()
        LOGGER.debug("get_handler_config handler_id=%s", handler_id)
        try:
            parsed_id = uuid.UUID(str(handler_id))
        except (TypeError, ValueError) as exc:
            return None, exc
        return self.storage.get_config(parsed_id)

    @retry(max_retries=MAX_RETRIES, delay_function=DELAY_FUNCTION)
        return self._call("GetHandlerConfig", ccips_pb2.HandlerRequest(handler_id=str(handler_id)))

    def get_all_configs(self) -> Tuple[Any, Optional[Exception]]:
        self.check_connection()
        LOGGER.debug("get_all_configs")
        return self.storage.get_all_configs()
        return self._call("GetAllConfigs", ccips_pb2.EmptyRequest())

    @retry(max_retries=MAX_RETRIES, delay_function=DELAY_FUNCTION)
    def process_rekey_notification(self, notification: Any) -> Optional[Exception]:
        self.check_connection()
        LOGGER.debug("process_rekey_notification payload=%s", grpc_message_to_json_string(notification))
        return None
        _, err = self._call("ProcessRekeyNotification", ccips_pb2.JsonRequest(
            payload_json=json.dumps(notification, default=vars),
        ))
        return err

    @retry(max_retries=MAX_RETRIES, delay_function=DELAY_FUNCTION)
    def process_rekey(self, handler_id: str, ipsec_name: str) -> Tuple[Any, Optional[Exception]]:
        self.check_connection()
        try:
            parsed_id = uuid.UUID(str(handler_id))
        except (TypeError, ValueError):
            return None, ValueError("Invalid handler_id")

        if not ipsec_name or not ipsec_name.strip():
            return None, ValueError("Invalid ipsec_name")

        LOGGER.info("process_rekey handler_id=%s ipsec_name=%s", parsed_id, ipsec_name.strip())
        try:
            handler = self.storage._get_handler(parsed_id)
            if handler is None:
                return {"status": "Handler not found; rekey ignored"}, None

            notification = SADBExpireNotification()
            notification.EventType = "sadb-expire"
            notification.IPsecName = ipsec_name.strip()
            notification.SoftLifeTime = True
            result = handler.process_rekey(notification, parsed_id)
            status = result.get("status")
            if status in {"already_processed", "spi_mismatch", "not_found"}:
                return result, None
            if status != "processed":
                return None, RuntimeError(f"Unexpected rekey result: {result}")

            return {
                "status": "Rekey processed successfully",
                "old_spi": result["old_spi"],
                "new_spi": result["new_spi"],
                "ipsec_name": result["ipsec_name"],
            }, None
        except Exception as exc:
            LOGGER.exception("CCIPS rekey failed for handler %s", parsed_id)
            return None, exc
        return self._call("ProcessRekey", ccips_pb2.RekeyRequest(
            handler_id=str(handler_id), ipsec_name=ipsec_name,
        ))
Loading