CI pipeline and multiple module fixes

Lluis Gifre Renomrequested to merge
fix/ci-pipeline into develop

Script deploy/crdb.sh:

  • Modified port exposing CockroachDB Mgmt GUI to prevent conflicts

Common - MessageBroker - NATS:

  • Enabled URI-based settings
  • Enabled support for optional authentication

Common - MethodWrappers:

  • Added missing "raise" statement in decorator

Common Tools - gRPC Config Rules:

  • Added field to select new action for config rules being updated

Context component:

  • Enabled URI-based settings for CockroachDB
  • Added storage of ACL ConfigRules
  • Added support for all Constraint Kinds
  • Corrected ConstraintKinds in ConstraintModel
  • Corrected UUID composition in ConfigRules to prevent collisions
  • Corrected UUID composition in Constraints to prevent collisions
  • Corrected classification of ConfigRules to upsert/delete
  • Corrected classification of Constraints to upsert/delete

Device component:

  • Improved update of ConfigRules to be changed in Context
  • Added conditions to prevent reaching the driver when there is nothing to configure/deconfigure

Device component - Emulated driver:

  • Minor corrections in unitary tests

Device component - TransportAPI driver:

  • Enabled support for HTTP/HTTPS
  • Added support for optional authentication
  • Improved parsing of SIPs

Monitoring component:

  • Added delays in event collection to prevent overloading CPU
  • Added missing KPI creation in GetStreamKpi unitary test
  • Added log messages in unitary tests for issue traceability
  • Added message broker termination statement in unitary tests

PathComp component - Backend:

  • Increased number of edges and endpoints per device
  • Reduced number of contexts

PathComp component - Frontend:

  • Added control of number of endpoints per link

Service component:

  • Load service from Context at UpdateService to get correct Context-generated UUIDs
  • Added update of ServiceType during UpdateService
  • Update ConfigRules in Context when UpdateService is executed
  • Defined SettingsHandler class to help in retrieving settings
  • Updated ServiceHandlers to use the new SettingsHandler class

Slice component:

  • Load slice from Context at UpdateSlice to get correct Context-generated UUIDs

WebUI component:

  • Removed UUIDs in topology view
  • Replaced list of endpoints in device list page by number of endpoints and number of configured rules
  • Added field "name" in Service and Slice list and detail pages

Hackfest - MockOSM:

  • Corrected default IP address
  • Corrected error retrieval on connection

Hackfest - TAPI server:

  • Updated methods to properly retrieve/store connectivity services

Test tools - MockOSM:

  • Corrected error retrieval on connection
Edited by Lluis Gifre Renom

Merge request reports

deploy/crdb.sh

+9 −7
Changes for deploy/crdb.sh: 9 added lines, 7 removed lines.
Original line number Diff line number Diff line
@@ -111,7 +111,7 @@ function crdb_deploy_single() { 
    echo



    echo "CockroachDB Port Mapping"

    echo ">>> Expose CockroachDB SQL port (26257)"

    echo ">>> Expose CockroachDB SQL port (26257->26257)"

    CRDB_SQL_PORT=$(kubectl --namespace ${CRDB_NAMESPACE} get service cockroachdb-public -o 'jsonpath={.spec.ports[?(@.name=="sql")].port}')

    PATCH='{"data": {"'${CRDB_SQL_PORT}'": "'${CRDB_NAMESPACE}'/cockroachdb-public:'${CRDB_SQL_PORT}'"}}'

    kubectl patch configmap nginx-ingress-tcp-microk8s-conf --namespace ingress --patch "${PATCH}"
@@ -122,12 +122,13 @@ function crdb_deploy_single() { 
    kubectl patch daemonset nginx-ingress-microk8s-controller --namespace ingress --patch "${PATCH}"

    echo



    echo ">>> Expose CockroachDB HTTP Mgmt GUI port (8080)"

    echo ">>> Expose CockroachDB HTTP Mgmt GUI port (8080->8081)"

    CRDB_GUI_PORT_EXT="8081"

    CRDB_GUI_PORT=$(kubectl --namespace ${CRDB_NAMESPACE} get service cockroachdb-public -o 'jsonpath={.spec.ports[?(@.name=="http")].port}')

    PATCH='{"data": {"'${CRDB_GUI_PORT}'": "'${CRDB_NAMESPACE}'/cockroachdb-public:'${CRDB_GUI_PORT}'"}}'

    PATCH='{"data": {"'${CRDB_GUI_PORT_EXT}'": "'${CRDB_NAMESPACE}'/cockroachdb-public:'${CRDB_GUI_PORT}'"}}'

    kubectl patch configmap nginx-ingress-tcp-microk8s-conf --namespace ingress --patch "${PATCH}"



    PORT_MAP='{"containerPort": '${CRDB_GUI_PORT}', "hostPort": '${CRDB_GUI_PORT}'}'

    PORT_MAP='{"containerPort": '${CRDB_GUI_PORT_EXT}', "hostPort": '${CRDB_GUI_PORT_EXT}'}'

    CONTAINER='{"name": "nginx-ingress-microk8s", "ports": ['${PORT_MAP}']}'

    PATCH='{"spec": {"template": {"spec": {"containers": ['${CONTAINER}']}}}}'

    kubectl patch daemonset nginx-ingress-microk8s-controller --namespace ingress --patch "${PATCH}"
@@ -275,12 +276,13 @@ function crdb_deploy_cluster() { 
    kubectl patch daemonset nginx-ingress-microk8s-controller --namespace ingress --patch "${PATCH}"

    echo



    echo ">>> Expose CockroachDB HTTP Mgmt GUI port (8080)"

    echo ">>> Expose CockroachDB HTTP Mgmt GUI port (8080->8081)"

    CRDB_GUI_PORT_EXT="8081"

    CRDB_GUI_PORT=$(kubectl --namespace ${CRDB_NAMESPACE} get service cockroachdb-public -o 'jsonpath={.spec.ports[?(@.name=="http")].port}')

    PATCH='{"data": {"'${CRDB_GUI_PORT}'": "'${CRDB_NAMESPACE}'/cockroachdb-public:'${CRDB_GUI_PORT}'"}}'

    PATCH='{"data": {"'${CRDB_GUI_PORT_EXT}'": "'${CRDB_NAMESPACE}'/cockroachdb-public:'${CRDB_GUI_PORT}'"}}'

    kubectl patch configmap nginx-ingress-tcp-microk8s-conf --namespace ingress --patch "${PATCH}"



    PORT_MAP='{"containerPort": '${CRDB_GUI_PORT}', "hostPort": '${CRDB_GUI_PORT}'}'

    PORT_MAP='{"containerPort": '${CRDB_GUI_PORT_EXT}', "hostPort": '${CRDB_GUI_PORT_EXT}'}'

    CONTAINER='{"name": "nginx-ingress-microk8s", "ports": ['${PORT_MAP}']}'

    PATCH='{"spec": {"template": {"spec": {"containers": ['${CONTAINER}']}}}}'

    kubectl patch daemonset nginx-ingress-microk8s-controller --namespace ingress --patch "${PATCH}"

hackfest/mock_osm/WimconnectorIETFL2VPN.py

+1 −1
Changes for hackfest/mock_osm/WimconnectorIETFL2VPN.py: 1 added line, 1 removed line.
Original line number Diff line number Diff line
@@ -73,7 +73,7 @@ class WimconnectorIETFL2VPN(SdnConnectorBase): 
            response = requests.get(endpoint, auth=self.auth)

            http_code = response.status_code

        except requests.exceptions.RequestException as e:

            raise SdnConnectorError(e.message, http_code=503)

            raise SdnConnectorError(e.response, http_code=503)



        if http_code != 200:

            raise SdnConnectorError("Failed while authenticating", http_code=http_code)

hackfest/mock_osm/__main__.py

+1 −1
Changes for hackfest/mock_osm/__main__.py: 1 added line, 1 removed line.
Original line number Diff line number Diff line
@@ -18,7 +18,7 @@ from .MockOSM import MockOSM 
LOGGER = logging.getLogger(__name__)

LOGGER.setLevel(logging.DEBUG)



WIM_URL = 'http://10.0.2.15:80'

WIM_URL = 'http://10.0.2.10:80'

WIM_USERNAME = 'admin'

WIM_PASSWORD = 'admin'

hackfest/tapi/server/tapi_server/controllers/tapi_connectivity_controller.py

+36 −2
Changes for hackfest/tapi/server/tapi_server/controllers/tapi_connectivity_controller.py: 36 added lines, 2 removed lines.
Original line number Diff line number Diff line
@@ -2559,8 +2559,42 @@ def data_tapi_commoncontext_tapi_connectivityconnectivity_context_post(body=None 
    :rtype: None

    """

    if connexion.request.is_json:

        body = TapiConnectivityConnectivityContextWrapper.from_dict(connexion.request.get_json())  # noqa: E501

    return 'do some magic!'

        #body = TapiConnectivityConnectivityContextWrapper.from_dict(connexion.request.get_json())  # noqa: E501

        raw_body = connexion.request.get_json()

        if "tapi-connectivity:connectivity-service" in raw_body:

            raw_body["connectivity-service"] = raw_body.pop("tapi-connectivity:connectivity-service")

        if isinstance(raw_body["connectivity-service"], list) and len(raw_body["connectivity-service"]) > 0:

            raw_body["connectivity-service"] = raw_body["connectivity-service"][0]

        

        connectivity_service = raw_body["connectivity-service"]

        if "connectivity-constraint" in connectivity_service:

            connectivity_constraint = connectivity_service.pop("connectivity-constraint")

            if "requested-capacity" in connectivity_constraint:

                connectivity_service["requested-capacity"] = connectivity_constraint.pop("requested-capacity")

            if "connectivity-direction" in connectivity_constraint:

                connectivity_service["connectivity-direction"] = connectivity_constraint.pop("connectivity-direction")



        body = TapiConnectivityConnectivityServiceWrapper.from_dict(raw_body)  # noqa: E501



    connection = TapiConnectivityConnection(

        uuid=body.connectivity_service.uuid,

        connection_end_point=[

            TapiConnectivityConnectionEndPointRef(

                node_edge_point_uuid="node-1-port-3", connection_end_point_uuid="cep13"),

            TapiConnectivityConnectionEndPointRef(

                node_edge_point_uuid="node-3-port-2", connection_end_point_uuid="cep32"),

        ]

    )

    connection_ref = TapiConnectivityConnectionRef(connection.uuid)

    body.connectivity_service.connection = [ connection_ref ]



    if database.context.connectivity_context is None:

        database.context.connectivity_context = TapiConnectivityConnectivityContext(

            connectivity_service=[], connection=[]

        )



    database.context.connectivity_context.connection.append(connection)

    database.context.connectivity_context.connectivity_service.append(body.connectivity_service)





def data_tapi_commoncontext_tapi_connectivityconnectivity_context_put(body=None):  # noqa: E501

src/common/message_broker/backend/nats/NatsBackend.py

+12 −6
Changes for src/common/message_broker/backend/nats/NatsBackend.py: 12 added lines, 6 removed lines.
Original line number Diff line number Diff line
@@ -19,18 +19,24 @@ from common.message_broker.Message import Message 
from .._Backend import _Backend

from .NatsBackendThread import NatsBackendThread



DEFAULT_NATS_URI = 'nats://127.0.0.1:4222'

#NATS_URI_TEMPLATE = 'nats://{:s}:{:s}@nats.{:s}.svc.cluster.local:{:s}' # with authentication

NATS_URI_TEMPLATE = 'nats://nats.{:s}.svc.cluster.local:{:s}'

NATS_URI_TEMPLATE_AUTH = 'nats://{:s}:{:s}@nats.{:s}.svc.cluster.local:{:s}'

NATS_URI_TEMPLATE_NOAUTH = 'nats://nats.{:s}.svc.cluster.local:{:s}'



class NatsBackend(_Backend):

    def __init__(self, **settings) -> None: # pylint: disable=super-init-not-called

        nats_uri = get_setting('NATS_URI', settings=settings, default=None)

        if nats_uri is None:

            nats_namespace   = get_setting('NATS_NAMESPACE', settings=settings)

            nats_client_port = get_setting('NATS_CLIENT_PORT', settings=settings)

        if nats_namespace is None or nats_client_port is None:

            nats_uri = get_setting('NATS_URI', settings=settings, default=DEFAULT_NATS_URI)

            nats_username    = get_setting('NATS_USERNAME', settings=settings, default=None)

            nats_password    = get_setting('NATS_PASSWORD', settings=settings, default=None)

            if nats_username is None or nats_password is None:

                nats_uri = NATS_URI_TEMPLATE_NOAUTH.format(

                    nats_namespace, nats_client_port)

            else:

            nats_uri = NATS_URI_TEMPLATE.format(nats_namespace, nats_client_port)

                nats_uri = NATS_URI_TEMPLATE_AUTH.format(

                    nats_username, nats_password, nats_namespace, nats_client_port)



        self._terminate = threading.Event()

        self._nats_backend_thread = NatsBackendThread(nats_uri)

        self._nats_backend_thread.start()