Commit 16d012eb authored by Luis de la Cal's avatar Luis de la Cal
Browse files

- Added GetConfiguredACLRules RPC and ACLRules message to AM protobuf file to... 

- Added GetConfiguredACLRules RPC and ACLRules message to AM protobuf file to allow exporting ACL rules created by this component to other components.
- Implemented new RPC in the AM component to allow exporting ACL rules created to other components.
- Increased KPI monitoring aggregation time interval from 5 seconds to 60 seconds.
- Cleaned up CAD protobuf file.
- Added useful logging statements to CAD and AM components.
- Formatted CAD and AM code.
- Removed unused gRPC channels in AM.
- Restructured the cryptomining detector model directory.
- Renamed cryptomining detector model file to include relevant details to help better identify the model.
- Improved cryptomining detector model loading by removing the need for the model filename to be hardcoded in the code.
- Removed old cryptomining detector models.
- Refactored CAD code to improve readability.
- Added script to automatically copy compiled protobuf files to DAD component.
- Updated complete_deploy.sh to conform to the new TFS Release 2 deployment process.
- Updated CAD and AM protbuf file messages and RPCs to improve readability and clarity.
- Updated CAD and AM client and service to use the new RPCs names.
- Changed CAD output message to use the cryptomining detector model filename as identifier.
- Added ConnectionInfo class in CAD component to facilitate comparison and serialization of connection information.
- Removed test statements used to check the time taken by the cryptomining detector to perform the inference using different batch sizes.
- Added a function to properly measure the time taken by the cryptomining detector model to process a batch of connection statistics.
- Added IP addresses corresponding to known cryptomining connections to validate the cryptomining detector model performance in the classification task.
- Implemented several metrics in the CAD component to monitor the performance of the cryptomining detector model in the classification task.
- Implemented a function to export the performance metrics of the cryptomining detector in the classification task to an external file.
- Added script to retrieve performance metrics of the cryptomining detector in the classification task from CAD container.
parent bec9ad8d

cad_onnx_log.txt

deleted100644 → 0
+0 −18086

File deleted.

Preview size limit exceeded, changes collapsed.

complete_deploy.sh

+8 −1
Original line number Diff line number Diff line 
source my_deploy.sh; ./deploy.sh; source tfs_runtime_env_vars.sh; ofc22/run_test_01_bootstrap.sh; ofc22/run_test_02_create_service.sh
 
 No newline at end of file
 
./src/tests/ofc22/run_test_03_delete_service.sh

./src/tests/ofc22/run_test_04_cleanup.sh

source src/tests/ofc22/deploy_specs.sh

source my_deploy.sh

./deploy/all.sh

source tfs_runtime_env_vars.sh

ofc22/run_test_01_bootstrap.sh

ofc22/run_test_02_create_service.sh
 
 No newline at end of file

error_monitoring.txt

deleted100644 → 0
+0 −23
Original line number Diff line number Diff line 
[2022-09-30 10:06:58,753] {/var/teraflow/monitoring/service/MonitoringServiceServicerImpl.py:151} INFO - IncludeKpi

INFO:monitoringservice-server:IncludeKpi

[2022-09-30 10:06:58,754] {/var/teraflow/monitoring/service/MonitoringServiceServicerImpl.py:194} INFO - getting Kpi by KpiID

INFO:monitoringservice-server:getting Kpi by KpiID

[2022-09-30 10:06:58,764] {/var/teraflow/monitoring/service/MonitoringServiceServicerImpl.py:215} ERROR - GetKpiDescriptor exception

Traceback (most recent call last):

  File "/var/teraflow/monitoring/service/MonitoringServiceServicerImpl.py", line 196, in GetKpiDescriptor

    kpi_db = self.sql_db.get_KPI(int(request.kpi_id.uuid))

ValueError: invalid literal for int() with base 10: 'kpi_id {\n  uuid: "17"\n}\n'

ERROR:monitoringservice-server:GetKpiDescriptor exception

Traceback (most recent call last):

  File "/var/teraflow/monitoring/service/MonitoringServiceServicerImpl.py", line 196, in GetKpiDescriptor

    kpi_db = self.sql_db.get_KPI(int(request.kpi_id.uuid))

ValueError: invalid literal for int() with base 10: 'kpi_id {\n  uuid: "17"\n}\n'

[2022-09-30 10:06:58,780] {/var/teraflow/monitoring/service/MonitoringServiceServicerImpl.py:156} WARNING - Ignoring sample with KPIId(kpi_id {

  uuid: "kpi_id {\n  uuid: \"17\"\n}\n"

}

): not found in database

WARNING:monitoringservice-server:Ignoring sample with KPIId(kpi_id {

  uuid: "kpi_id {\n  uuid: \"17\"\n}\n"

}

): not found in database

[2022-09-30 10:06:58,807] {/var/teraflow/monitoring/service/MonitoringServiceServicerImpl.py:151} INFO - IncludeKpi

expose_ingress_grpc.sh

deleted100755 → 0
+0 −55
Original line number Diff line number Diff line 
#!/bin/bash

# Copyright 2022-2023 ETSI TeraFlowSDN - TFS OSG (https://tfs.etsi.org/)

#

# Licensed under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.

# You may obtain a copy of the License at

#

#      http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.



########################################################################################################################

# Define your deployment settings here

########################################################################################################################



# If not already set, set the name of the Kubernetes namespace to deploy to.

export TFS_K8S_NAMESPACE=${TFS_K8S_NAMESPACE:-"tfs"}



# If not already set, set the list of components you want to build images for, and deploy.

export TFS_COMPONENTS=${TFS_COMPONENTS:-"context device automation policy service compute monitoring l3_attackmitigator l3_centralizedattackdetector webui"}



########################################################################################################################

# Automated steps start here

########################################################################################################################



echo "Exposing GRPC ports for components..."

for COMPONENT in $TFS_COMPONENTS; do

    echo "Processing '$COMPONENT' component..."



    SERVICE_GRPC_PORT=$(kubectl get service ${COMPONENT}service --namespace $TFS_K8S_NAMESPACE -o 'jsonpath={.spec.ports[?(@.name=="grpc")].port}')

    echo "    '$COMPONENT' service port: $SERVICE_GRPC_PORT"

    if [ -z "${SERVICE_GRPC_PORT}" ]; then

        printf "\n"

        continue;

    fi



    COMPONENT_OBJNAME=$(echo "${COMPONENT}" | sed "s/\_/-/")

    PATCH='{"data": {"'${SERVICE_GRPC_PORT}'": "'$TFS_K8S_NAMESPACE'/'${COMPONENT_OBJNAME}service':'${SERVICE_GRPC_PORT}'"}}'

    #echo "PATCH: ${PATCH}"

    kubectl patch configmap nginx-ingress-tcp-microk8s-conf --namespace ingress --patch "${PATCH}"



    PORT_MAP='{"containerPort": '${SERVICE_GRPC_PORT}', "hostPort": '${SERVICE_GRPC_PORT}'}'

    CONTAINER='{"name": "nginx-ingress-microk8s", "ports": ['${PORT_MAP}']}'

    PATCH='{"spec": {"template": {"spec": {"containers": ['${CONTAINER}']}}}}'

    #echo "PATCH: ${PATCH}"

    kubectl patch daemonset nginx-ingress-microk8s-controller --namespace ingress --patch "${PATCH}"



    printf "\n"

done



echo "Done!"

get-pip.py

deleted100644 → 0
+0 −32275

File deleted.

Preview size limit exceeded, changes collapsed.