Merge branch 'feat/hpsr23_tests' into 'develop'

# Tests for P4 functionality of TeraFlowSDN

This directory contains the necessary scripts and configurations to run tests for the P4 functionality of TFS.

## Basic scripts

To run the experiments you should use the five scripts in the following order:

```

setup.sh

run_test_01_bootstrap.sh

run_test_02_create_service.sh

run_test_03_delete_service.sh

run_test_04_cleanup.sh

```

The setup script copies the necessary artifacts to the SBI service pod. It should be run just once, after a fresh install of TFS.

The bootstrap script registers the context, topology, links and, devices to TFS.

The create service scripts establishes a service between two endpoints.

The delete service script delete the aforementioned service.

Cleanup script deletes all the objects (context, topology, links, devices) from TFS.

## Objects file

The above bash scripts make use of the corresponding python scripts found under `./tests/` directory.

More important is the `./tests/Objects.py` file, which contains the definition of the Context, Topology, Devices, Links, Services. **This is the file that need changes in case of a new topology.**

Check the `./tests/Objects.py` file before running the experiment to make sure that the switches details are correct (ip address, port, etc.)

## Mininet topologies

In the `./mininet/` directory there are different mininet topology examples. The current `./tests/Objects.py` file corresponds to the `./mininet/8switch3path.py` topology. Additionally there is a backup file `./tests/topologies/6switchObjects.py` which corresponds to the `./mininet/6switch2path.py`.

## P4 artifacts

In the `./p4/` directory there are the compiled p4 artifacts that contain the pipeline that will be pushed to the p4 switch, along with the p4-runtime definitions. 

The `./setup.sh` script copies from this directory. So if you need to change p4 program, make sure to put the compiled artifacts here.

## Latency probe

In the `./probe/` directory there is a little program which calculates latency between two hosts in mininet and sends them to the Monitoring component. For specific instructions, refer to the corresponding `./probe/README.md` file.

#!/usr/bin/python

#  Copyright 2022-2023 ETSI TeraFlowSDN - TFS OSG (https://tfs.etsi.org/)

#  Copyright 2019-present Open Networking Foundation

#

#  Licensed under the Apache License, Version 2.0 (the "License");

#  you may not use this file except in compliance with the License.

#  You may obtain a copy of the License at

#

#      http://www.apache.org/licenses/LICENSE-2.0

#

#  Unless required by applicable law or agreed to in writing, software

#  distributed under the License is distributed on an "AS IS" BASIS,

#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

#  See the License for the specific language governing permissions and

#  limitations under the License.

import argparse

from mininet.cli import CLI

from mininet.log import setLogLevel

from mininet.net import Mininet

from mininet.node import Host

from mininet.topo import Topo

from stratum import StratumBmv2Switch

CPU_PORT = 255

class IPv4Host(Host):

    """Host that can be configured with an IPv4 gateway (default route).

    """

    def config(self, mac=None, ip=None, defaultRoute=None, lo='up', gw=None,

               **_params):

        super(IPv4Host, self).config(mac, ip, defaultRoute, lo, **_params)

        self.cmd('ip -4 addr flush dev %s' % self.defaultIntf())

        self.cmd('ip -6 addr flush dev %s' % self.defaultIntf())

        self.cmd('ip -4 link set up %s' % self.defaultIntf())

        self.cmd('ip -4 addr add %s dev %s' % (ip, self.defaultIntf()))

        if gw:

            self.cmd('ip -4 route add default via %s' % gw)

        # Disable offload

        for attr in ["rx", "tx", "sg"]:

            cmd = "/sbin/ethtool --offload %s %s off" % (

                self.defaultIntf(), attr)

            self.cmd(cmd)

        def updateIP():

            return ip.split('/')[0]

        self.defaultIntf().updateIP = updateIP

class TutorialTopo(Topo):

    """Basic Server-Client topology with IPv4 hosts"""

    def __init__(self, *args, **kwargs):

        Topo.__init__(self, *args, **kwargs)

        # Switches

        # gRPC port 50001

        switch1 = self.addSwitch('switch1', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # gRPC port 50002

        switch2 = self.addSwitch('switch2', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # gRPC port 50003

        switch3 = self.addSwitch('switch3', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # gRPC port 50004

        switch4 = self.addSwitch('switch4', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # gRPC port 50005

        switch5 = self.addSwitch('switch5', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # gRPC port 50006

        switch6 = self.addSwitch('switch6', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # gRPC port 50007

        switch7 = self.addSwitch('switch7', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # gRPC port 50008

        switch8 = self.addSwitch('switch8', cls=StratumBmv2Switch, cpuport=CPU_PORT)

        # Hosts

        client = self.addHost('client', cls=IPv4Host, mac="aa:bb:cc:dd:ee:11",

                            ip='10.0.0.1/24', gw='10.0.0.100')

        server = self.addHost('server', cls=IPv4Host, mac="aa:bb:cc:dd:ee:22",

                            ip='10.0.0.2/24', gw='10.0.0.100')

        # Switch links

        self.addLink(switch1, switch2)  # Switch1:port 1, Switch2:port 1

        self.addLink(switch1, switch4)  # Switch1:port 2, Switch4:port 1

        self.addLink(switch1, switch6)  # Switch1:port 3, Switch6:port 1

        self.addLink(switch2, switch3)  # Switch2:port 2, Switch3:port 1

        self.addLink(switch4, switch5)  # Switch4:port 2, Switch5:port 1

        self.addLink(switch6, switch7)  # Switch6:port 2, Switch7:port 1

        self.addLink(switch3, switch8)  # Switch3:port 2, Switch8:port 1

        self.addLink(switch5, switch8)  # Switch5:port 2, Switch8:port 2

        self.addLink(switch7, switch8)  # Switch7:port 2, Switch8:port 3

        # Host links

        self.addLink(client, switch1)   # Switch1: port 4

        self.addLink(server, switch8)   # Switch8: port 4

def main():

    net = Mininet(topo=TutorialTopo(), controller=None)

    net.start()

    #get hosts

    client = net.hosts[0]

    client.setARP('10.0.0.2', 'aa:bb:cc:dd:ee:22')

    server = net.hosts[1]

    server.setARP('10.0.0.1', 'aa:bb:cc:dd:ee:11')

    CLI(net)

    net.stop()

    print '#' * 80

    print 'ATTENTION: Mininet was stopped! Perhaps accidentally?'

    print 'No worries, it will restart automatically in a few seconds...'

    print 'To access again the Mininet CLI, use `make mn-cli`'

    print 'To detach from the CLI (without stopping), press Ctrl-D'

    print 'To permanently quit Mininet, use `make stop`'

    print '#' * 80

if __name__ == "__main__":

    parser = argparse.ArgumentParser(

        description='Mininet topology script for 2x2 fabric with stratum_bmv2 and IPv4 hosts')

    args = parser.parse_args()

    setLogLevel('info')

    main()

# Probe for P4 mininet devices

Step 1:

To copy the necessary files, run:

```

probe-tfs/deploy.sh

```

Step 2:

To connect to the mininet docker, run:

```

probe-tfs/connect-to-mininet.sh

```

Step 3:

From inside the mininet docker, run:

```

./tfsagent

```

Step 4 (on another terminal):

Establish the service:

```

src/tests/p4/run_test_02_create_service.sh

```

Step 5:

From inside mininet (make mn-cli):

```

client ./tfsping

```

Step 6 (on another terminal):

To check the latest monitoring samples, run

```

python src/tests/p4/probe/monitoring_kpis.py

```

%% Cell type:markdown id: tags:

# Checking the monitoring component

%% Cell type:code id: tags:

``` python

import time

import datetime

import uuid

import random

from dotenv import load_dotenv

from IPython.display import clear_output, display, HTML

from common.tools.timestamp.Converters import timestamp_utcnow_to_float, timestamp_float_to_string

from common.tools.grpc.Tools import grpc_message_to_json_string

from common.proto.kpi_sample_types_pb2 import KpiSampleType

from common.proto.monitoring_pb2 import KpiDescriptor, KpiId, KpiQuery, Kpi

from monitoring.client.MonitoringClient import MonitoringClient

```

%% Cell type:code id: tags:

``` python

load_dotenv()

monitoring_client = MonitoringClient()

uuid.uuid4().hex

```

%% Output

    '0abfb00117d4461b9fa5085bee4be58f'

%% Cell type:code id: tags:

``` python

kpi_description: KpiDescriptor = KpiDescriptor()

kpi_description.kpi_description = "Security status of service {}".format(uuid.uuid4().hex)

kpi_description.service_id.service_uuid.uuid = "608df176-90b8-5950-b50d-1810c6eaaa5d"

kpi_description.kpi_sample_type = KpiSampleType.KPISAMPLETYPE_UNKNOWN

new_kpi = monitoring_client.SetKpi(kpi_description)

print("Created KPI {}: ".format(grpc_message_to_json_string(new_kpi)))

```

%% Output

    Created KPI {"kpi_id": {"uuid": "1"}}:

%% Cell type:code id: tags:

``` python

kpi_id = input("What is the KPI ID?")

query = KpiQuery()

query.kpi_ids.append(KpiId(**{"kpi_id": {"uuid": kpi_id}}))

query.last_n_samples = 10

while True:

    kpi = Kpi()

    kpi.kpi_id.kpi_id.uuid = new_kpi.kpi_id.uuid

    kpi.timestamp.timestamp = timestamp_utcnow_to_float()

    kpi.kpi_value.int32Val = random.randint(10, 4000)

    # monitoring_client.IncludeKpi(kpi)

    response = monitoring_client.QueryKpiData(query)

    # print(response)

    table = f"""<table>

    <thead>

        <tr><th colspan=3>{datetime.datetime.now()}</th></tr>

        <tr><th>KPI ID</th><th>Timestamp</th><th>Value</th></tr>

    <thead>

    <tbody>"""

    for kpi in response.raw_kpi_lists:

        cur_kpi_id = kpi.kpi_id.kpi_id.uuid

        for i, raw_kpi in enumerate(kpi.raw_kpis):

            # print(cur_kpi_id, raw_kpi.timestamp.timestamp, raw_kpi.kpi_value)

            table += "<tr><td>{} - {}</td><td>{}</td><td>{}</td></tr>".format(

                i, cur_kpi_id, timestamp_float_to_string(raw_kpi.timestamp.timestamp), raw_kpi.kpi_value

            )

    table += "</tbody></table>"

    display(HTML(table))

    time.sleep(5)

    clear_output(wait=True)

```

%% Output

    ---------------------------------------------------------------------------

    KeyboardInterrupt                         Traceback (most recent call last)

Cell     In [4], line 31

         29 table += "</tbody></table>"

         30 display(HTML(table))

    ---> 31 time.sleep(5)

         32 clear_output(wait=True)

    KeyboardInterrupt:

# Copyright 2022-2023 ETSI TeraFlowSDN - TFS OSG (https://tfs.etsi.org/)

#

# Licensed under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.

# You may obtain a copy of the License at

#

#      http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

# File to monitor the latest *n* samples from the KPI ID *id*

# and updates it every *i* seconds

#

# Author: Carlos Natalino <carlos.natalino@chalmers.se>

import argparse

import datetime

import time

from common.proto.kpi_sample_types_pb2 import KpiSampleType

from common.proto.monitoring_pb2 import KpiDescriptor, KpiId, KpiQuery

from common.tools.grpc.Tools import grpc_message_to_json_string

from common.tools.timestamp.Converters import timestamp_float_to_string

from monitoring.client.MonitoringClient import MonitoringClient

if __name__ == "__main__":

    parser = argparse.ArgumentParser()

    parser.add_argument(

        "-n",

        "--last-n-samples",

        default=10,

        type=int,

        help="Number of latest samples of the KPI to show.",

    )

    parser.add_argument(

        "-s",

        "--sleep",

        default=5,

        type=int,

        help="Seconds between consecutive refreshes.",

    )

    parser.add_argument("-id", "--kpi-id", help="KPI ID, if known.")

    args = parser.parse_args()

    monitoring_client = MonitoringClient()

    if args.kpi_id is None:

        service_uuid = "608df176-90b8-5950-b50d-1810c6eaaa5d"

        kpi_description: KpiDescriptor = KpiDescriptor()

        kpi_description.kpi_description = "Security status of service {}".format(

            service_uuid

        )

        kpi_description.service_id.service_uuid.uuid = service_uuid

        kpi_description.kpi_sample_type = KpiSampleType.KPISAMPLETYPE_UNKNOWN

        new_kpi = monitoring_client.SetKpi(kpi_description)

        print("Created KPI {}: ".format(grpc_message_to_json_string(new_kpi)))

        kpi_id = new_kpi.kpi_id.uuid

    else:

        kpi_id = args.kpi_id

    query = KpiQuery()

    query.kpi_ids.append(KpiId(**{"kpi_id": {"uuid": kpi_id}}))

    query.last_n_samples = args.last_n_samples

    while True:

        print(chr(27) + "[2J")

        response = monitoring_client.QueryKpiData(query)

        print("{}\t{}\t{:<20}\t{}".format("Index", "KPI ID", "Timestamp", "Value"))

        for kpi in response.raw_kpi_lists:

            cur_kpi_id = kpi.kpi_id.kpi_id.uuid

            for i, raw_kpi in enumerate(kpi.raw_kpis):

                print(

                    "{}\t{}\t{}\t{}".format(

                        i,

                        cur_kpi_id,

                        timestamp_float_to_string(raw_kpi.timestamp.timestamp),

                        raw_kpi.kpi_value.floatVal,

                    )

                )

        print("Last update:", datetime.datetime.now().strftime("%H:%M:%S"))

        time.sleep(args.sleep)