Tutorial cleanup

scripts/create_component.sh

+#!/bin/bash
+# Copyright 2021-2023 H2020 TeraFlow (https://www.teraflow-h2020.eu/)
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+COMPONENT_NAME=$1
+PROJECTDIR=`pwd`
+
+mkdir -p ${PROJECTDIR}/src/${COMPONENT_NAME}
+mkdir -p ${PROJECTDIR}/src/${COMPONENT_NAME}/client
+mkdir -p ${PROJECTDIR}/src/${COMPONENT_NAME}/service
+mkdir -p ${PROJECTDIR}/src/${COMPONENT_NAME}/tests
+
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/client/__init__.py
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/service/__init__.py
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/tests/__init__.py
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/.gitlab-ci.yml
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/__init__.py
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/Config.py
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/Dockerfile
+touch ${PROJECTDIR}/src/${COMPONENT_NAME}/requirements.in
+
+cd ${PROJECTDIR}/src
+python gitlab-ci.yml_generator.py -t latest forecaster
+
+cd ${PROJECTDIR}/src/${COMPONENT_NAME}
+mv .gitlab-ci.yml gitlab-ci.yaml
+${PROJECTDIR}/scripts/add_license_header_to_files.sh
+mv gitlab-ci.yaml .gitlab-ci.yml

tutorial/1-2-install-microk8s.md

@@ -83,17 +83,27 @@ microk8s config > $HOME/.kube/config
 sudo reboot
 ```
 
-## 1.2.6. Check status of Kubernetes
+## 1.2.6. Check status of Kubernetes and addons
+To retrieve the status of Kubernetes __once__, run the following command:
 ```bash
 microk8s.status --wait-ready
 ```
 
+To retrieve the status of Kubernetes __periodically__ (e.g., every 1 second), run the following command:
+```bash
+watch -n 1 microk8s.status --wait-ready
+```
 
 ## 1.2.7. Check all resources in Kubernetes
+To retrieve the status of the Kubernetes resources __once__, run the following command:
 ```bash
-microk8s.kubectl get all --all-namespaces
+kubectl get all --all-namespaces
 ```
 
+To retrieve the status of the Kubernetes resources __periodically__ (e.g., every 1 second), run the following command:
+```bash
+watch -n 1 kubectl get all --all-namespaces
+```
 
 ## 1.2.8. Enable addons
 The Addons enabled are:
@@ -106,10 +116,14 @@ The Addons enabled are:
 microk8s.enable dns hostpath-storage ingress registry
 ```
 
-__Note__: enabling some of the addons might take few minutes.
-          [Check status](./1-2-install-microk8s.md#124-check-status-of-kubernetes) periodically until all addons are
-          shown as enabled. Then [Check resources](./1-2-install-microk8s.md#125-check-all-resources-in-kubernetes)
-          periodically until all pods are Ready and Running.
+__Important__: Enabling some of the addons might take few minutes. Do not proceed with next steps until the addons are
+               ready. Otherwise, the deployment might fail. To confirm everything is up and running:
+1. Periodically
+   [Check the status of Kubernetes](./1-2-install-microk8s.md#126-check-status-of-kubernetes)
+   until you see the addons [dns, ha-cluster, hostpath-storage, ingress, registry, storage] in the enabled block.
+2. Periodically
+   [Check Kubernetes resources](./1-2-install-microk8s.md#127-check-all-resources-in-kubernetes)
+   until all pods are __Ready__ and __Running__.
 
 
 ## 1.2.9. Stop, Restart, and Redeploy

tutorial/2-2-ofc22.md

@@ -2,7 +2,9 @@
 
 This functional test reproduces the live demonstration "Demonstration of Zero-touch Device and L3-VPN Service Management
 Using the TeraFlow Cloud-native SDN Controller" carried out at
-[OFC'22](https://www.ofcconference.org/en-us/home/program-speakers/demo/).
+[OFC'22](https://ieeexplore.ieee.org/document/9748575).
+
+
 
 
 ## 2.2.1. Functional test folder
@@ -33,8 +35,9 @@ To run this functional test, it is assumed you have deployed a MicroK8s-based Ku
 controller instance as described in the [Tutorial: Deployment Guide](./1-0-deployment.md), and you configured the Python
 environment as described in
 [Tutorial: Run Experiments Guide > 2.1. Configure Python Environment](./2-1-python-environment.md).
-Remember to source the scenario settings appropriately, e.g., `cd ~/tfs-ctrl && source my_deploy.sh` in each terminal
-you open.
+Remember to source the scenario settings, e.g., `cd ~/tfs-ctrl && source ofc22/deploy_specs.sh` in each terminal you open.
+Then, re-build the protocol buffers code from the proto files:
+`./proto/generate_code_python.sh`
 
 
 ## 2.2.4. Access to the WebUI and Dashboard

tutorial/2-4-ecoc22.md

 # 2.4. ECOC'22 Demo - Disjoint DC-2-DC L3VPN Service (WORK IN PROGRESS)
 
 This functional test reproduces the experimental assessment of "Experimental Demonstration of Transport Network Slicing
-with SLA Using the TeraFlowSDN Controller" presented at [ECOC'22](https://www.ecoc2022.org/).
+with SLA Using the TeraFlowSDN Controller" presented at [ECOC'22](https://www.optica.org/en-us/events/topical_meetings/ecoc/schedule/?day=Tuesday#Tuesday).
 
 ## 2.4.1. Functional test folder
 
@@ -27,10 +27,7 @@ To run this functional test, it is assumed you have deployed a MicroK8s-based Ku
 controller instance as described in the [Tutorial: Deployment Guide](./1-0-deployment.md), and you configured the Python
 environment as described in
 [Tutorial: Run Experiments Guide > 2.1. Configure Python Environment](./2-1-python-environment.md).
-Remember to source the scenario settings appropriately, e.g., `cd ~/tfs-ctrl && source my_deploy.sh` in each terminal
-you open.
-Next, remember to source the environment variables created by the deployment, e.g.,
-`cd ~/tfs-ctrl && source tfs_runtime_env_vars.sh`.
+Remember to source the scenario settings, e.g., `cd ~/tfs-ctrl && source ecoc22/deploy_specs.sh` in each terminal you open.
 Then, re-build the protocol buffers code from the proto files:
 `./proto/generate_code_python.sh`
 

tutorial/3-0-development.md

@@ -7,4 +7,6 @@ this guide assumes you are using the Oracle VirtualBox-based VM running MicroK8s
 ## Table of Content:
 - [3.1. Configure VSCode and Connect to the VM](./3-1-configure-vscode.md)
 - [3.2. Development Commands, Tricks, and Hints (WORK IN PROGRESS)](./3-2-develop-cth.md)
-- [3.3. Debugging individual components in VSCode](./3.3-debug-comp.md)
+- [3.3. Debugging individual components in VSCode](./3-3-debug-comp.md)
+
+- [3.X. Developing a new component: Forecaster (WORK IN PROGRESS)](./3-X-develop-new-component.md)

tutorial/3-X-develop-new-component.md

+# 3.X. Developing a new component: Forecaster (WORK IN PROGRESS)
+
+
+## 3.X.1. Preliminary requisites
+As any microservice-based architecture, the components of TeraFlowSDN can be implemented using different programming languages.
+For the sake of simplicity, and given it is the most widely used programming language in TeraFlow, this tutorial page assumes the reader will use Python.
+
+This tutorial assumes you hace successfully completed the steps in
+[2.1. Configure the Python Environment](./2-1-python-environment.md) and
+[3.1. Configure VSCode and Connect to the VM](./3-1-configure-vscode.md) to prepare your environment.
+
+
+## 3.X.2. Create the component folder structure
+The source code of each component of TeraFlowSDN is hosted in a particular folder within the `src` folder.
+Within that folder, typically, 3 subfolders are created:
+- Folder `client`: contains a client implementation that the rest of components can use to interact with the component.
+  See details in [3.X.4. Create the component client](./3-X-develop-new-component.md#3x4-create-the-component-client).
+- Folder `service`: contains the implementation of the service logic.
+  See details in [3.X.5. Create the component service](./3-X-develop-new-component.md#3x5-create-the-component-service).
+- Folder `tests`: contains the set of unitary tests to be executed over the component to ensure it is properly implemented.
+  See details in [3.X.6. Create the component tests](./3-X-develop-new-component.md#3x6-create-the-component-tests).
+- File `__init__.py`: defines the component as a sub-package of TeraFlowSDN to facilitate imports.
+- File `.gitlab-ci.yml`: defines the GitLab CI/CD settings to build, test, and deploy the component in an automated manner.
+- File `Config.py`: contains particular configuration settings and constants for the component.
+- File `Dockerfile`: defines the recipe to construct the Docker image for the component.
+- File `requirements.in`: defines the Python dependencies that are required by this component.
+
+You can automate the creation of this file structure running the following command.
+In this example, we create the `forecaster` component.
+```bash
+cd ~/tfs-ctrl
+scripts/create_component.sh forecaster
+```
+
+
+## 3.X.3. gRPC Proto messages and services
+The components, e.g., microservices, of the TeraFlowSDN controller, in general, use a gRPC-based open API to interoperate.
+All the protocol definitions can be found in sub-folder `proto` within the root project folder.
+For additional details on gRPC, visit the official web-page [gRPC](https://grpc.io/).
+
+In general, each component has an associated _proto_ file named as the name of the component in snake_case.proto.
+For instance, the _proto_ file for the `forecaster` component being developed in this tutorial is `proto/forecaster.proto` and implements 3 RPC methods:
+- `rpc GetForecastOfTopology (context.TopologyId) returns (Forecast) {}´:
+  Takes a topology identifier as parameter, and computes the aggregated forecast for the topology.
+- `rpc GetForecastOfLink(context.LinkId) returns (Forecast) {}´:
+  Takes a link identifier as parameter, and computes the aggregated forecast for that link.
+- `rpc CheckService (context.ServiceId) returns (ForecastPrediction) {}´:
+  Takes a service identifier as parameter, computes the forecast for the connections of that service, and retrieves a value indicating if the resources can support the demand.
+
+
+## 3.X.4. Create the component client
+Each component has, by default, a pre-defined client that other components can import to inter-communicate.
+The client module, by default, is named as the component's name concatenated with `client`, and written in CamelCase.
+For instance, the client for the `forecaster` component would be `ForecasterClient.py`.
+
+This file contains a class with the same name as the file, e.g., `ForecasterClient`, and implements 3 main methods, plus one method for each RPC method supported by the service. These methods are:
+- Main methods:
+  - `__init__(host=None, port=None)`: constructor of the client class.
+  - `connect(self)`: triggers the connection of the client to the service pointed by host and port class parameters.
+  - `close(self)`: disconnects the client from the service.
+- RPC methods: one for each RPC method defined in the associated service within the proto file, e.g., `proto/forecaster.proto`.
+
+Create file `` 
+
+
+
+## 3.X.3. Connect VSCode to the VM through "Remote SSH" extension
+- Right-click on "TFS-VM"
+- Select "Connect to Host in Current Window"
+- Reply to the questions asked
+  - Platform of the remote host "TFS-VM": Linux
+  - "TFS-VM" has fingerprint "<fingerprint>". Do you want to continue?: Continue
+  - Type tfs user's password: tfs123
+- You should be now connected to the TFS-VM.
+
+__Note__: if you get a connection error message, the reason might be due to wrong SSH server fingerprint. Edit file
+          "<...>/.ssh/known_hosts" on your local user account, check if there is a line starting with
+          "[127.0.0.1]:2200" (assuming previous port forwarding configuration), remove the entire line, save the file,
+          and retry connection.
+
+
+## 3.X.4. Add SSH key to prevent typing the password every time
+This step creates an SSH key in the VM and installs it on the VSCode to prevent having to type the password every time.
+
+- In VSCode (connected to the VM), click menu "Terminal > New Terminal"
+- Run the following commands on the VM's terminal through VSCode
+```bash
+ssh-keygen -t rsa -b 4096 -f ~/.ssh/tfs-vm.key
+  # leave password empty
+ssh-copy-id -i ~/.ssh/tfs-vm.key.pub tfs@10.0.2.10
+  # tfs@10.0.2.10's password: <type tfs user's password: tfs123>
+rm .ssh/known_hosts 
+```
+
+- In VSCode, click left "Explorer" panel to expand, if not expanded, and click "Open Folder" button.
+  - Choose "/home/tfs/"
+  - Type tfs user's password when asked
+  - Trust authors of the "/home/tfs [SSH: TFS-VM]" folder when asked
+- Right click on the file "tfs-vm.key" in the file explorer
+  - Select "Download..." option
+  - Download the file into your user's accout ".ssh" folder
+- Delete files "tfs-vm.key" and "tfs-vm.key.pub" on the TFS-VM.
+
+- In VSCode, click left "Remote Explorer" panel to expand
+  - Click the "gear" icon next to "SSH TARGETS" on top of "Remote Explorer" bar
+  - Choose to edit "<...>/.ssh/config" file (or equivalent)
+  - Find entry "Host TFS-VM" and update it as follows:
+```
+Host TFS-VM
+    HostName 127.0.0.1
+    Port 2200
+    ForwardX11 no
+    User tfs
+    IdentityFile "<path to the downloaded identity private key file>"
+```
+  - Save the file
+- From now, VSCode will use the identity file to connect to the TFS-VM instead of the user's password.
+
+
+## 3.X.5. Install VSCode Python Extension (in VSCode server)
+This step installs Python extensions in VSCode server running in the VM.
+
+- In VSCode (connected to the VM), click left button "Extensions"
+- Search "Python" extension in the extension Marketplace.
+- Install official "Python" extension released by Microsoft.
+  - By default, since you're connected to the VM, it will be installed in the VSCode server running in the VM.
+
+- In VSCode (connected to the VM), click left button "Explorer"
+- Click "Ctrl+Alt+P" and type "Python: Select Interpreter". Select option "Python: 3.9.13 64-bit ('tfs')"
+
+in terminal: set python path to be used by VSCode:
+`echo "PYTHONPATH=./src" > ~/tfs-ctrl/.env`