README.md

# ETSI MEC Sandbox Frontend

>_NOTE: This repository contains the MEC Sandbox Frontend; the backend portion of the MEC Sandbox is realized using the open source project [AdvantEDGE](https://github.com/InterDigitalInc/AdvantEDGE)_

MEC Sandbox is an interactive environment that enables users to learn & experiment with ETSI MEC Service APIs.
These standardised RESTful APIs are targeted towards MEC application developers to expose the value added services
offered by MEC, including real time access to network and context information, as well as location awareness.
The design principles for developing the APIs have also been specified in ETSI GS MEC 009, along with http methods,
templates, conventions and patterns. The MEC service APIs are available in YAML and JSON format at https://forge.etsi.org,
presented via OpenAPI compliant descriptions.

MEC Sandbox provides the user with a choice of scenarios combining different network technologies (4G, 5G, Wi-Fi) and
terminal types. Combining these assets in a geolocated environment, a user can gain hands-on experience on the behaviour
and capabilities of the Location (MEC013), Radio Network Information (MEC012) and WLAN Information (MEC028) service APIs.
Such contextual information can offer significant differential performance for edge based MEC applications.

MEC Sandbox deploys the [AdvantEDGE edge emulation platform](https://github.com/InterDigitalInc/AdvantEDGE) to provide
real-time access to ETSI MEC service implementations.

The MEC Sandbox frontend (provided in this repo) uses the AdvantEDGE platform & service REST APIs to create secure,
user-specific sandboxes where signed-in users can:
- Deploy & dynamically configure a scenario
- Try-out the MEC Service APIs using Swagger UI
- Interact with the MEC Services directly from a user MEC Application

## Getting Started
To use the MEC Sandbox, check out the [live deployment](https://try-mec.etsi.org).

To learn more about the MEC Sandbox, have a look at the [wiki page](https://mecwiki.etsi.org/index.php?title=MEC_Sandbox_Help).

To install the MEC Sandbox, follow the installation instructions below.

## Install MEC Sandbox

### Prerequisites
MEC Sandbox has the following prerequisites:

1. MEC Sandbox (AdvantEDGE + MEC Sandbox frontend) VM:
   - [Hardware requirements](https://github.com/InterDigitalInc/AdvantEDGE/wiki/env-hw)
1. Long-term Storage (MinIO + Thanos Compactor) VM:
   - [Hardware requirements](https://github.com/InterDigitalInc/AdvantEDGE/wiki/env-hw)
1. [AdvantEDGE](https://github.com/InterDigitalInc/AdvantEDGE/wiki) runtime & development environments on both VMs
   - Install [runtime environment](https://github.com/InterDigitalInc/AdvantEDGE/wiki/runtime-environment)
   - Install [development environment](https://github.com/InterDigitalInc/AdvantEDGE/wiki/development-environment)
1. OAuth Applications using external providers
   - **GitHub:**
     - github.com User Account --> Settings --> Developer settings --> OAuth Apps --> New OAuth App
     - _Application Name:_ ETSI MEC Sandbox
     - _Homepage URL:_ https://try-mec.etsi.org
     - _Application description:_ Optional description...
     - _Authorization callback URL:_ https://try-mec.etsi.org/platform-ctrl/v1/authorize
     - **NOTES:**
       - You can also add an application logo if you want users to see one during authentication
       - Once created, keep note of the OAuth App Client ID & Secret; this secret is only temporarily available on GitHub... keep a copy in a safe place!
       - If you need to generate a new secret the MEC Sandbox will have to be updated accordingly
   - **GitLab:**
     - gitlab.com User Account --> Settings --> Applications --> Add new application
     - _Name:_ ETSI MEC Sandbox
     - _Redirect URI:_ https://try-mec.etsi.org/platform-ctrl/v1/authorize
     - _Confidential:_ Unchecked
     - _Scopes:_ read_user
     - **NOTES:**
       - Once created, keep note of the OAuth App Client ID & Secret

### Deploy Long-term Storage (MinIO + Thanos Compactor)

1. Clone AdvantEDGE repository:
   - Fetch required AdvantEDGE baseline:
     - Clone AdvantEDGE repo to the Long-term Storage VM
     - Place repo in home folder
     - Checkout the required AdvantEDGE version tag
1. Configure Long-term Storage:
   - Configure object store:
     - `vi ~/AdvantEDGE/config/secrets.yaml`
       - Set minio accessKey to object store access key value (minimum 3 chars)
       - Set minio secretKey to object store secret key value (minimum 8 chars)
     - `vi ~/AdvantEDGE/config/thanos.yaml`
       - Set endpoint to VM FQDN
         - **NOTE:** Make sure _/etc/hosts_ file contains entry for VM FQDN that points to VM private IP address
       - Set access_key to object store access key value 
       - Set secret_key to object store secret key value
   - Configure deployment:
     - `vi ~/AdvantEDGE/config/long-term-storage-repocfg.yaml`
       - Set uid & gid
       - Set host name to FQDN
       - Set Thanos compactor retention times
     - Overwrite repocfg file with provided long-term storage repocfg:
       - `cp ~/AdvantEDGE/config/long-term-storage-repocfg.yaml ~/AdvantEDGE/.meepctl-repocfg.yaml`
1. Create MinIO & Thanos Secrets:
   - MinIO:
     - Install Python2:
       - Ubuntu 18.04: `sudo apt install python-minimal`
       - Ubuntu 20.04: `sudo apt install python2-minimal`
     - Install Python-pip:
       - Ubuntu 18.04: `sudo apt install python-pip`
       - Ubuntu 20.04: `sudo apt install python3-pip`
     - Install Pyyaml: `pip install pyyaml`
     - Create secrets:
       - Ubuntu 18.04: `python ~/AdvantEDGE/config/configure-secrets.py set ~/AdvantEDGE/config/secrets.yaml`
       - Ubuntu 20.04: `python3 ~/AdvantEDGE/config/configure-secrets.py set ~/AdvantEDGE/config/secrets.yaml`
   - Thanos:
     - `~/AdvantEDGE/config/configure-thanos-secrets.sh`
   - **NOTE:** Once the kubernetes secrets have been created, it is recommended to remove the config files containing the clear-text secrets
1. Deploy MinIO & Thanos Compactor:
   - Build meepctl:
     - `~/AdvantEDGE/go-apps/meepctl/install.sh`
     - **NOTE:** On first install the IP address & gitdir must be configured
   - Deploy dependency pods:
     - `meepctl deploy dep`
   - Deploy Core pods:
     - `meepctl deploy core`
     - **NOTE:** Core pods deployment will only install the cert-manager rules to trigger certificate installation
1. Log In & Create buckets:
   - Using a browser, access the long-term storage VM via its FQDN
   - Sign in using the provisioned credentials (access key & secret key)
   - Create the following buckets if they do not exist:
     - thanos
     - thanos-archive

### Build & Deploy MEC Sandbox (AdvantEDGE + MEC Sandbox Frontend)

1. Stop AdvantEDGE (if running)
   - Delete user sandboxes
     - Using AdvantEDGE frontend:
       - Login as admin @ https://try-mec.etsi.org
       - Open AdvantEDGE frontend @ https://try-mec.etsi.org/alt
       - Go to Exec Page and delete user sandboxes
     - Manually:
       - `helm ls -A --short | grep <sandbox-name> | xargs -L1 helm delete`
   - Delete core & dependency microservices:
     - `meepctl delete core`
     - `meepctl delete dep`
     - **NOTE:** This will clear all sessions in Redis DB
1. Clone MEC Sandbox & AdvantEDGE repositories:
   - Fetch required MEC Sandbox baseline:
     - Clone this repo to the MEC Sandbox deployment VM
     - Place repo in home folder
   - Fetch required AdvantEDGE baseline:
     - Clone AdvantEDGE repo to the MEC Sandbox deployment VM
     - Place repo in home folder
     - Checkout the required AdvantEDGE version tag
1. Configure MEC Sandbox:
   - Configure object store:
     - `vi ~/AdvantEDGE/config/thanos.yaml` and `vi ~/AdvantEDGE/config/thanos-archive.yaml`
       - Set endpoint to long-term storage VM FQDN
         - **NOTE:** Make sure _/etc/hosts_ file contains entry for VM FQDN that points to VM private IP address (if on same local network)
       - Set access_key to object store access key value 
       - Set secret_key to object store secret key value
   - Configure session & OAuth secrets:
     - `vi ~/mec-sandbox/config/secrets.yaml`
       - Set a unique session encryption key
       - Set the OAuth provider client ID & secrets using the values from the OAuth Applications
   - Configure deployment:
     - `vi ~/mec-sandbox/config/.meepctl-repocfg.yaml`
       - Set Prometheus external labels
         - **NOTE:** This uniquely identifies Prometheus data sources in the long-term storage data buckets
   - Configure Alert Manager slack channel webhook:
     - Create incoming webhook:
       - Create target slack channel in slack workspace
       - Go to `Manage Apps` in workspace settings
       - Search for `Incoming Webhooks` app
       - Click `Add to Slack`
         - **NOTE:** The webhook url can be found by looking at the app configuration
       - Choose a slack channel to post messages to
       - Click `Add Incoming WebHooks integration`
       - Copy the `Webhook URL`
       - **NOTE:** There should be a message in the selected slack channel indicating that an incoming webhook has be added 
     - Set webhook url in Prometheus values override:
       - `vi ~/mec-sandbox/config/values/meep-prometheus.yaml`
       - Set `slack_api_url` to your slack channel webhook url
       - Set `channel` to your slack channel
1. Build & Deploy STF Frontend:
   - SSH to MEC Sandbox deployment VM
   - Build STF Frontend:
     - `cd ~/mec-sandbox`
     - `./build.sh`
   - Deploy STF Frontend & Configuration files:
     - `./deploy.sh`
     - **NOTE:** This updates the `~/.meep/user/` folder with the user frontend, sandbox swagger UI, and user chart values.yaml overrides
     - **NOTE:** This also copies the AdvantEDGE configuration files to their respective locations in your local AdvantEDGE clone `~/AdvantEDGE/`. Running `git status` in your AdvantEDGE clone will list the updated (or new) files.
1. Create Secrets:
   - Session & OAuth:
     - Install Python2: `sudo apt install python-minimal`
     - Install Python-pip: `sudo apt install python-pip`
     - Install Pyyaml: `pip install pyyaml`
     - Create secrets:
       - `python ~/AdvantEDGE/config/configure-secrets.py set ~/AdvantEDGE/config/secrets.yaml`
   - Thanos:
     - `~/AdvantEDGE/config/configure-thanos-secrets.sh`
   - **NOTE:** Once the kuberentes secrets have been created, it is recommended to remove the config files containing the clear-text secrets
1. Build & Deploy AdvantEDGE:
   - Build meepctl:
     - `~/AdvantEDGE/go-apps/meepctl/install.sh`
   - Deploy dependency pods:
     - `meepctl deploy dep`
   - Build & Dockerize:
     - `meepctl build all`
     - `meepctl dockerize all`
   - Prune old docker images:
     - `docker image prune`
   - Deploy Core pods:
     - `meepctl deploy core`
     - **NOTE:** Wait for dep pods to be successfully deployed before deploying core

### Configure Scenarios
If necessary, configure the MEC Sandbox scenarios as follows:

1. Provide _admin_ privileges to a user account
   - SSH to MEC Sandbox machine
   - SSH into _meep-postgis-0_ pod
   - Login & connect to PostgreSQL user role database:
     - `psql -U postgres`
     - `\c meep_platform_ctrl`
   - Add entry to give _admin_ rights to a specific username from a specific provider. Examples:
     - `INSERT INTO users (provider, username, password, sboxname, role) VALUES ('github', 'user1', '', 'sbx-user1','admin');`
     - `INSERT INTO users (provider, username, password, sboxname, role) VALUES ('gitlab', 'user2', '', 'sbx-user2','admin');`
   - List configured user roles to verify proper configuration:
     - `SELECT * from users;`
1. Sign in to MEC Sandbox using an account with _admin_ privileges
   - Login as admin user @ https://try-mec.etsi.org
1. Open the AdvantEDGE frontend
   - Open AdvantEDGE frontend @ https://try-mec.etsi.org/alt
1. Import & save MEC Sandbox scenarios
   - In the AdvantEDGE Configuration view, import the required scenario _yaml_ file
   - Save the imported scenario to the AdvantEDGE backend to make it available in the MEC Sandbox

## Licensing
MEC Sandbox is a private repo with all rights reserved to ETSI.
```
Copyright (c) 2020 ETSI.  All rights reserved.
```