Merge branch '16-expose-all-configurable-parameters-during-kubernetes-deployment' into 'develop'

# OpenSlice Installation Guide with Helm

<!-- 

## Pre-requisites

### Required Software

- A Kubernetes cluster up and running with enough disk to allocate about 12GB of PV storage. (Storage requirements can be changed in the respective PV/PVC templates)

- Helm installed for managing Kubernetes packages.

### Additional Configuration

### Additional Configuration

- **Ingress Controller**: An Nginx ingress controller is required. If you use another type of ingress controller, you'll need to modify `[repo-root]/kubernetes/helm/openslice/templates/openslice-ingress.yaml` to conform to your ingress controller's requirements.

- **Ingress Controller**: An Nginx ingress controller is required. If you use another type of ingress controller, you'll need to modify `[repo-root]/kubernetes/helm/openslice/templates/openslice-ingress.yaml` to conform to your ingress controller's requirements.

# Uninstall helm chart

# Uninstall helm chart

```

```

helm uninstall myopenslice --namespace openslice

helm uninstall myopenslice --namespace openslice

``` -->

# OpenSlice Installation Guide with Kubernetes

## Requirements

### Hardware requirements:

| **Minimum Hardware Requirements** | **Recomended Hardware Requirements** |

| --------------------------------- | ------------------------------------ |

| 4 CPU cores                       | 8 CPU cores                          |

| 8 GB RAM                          | 16 GB RAM                            |

| 20 GB storage                     | 40 GB storage                        |

### Software Requirements:

* **git:** For cloning the project repository.

* **Kubernetes:** A running cluster where OpenSlice will be deployed.

* **Helm:** For managing the deployment of OpenSlice.

* **Ingress Controller:** Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource. An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name-based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic. You must have an Ingress controller to satisfy an Ingress. 

    * An Nginx ingress controller is required, which can be installed using [this guide](https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/). 

    * If you use another type of ingress controller, you'll need to modify `[repo-root]/kubernetes/helm/openslice/templates/openslice-ingress.yaml` to conform to your ingress controller's requirements.

* **Network Load Balancer:** Required for exposing the service (e.g., GCP, AWS, Azure, MetalLB).

* **Domain/IP Address:** Necessary for accessing the application. This should be configured in `[repo-root]/kubernetes/helm/openslice/values.yaml` under `rooturl`.

## Preparing the environment

### 1. Setting Up A Kubernetes Cluster

Refer to the official Kubernetes documentation for setting up a cluster. Ensure your cluster meets the hardware requirements specified above.

### 2. Installing Helm

Helm must be installed on your machine to deploy OpenSlice via Helm charts. Follow the [official Helm installation guide](https://helm.sh/docs/intro/install/).

## Downloading the Project

### 1. Create a New Folder to Download the Project

```bash

mkdir openslice

cd openslice

```

### 2. Download the Project Code

Clone the project code from the GitLab repository. 

**Note:** This process will be simplified once the charts are published in the GitLab registry, requiring only the chart to be pulled.

```bash

git clone https://labs.etsi.org/rep/osl/code/org.etsi.osl.main.git

cd kubernetes/helm/openslice/

```

### 3. Deploy the Helm Chart

Before deploying the Helm chart, ensure you have configured the necessary components as detailed in the [Configure Helm Chart Services](#configure-helm-chart-services) section. By default, the `main` branch is selected for deployment.

We recommend using:

- The **main** branch for the most stable experience.

- The **develop** branch for access to the latest features. For the develop branch installation, it's strongly advisable to follow the [develop documentation](https://osl.etsi.org/documentation/develop/deployment/).

Deploy OpenSlice with Helm:

```bash

helm install myopenslice . --namespace openslice --create-namespace

```

## Configure Helm Chart Services

When deploying OpenSlice with Helm, service configurations are handled through the `values.yaml` file. This file allows you to define all necessary configurations for your deployment, including database credentials, service URLs, and logging levels. Below are examples of how to configure your services in Helm based on your provided values.

### Configuring Services

#### 1. Database Configuration

To configure MySQL and other related services, you can directly set the values in your `values.yaml` file under the `oscreds` and `mysql` sections. For example:

```yaml

oscreds:

  mysql:

    username: "root"

    password: "letmein"

    openslicedb: "osdb"

    keycloak: 

      database: "keycloak"

      username: "keycloak"

      password: "password"

      adminpassword: "keycloakadminpass"

    portal:

      database: "osdb"

      username: "portaluser"

      password: "12345"

```

#### 2. External Services Configuration

For configuring external services like Bugzilla and CentralLog, specify their URLs and credentials in the `values.yaml` file:

```yaml

bugzillaurl: "example.com:443/bugzilla"

bugzillakey: "VH2Vw0iI5aYgALFFzVDWqhACwt6Hu3bXla9kSC1Z"

main_operations_product: "Main Site Operations" // this is the default product to issue tickets

centrallogurl: "http://elk_ip:elk_port/index_name/_doc"

```

Bugzilla should have the following components under the specified product:  

* NSD Deployment Request: Component used to schedule deployment req  

* Onboarding: Issues related to VNF/NSD Onboarding  

* Operations Support: Default component for operations support  

* Validation: Use to track validation processes of VNFs and NSDs  

* VPN Credentials/Access: Used for requesting VPN Credentials/Access   

Also in the 'Main Site Operations' product, a version named 'unspecified' must be created.

#### 3. Keycloak Configuration

Keycloak settings, including the database and admin password, are part of the `oscreds.mysql.keycloak` section. If you need to adjust Keycloak-specific settings like realms or client configurations, you'll likely need to customize your Helm chart further or manage these settings directly within Keycloak after deployment. The Keycloak realm configuration that is imported by default can be found under `kubernetes/helm/openslice/files/keycloak-init/realm-export.json`.

```yaml

oscreds:

  mysql:

    keycloak: 

      database: "keycloak"

      username: "keycloak"

      password: "password"

      adminpassword: "keycloakadminpass"

```

#### 4. Application and Logging Configuration

Application-specific configurations, such as OAuth client secrets and logging levels, can be set in the `spring` section:

```yaml

spring:

  oauthClientSecret: "secret"

  loglevel: "INFO"

```

#### 5. Ingress and Root URL

To configure the ingress controller and root URL for OpenSlice, update the rooturl field with your ingress load balancer IP or domain. This setting is crucial for external access to your application:

```yaml

rooturl: "http://openslice.com" # Example domain

# or

rooturl: "http://3.15.198.35:8080" # Example IP with port

```

#### 6. Persistent Volume for MySQL

For persistent storage, especially for MySQL, define the storage size under the `mysql` section. This ensures that your database retains data across pod restarts and deployments.

```yaml

mysql:

  storage: "10Gi"

```

## Configure Web UI

In folder `kubernetes/helm/openslice/files/org.etsi.osl.portal.web/src/js` edit the `config.js.default` configuration file with your configuration.

```

{      

    TITLE: "Openslice demo",

    WIKI: "{{ .Values.rooturl }}",

    BUGZILLA: "{{ .Values.rooturl }}/bugzilla/",

    STATUS: "{{ .Values.rooturl }}/",

    APIURL: "{{ .Values.rooturl }}",

    WEBURL: "{{ .Values.rooturl }}/nfvportal",

    APIOAUTHURL: "{{ .Values.rooturl }}/auth/realms/openslice",

    APITMFURL: "{{ .Values.rooturl }}/tmf-api/serviceCatalogManagement/v4"

}

```

Rename `config.js.default` to `config.js`. This is **mandatory** for the configuration file to be discoverable.

## Configure TMF Web UI

In the folder `kubernetes/helm/openslice/files/org.etsi.osl.tmf.web/src/assets/config` there are 3 files available for configuration:

* config.prod.json (Basic information + API configuration)

* theming.scss (CSS color palette theming)

* config.theming.json (HTML configuration - Logo, Favicon, Footer)

The first 2 files above (i.e. config.prod.json, theming.scss) are essential for the successful deployment of OpenSlice.

Therefore, ensure that you check the `config.prod.json` and `theming.scss` files and readjust to your deployment if needed.

Default versions of such files exist in this path, which must be renamed to not include the `.default` suffix, before deploying OpenSlice.

E.g. Edit "TITLE", "WIKI", etc properties with your domain title. Also configure TMF's API and Keycloak's location for the web application, if needed.

```

{	      

    "TITLE": "OpenSlice by ETSI",

    "PORTALVERSION":"2024-Q2 1.0.0-SNAPSHOT",

    "WIKI": "https://osl.etsi.org/documentation",

    "BUGZILLA": "{BASEURL}/bugzilla/",

    "STATUS": "http://status.localhost/",

    "WEBURL": "{BASEURL}",

    "PORTAL_REPO_APIURL": "{BASEURL}/osapi",

    "ASSURANCE_SERVICE_MGMT_APIURL": "{BASEURL}/oas-api",

    "APITMFURL": "{BASEURL}/tmf-api",

    "OAUTH_CONFIG" : {

        "issuer": "{BASEURL}/auth/realms/openslice",

        "loginUrl": "{BASEURL}/auth/realms/openslice/protocol/openid-connect/auth",

        "tokenEndpoint": "{BASEURL}/auth/realms/openslice/protocol/openid-connect/token",

        "userinfoEndpoint": "{BASEURL}/auth/realms/openslice/protocol/openid-connect/userinfo",

        "redirectUri": "{BASEURL}/redirect",

        "logoutUrl": "{BASEURL}/auth/realms/openslice/protocol/openid-connect/logout", 

        "postLogoutRedirectUri": "{BASEURL}",

        "responseType": "code",

        "oidc": false,

        "clientId": "osapiWebClientId",

        "dummyClientSecret": "secret",

        "requireHttps": false,

        "useHttpBasicAuth": true,

        "clearHashAfterLogin": false,

        "showDebugInformation": true

    }

}

```

> The {BASEURL} placeholder in the file automatically detects the Origin (Protocol://Domain:Port) of the deployment and applies it to every respective property. E.g. If you are attempting a local deployment of Openslice, then {BASEURL} is automatically translated to "http://localhost". Similarly, you may use {BASEURL} to translate to a public deployment configuration, e.g. "https://portal.openslice.io".

If further customization, apart from the default provided, is needed for branding (Logo, Footer) then `config.theming.json` needs to be created in io.openslice.tmf.web/src/assets/config directory, as follows:

```bash

# Starting from the root project directory

cd kubernetes/helm/openslice/files/org.etsi.osl.tmf.web/src/assets/config

```

```bash

sudo cp config.theming.default.json config.theming.json

```

#This seems irrelevant now.

> ***IMPORTANT NOTE:***

If you want to apply changes to the JSON configuration files without the need to rebuild the application, you have to apply the changes at the `org.etsi.osl.tmf.web/dist/io-openslice-portal-web/assets/config` directory. Although, it is <u>mandatory</u> to also apply these changes to the `org.etsi.osl.tmf.web/src/assets/config` for <u>persistancy</u>, as after any future rebuild of OpenSlice the `/dist` directory is being overwritten along with its contents. The OpenSlice team strongly recommends to always apply your changes to the TMF web UI configuration files at `org.etsi.osl.tmf.web/src/assets/config` and rebuild the application.

### 3. Deploy the Helm Chart

After configuring the services, and editing the docker compose file accordingly, the docker compose instantiation command can be performed.

```bash

cd kubernetes/helm/openslice/

helm install myopenslice . --namespace openslice --create-namespace

```

## Validating deployments and container monitoring

In a Kubernetes environment, you can monitor the status of your deployments and containers using `kubectl`, the Kubernetes command-line tool, which provides powerful capabilities for inspecting the state of resources in your cluster.

## Post installation steps 

After the successful deployment of OpenSlice, to ensure the E2E user experience, **this section is mandatory**. It contains crucial configuration in regard of authentication and user creation.

### Checking the Status of your application's deployment

To check the status of your deployment, use the following commands. The output should be similar:

```bash

kubectl get pods -n openslice -o wide

NAME                               READY   STATUS    RESTARTS   AGE    IP               NODE     NOMINATED NODE   READINESS GATES

activemq-59d4bfdb4b-bvjqr          1/1     Running   0          109s   192.168.43.97    kc-2     <none>           <none>

bugzilla-client-7dd7cb47cb-8qb8m   1/1     Running   0          100s   192.168.12.114   kc-3     <none>           <none>

centrallog-95bbf7867-k8fpt         1/1     Running   0          100s   192.168.12.107   kc-3     <none>           <none>

consul-b5dd76b76-64dzk             1/1     Running   0          107s   192.168.43.90    kc-2     <none>           <none>

keycloak-7c5b6bbc95-k2qfl          1/1     Running   0          105s   192.168.12.106   kc-3     <none>           <none>

manoclient-95f68f4c9-c9t6r         1/1     Running   0          104s   192.168.12.113   kc-3     <none>           <none>

mysql-portal-0                     1/1     Running   0          107s   192.168.43.99    kc-2     <none>           <none>

osom-6d548cf555-q8ptj              1/1     Running   0          104s   192.168.43.93    kc-2     <none>           <none>

osportalapi-5fff744db8-5g4zs       1/1     Running   0          103s   192.168.43.98    kc-2     <none>           <none>

osscapi-6d68b54d97-jn8tz           0/1     Running   0          102s   192.168.12.104   kc-3     <none>           <none>

portalweb-8469d57df4-94tfj         1/1     Running   0          101s   192.168.48.44    kc-nfs   <none>           <none>

tmfweb-868f7bb9c5-x4lfh            1/1     Running   0          102s   192.168.48.43    kc-nfs   <none>           <none>

```

```bash

kubectl get deployments  -n openslice -o wide

NAME              READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS            IMAGES                                      SELECTOR

activemq          1/1     1            1           2m15s   anactivemq            webcenter/activemq:5.14.3                   org.etsi.osl.service=activemq

bugzilla-client   1/1     1            1           2m6s    bugzilla-client       openslice/org.etsi.osl.bugzilla:latest      org.etsi.osl.service=bugzilla-client

centrallog        1/1     1            1           2m6s    centrallog            openslice/org.etsi.osl.centrallog.service   org.etsi.osl.service=centrallog

consul            1/1     1            1           2m13s   aconsul               consul                                      org.etsi.osl.service=consul

keycloak          1/1     1            1           2m11s   keycloak              quay.io/keycloak/keycloak:11.0.3            org.etsi.osl.service=keycloak

manoclient        1/1     1            1           2m10s   manoclient            openslice/org.etsi.osl.mano:latest          org.etsi.osl.service=manoclient

osom              1/1     1            1           2m10s   openslice-osom        openslice/org.etsi.osl.osom:latest          org.etsi.osl.service=osom

osportalapi       1/1     1            1           2m9s    openslice-portalapi   openslice/org.etsi.osl.portal.api:latest    org.etsi.osl.service=osportalapi

osscapi           1/1     1            1           2m8s    openslice-scapi       openslice/org.etsi.osl.tmf.api:latest       org.etsi.osl.service=osscapi

portalweb         1/1     1            1           2m7s    openslice-portalweb   openslice/org.etsi.osl.portal.web:latest    org.etsi.osl.service=portalweb

tmfweb            1/1     1            1           2m8s    openslice-tmfweb      openslice/org.etsi.osl.tmf.web:latest       org.etsi.osl.service=tmfweb

```

```bash

kubectl get services  -n openslice -o wide

NAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)              AGE     SELECTOR

activemq          ClusterIP   10.111.160.120   <none>        8161/TCP,61616/TCP   2m22s   org.etsi.osl.service=activemq

bugzilla-client   ClusterIP   10.101.43.28     <none>        13010/TCP            2m14s   org.etsi.osl.service=bugzilla-client

centrallog        ClusterIP   10.109.15.151    <none>        13013/TCP            2m14s   org.etsi.osl.service=centrallog

consul            ClusterIP   10.101.103.240   <none>        8500/TCP,8600/TCP    2m21s   org.etsi.osl.service=consul

keycloak          ClusterIP   10.110.216.62    <none>        8080/TCP,8443/TCP    2m19s   org.etsi.osl.service=keycloak

manoclient        ClusterIP   10.108.112.84    <none>        13011/TCP            2m18s   org.etsi.osl.service=manoclient

mysql-portal      ClusterIP   None             <none>        3306/TCP             2m19s   org.etsi.osl.service=mysql-portal

osom              ClusterIP   10.105.173.85    <none>        13100/TCP            2m18s   org.etsi.osl.service=osom

osportalapi       ClusterIP   10.104.121.164   <none>        13000/TCP            2m17s   org.etsi.osl.service=osportalapi

osscapi           ClusterIP   10.108.6.161     <none>        13082/TCP            2m16s   org.etsi.osl.service=osscapi

portalweb         ClusterIP   10.97.126.98     <none>        80/TCP               2m15s   org.etsi.osl.service=portalweb

tmfweb            ClusterIP   10.98.56.82      <none>        80/TCP               2m15s   org.etsi.osl.service=tmfweb

```

### Accessing Logs for Troubleshooting

If a pod is not in the expected state, you can access its logs for troubleshooting:

```bash

kubectl logs <pod-name> -n openslice

```

### Configure Keycloak server

The Keycloack server is managing authentication and running on a container at port 8080. It is also proxied to your host via the ingress resource under http://<your-domain>/auth. 

- Navigate to http://<your-domain>/auth/ or https://<your-domain>/auth/, (http://ipaddress:8080/auth/ or https://ipaddress:8443/auth/ which are directly accessible without proxy) 

- Navigate to Administration Console 

- Login with the credentials from section [Keycloak Configuration](#3-keycloak-configuration). Default values are:

    - user: admin 

    - password: KEYCLOAK_PASSWORD

> if you are running in HTTP you will get a message: HTTPS required.

To resolve this issue <u>when running in HTTP</u>: 

- Select the master realm from top left corner

- Go to login Tab and select "Require SSL": None

- Repeat for realm Openslice

> If you are running in HTTPS, then "Require SSL" can be left unchanged to external requests.

#### 1. Configure redirects

Navigate to realm Openslice > client > osapiWebClientId and change the Root URL to your domain. 

Also, insert your domain, e.g. http://example.org/*, at:

* Valid Redirect URIs

* Web Origins

#### 2. Configure email

Keycloak allows new users to register. Subsequently, this will also allow new users to register to the OpenSlice portal.

On Tab Login > check User registration, Verify email, Forgot password etc.

Also, enter the details on Realm > Email > Enable Authentication.

#### 3. Add an OpenSlice admin user

This step is mandatory so as to access the OpenSlice Web UI. To add an OpenSlice admin user you must:

- Navigate to manage/users and add an OpenSlice admin user, e.g. username=admin. 

- Set a password

- Navigate to Role Mappings and add ADMIN and MENTOR to Assigned Roles.

> That user is different from the Keycloak admin user. It is required to login and browse the OpenSlice Web UI. The Roles ADMIN and MENTOR guarantee full access through the Openslice UI, thus such a user is always required.

***IMPORTANT: The following is not currently supported in Kubernetes installation.**

### Keycloak at localhost

> **This is an important step if you run Keycloak on localhost!**

1 - Edit your Hosts File, adding the line below

```127.0.0.1 keycloak```

Hosts File Location:

 - In Linux/Unix, the file's location is at /etc/hosts 

 - In Windows, its location is at c:\Windows\System32\Drivers\etc\hosts

2 - Replace http://localhost/auth/ with http://keycloak:8080/auth/ in your Keycloak config for AngularJS and Angular (see examples below).

> Explanation

Nginx uses the http://keycloak:8080 URL, which is accessible via the internal docker system's network.

The Front-end (TS/Angular) shall also use the http://keycloak:8080.

This way, you will not get the invalid token error, as the API is acquiring the token from http://keycloak:8080 (internally) and the Front-end is getting verified by an issuer at the same URL, as well.

2.1 - For the Angular configuration (TMF portal UI), navigate to  org.etsi.osl.tmf.web/src/assets/config and edit config.prod.json

```bash

# Starting from the root project directory

cd org.etsi.osl.tmf.web/src/assets/config

```

```bash

nano config.prod.json

```

```

After editing it should look like the example bellow:

```yaml

{	      

  "TITLE": "OpenSlice by ETSI",

  "PORTALVERSION":"2023-Q3 1.2.0-SNAPSHOT",

  "WIKI": "https://openslice.readthedocs.io/en/stable/",

  "BUGZILLA": "{BASEURL}/bugzilla/",

  "STATUS": "http://status.localhost/",

  "WEBURL": "{BASEURL}",

  "PORTAL_REPO_APIURL": "{BASEURL}/osapi",

  "ASSURANCE_SERVICE_MGMT_APIURL": "{BASEURL}/oas-api",

  "APITMFURL": "{BASEURL}/tmf-api",

  "OAUTH_CONFIG" : {

      "issuer": "http://keycloak:8080/auth/realms/openslice",

      "loginUrl": "http://keycloak:8080/auth/realms/openslice/protocol/openid-connect/auth",

      "tokenEndpoint": "http://keycloak:8080/auth/realms/openslice/protocol/openid-connect/token",

      "userinfoEndpoint": "http://keycloak:8080/auth/realms/openslice/protocol/openid-connect/userinfo",

      "redirectUri": "{BASEURL}/redirect",

      "logoutUrl": "http://keycloak:8080/auth/realms/openslice/protocol/openid-connect/logout", 

      "postLogoutRedirectUri": "{BASEURL}",

      "responseType": "code",

      "oidc": false,

      "clientId": "osapiWebClientId",

      "dummyClientSecret": "secret",

      "requireHttps": false,

      "useHttpBasicAuth": true,

        clearHashAfterLogin": false,

      "showDebugInformation": true

  }

}

```

> Note the difference in changing {BASEURL} -> http://keycloak:8080

> If you want the changes to take place immediately without rebuilding the project, then repeat the process for org.etsi.osl.tmf.web/dist/org.etsi.osl.tmf.web/assets/config/config.prod.json

2.2 - For the AngularJS configuration (NVF portal UI), navigate to org.etsi.osl.portal.web/src/js and edit config.js

```bash

# Starting from the root project directory

cd org.etsi.osl.portal.web/src/js

```

```bash

nano config.js

```

after editing it should look like the example bellow:

```

var appConfig = angular.module('portalwebapp.config',[]);

appConfig.factory('APIEndPointService', function() {

   return {	      

      TITLE: "OpenSlice by ETSI",

      WIKI: "https://openslice.readthedocs.io/en/stable/",

      BUGZILLA: "ROOTURL/bugzilla/",

      STATUS: "ROOTURL/status/",

      APIURL: "http://localost:13000",

      WEBURL: "ROOTURL/nfvportal",

      APIOAUTHURL: "ROOTURL/auth/realms/openslice",

      APITMFURL: "ROOTURL/tmf-api/serviceCatalogManagement/v4"

   };

});

```

> Note the difference in "APIOAUTHURL" property

### NFV Orchestrator Configuration

After successfully deploying and configuring OpenSlice, you may configure its environment (e.g. the NFVO) that will facilitate the deployment of NFV artifacts.

See [NFV Orchestrator Configuration](./nfvoconfig.md).

# This is the version number of the application being deployed. This version number should be

# This is the version number of the application being deployed. This version number should be

# incremented each time you make changes to the application. Versions are not expected to

# incremented each time you make changes to the application. Versions are not expected to

# follow Semantic Versioning. They should reflect the version the application is using.

# follow Semantic Versioning. They should reflect the version the application is using.

appVersion: 1.16.0

appVersion: 1.0.0-SNAPSHOT

apiVersion: v1

kind: Config

clusters:

- name: development-cluster

  cluster:

    certificate-authority: path/to/ca.crt # Path to certificate authority file

    server: https://123.45.67.89:6443 # Cluster API server address

contexts:

- name: dev-user@development-cluster

  context:

    cluster: development-cluster

    namespace: development # Default namespace

    user: dev-user

current-context: dev-user@development-cluster

users:

- name: dev-user

  user:

    client-certificate: path/to/cert.crt # Path to the client certificate

    client-key: path/to/key.key # Path to the client key