README.md

# Proof of Concept

## Candidate CAMARA API - QoD Provisioning

For this first proof of concept, we decided to rely on the [CAMARA QoD Provisioning API](https://editor.swagger.io/?url=https://raw.githubusercontent.com/camaraproject/QualityOnDemand/r1.2/code/API_definitions/qod-provisioning.yaml ).

Such API has the following endpoints:

![](Documentation/Pictures/CAMARA-QoDProvisioning-API-Endpoints.png)

### Mapping to TMF Service Characteristics (of the operator’s Service)

Having chosen the candidate API, the first step is to find a way to map the possible requests to TMF Service characteristics of the operator’s service. By looking at API’s specification, it is clear that at least 3 operations are required: (i) the creation of a QoD profile, (ii) its deletion, and (iii) listing all active QoD profiles. Therefore, we can proceed with evaluating the payload required for creating a QoD Provisioning. This payload involves various fields, which can be translated to the TMF Service Characteristics:

- *qodProv.device.phoneNumber*
- *qodProv.device.networkAccessIdentifier*
- *qodProv.device.ipv4Address.privateAddress*
- *qodProv.device.ipv4Address.publicAddress*
- *qodProv.device.ipv4Address.publicPort*
- *qodProv.device.ipv6Address*
- *qodProv.qosProfile*
- *qodProv.sink*
- *qodProv.sinkCredential.credentialType*

In order to support interaction with OSL’s CAMARAaaS APIs, the operator service must be designed, at least, with these characteristics. 

Still, since there are various operations that can take place (CREATE and DELETE), it is also needed a characteristic to map this. Therefore, the operator’s service must also have a characteristics titled *qodProv.operation*. The DELETE operation is achieved based on a provisioning Id, and therefore another characteristics is needed: *qodProv.provisioningId.*

Finally, it is required a characteristic to store the provisionings that were enforced by the operator’s service. We can define this characteristic as *camaraResults*.

Therefore, for an operator’s service to be controlled by OSL’s CAMARA APIs, it needs to be designed with, at least, the following characteristics:

- *qodProv.device.phoneNumber*
- *qodProv.device.networkAccessIdentifier*
- *qodProv.device.ipv4Address.privateAddress*
- *qodProv.device.ipv4Address.publicAddress*
- *qodProv.device.ipv4Address.publicPort*
- *qodProv.device.ipv6Address*
- *qodProv.qosProfile*
- *qodProv.sink*
- *qodProv.sinkCredential.credentialType*
- *qodProv.operation*
- *qodProv.provisioningId*
- *camaraResults*

Additional characteristics are fully supported. Those can be custom characteristics that are required by the Operator’s Service.

In regard to the *camaraResults* characteristic, to allow interoperability, it must store a Stringified JSON Array with the enforced QoD Provisionings. **The schema of each provisioning should be the one defined in CAMARA’s QoD Provisioning API Specification.** 

### TMF Service Characteristics of the CAMARAaaS APIs

Considering the interactions that shall take place between the CAMARAaaS API and the Operator’s running Service and the architecture introduced before, it is clear that CAMARA APIs must interface with OSL’s Active MQ broker. Therefore, TMF Specific Service Characteristics are required to pass this information to the CAMARA APIs deployed through OSL:

- messageBroker.address - OSL's ActiveMQ Address (e.g. 10.10.10.10)
- messageBroker.port - OSL's ActiveMQ Port
- messageBroker.username - OSL’s ActiveMQ Username
- messageBroker.password - OSL’s ActiveMQ Password

Additionally, we also need another Service Characteristic to store the UUID of the Operator’s running Service that will be controlled through the CAMARA API:

- serviceUnderControl.uuid

Considering that the CAMARA API will be orchestrated by OSL, the client does not know where the API will be deployed, nor the credentials he should use to access it. Therefore, 4 additional characteristics are required. These will be automatically updated by OSL after the CAMARA API Service is deployed:

- camaraAPI.url - URL of the CAMARA API orchestrated by this service (view-only). This field will be automatically populated when the CAMARA API starts running
- camaraAPI.username - Username of the CAMARA API orchestrated by this service (view-only). This field will be automatically populated when the CAMARA API starts running
- camaraAPI.password - Password of the CAMARA API orchestrated by this service (view-only). This field will be automatically populated when the CAMARA API starts running
- camaraAPI.status - This characteristic (view-only) will be populated with the CAMARA API status (RUNNING, NOT_RUNNING)

Additionally, you may create a characteristic titled “*camaraAPI.results*”, which you can use to have visibility of the QoD Provisionings processed by the API, at OSL level. Still, this characteristic is not required.

Therefore, OSL’s CAMARA APIs must offer the following TMF Service Characteristics:

- messageBroker.address
- messageBroker.port
- messageBroker.username
- messageBroker.password
- serviceUnderControl.uuid
- camaraAPI.url
- camaraAPI.username
- camaraAPI.password
- camaraAPI.status
- camaraAPI.results

### Broker Connection

This API has a *ServiceEventManager* class that communicates with OpenSlice's ActiveMQ broker through two topics:

- `CATALOG.UPD.SERVICE`: Topic for catalog updates.
- `EVENT.SERVICE.ATTRCHANGED`: Topic for service attribute changes.

#### CATALOG.UPD.SERVICE

Whenever a new provisioning is created for an UE, the *ServiceEventManager*'s *update_service* method is called. This method sends a a service update message through OpenSlice's *CATALOG.UPD.SERVICE* topic. When OSL receives the request, it updates the Service with the new characteristics, which are then caught by the correspondent K8s Operator. After processing the request, the Operator adds the result to the Service-related CR *camaraResults* characteristic.

#### EVENT.SERVICE.ATTRCHANGED

The *ServiceEventManager* subscribes to this topic to obtain and process the update messages regarding the specified UE QoD Profile Enforcer OSL service. Whenever this service's characteristics are updated in OSL, this class catches the update message. Then, the class extracts the *camaraResults* characteristic, which contains all QoS provisionings applied to the UEs. 

These results are then processed by the *CamaraResultsProcessor* class, which updates each provisioning accordingly in the database.




## How To / Demonstration

### 1. Dummy Operator Service Service Design
We will start by looking at the dummy operator’s service we have created to demonstrate this Add-on. This Service will be offered as simple Custom Resource. You may find its Custom Resource Definition under `/DummyOperatorService/crd.yaml` Look at the CRD fields. Please notice that these were defined according what we disclosed before.

The first step is then to install this CRD in your kubernetes cluster. To this end, you may use the following command: `make create-dummy-operator-crd`

After creating the CRD in your Kubernetes cluster, you may access OSL’s Resource Inventory and you will see the just created resource there.

![](./Documentation/Pictures/DummyOperatorService-ResourceInventory.png)

The next step is to create a RFS Service to expose this resource. To do so, you may read the [Exposing Kubernetes Operators as a Service : Offering "Calculator as a Service" through OpenSlice](https://osl.etsi.org/documentation/latest/service_design/examples/ExposingCRDs_aaS_Example_Calculator/ExposingCRDs_aaS_Example_Calculator/) documentation page. 

Regarding the RFS Service, you must set the following characteristics:

- _CR_CHECKVAL_AVAILABLE = RUNNING
- _CR_CHECK_FIELD = spec.status

By setting this characteristics, you will rely on the value of `spec.status` to set the service as `active`. Ideally, the operator would have implemented an Operator for this Custom Resource. However, for demonstration purposes, we will use a Supervision rule to set `spec.status` to `RUNNING`

Then, you can proceed to create a CFS Service, which will incorporate the just created RFS Service. More information is available at: [Exposing Kubernetes Operators as a Service : Offering "Calculator as a Service" through OpenSlice](https://osl.etsi.org/documentation/latest/service_design/examples/ExposingCRDs_aaS_Example_Calculator/ExposingCRDs_aaS_Example_Calculator/). To create the CFS Service characteristics, you may use the Service Specification available at `/DummyOperatorService/OSLArtifacts/DummyOperatorService-CFS-Specification.json` . You may manually create the CFS Service, or you may onboard this Service Specification by making a POST request to *[{{url}}/tmf-api/serviceCatalogManagement/v4/serviceSpecification](https://www.notion.so/CAMARAaaS-OSL-15e11fa2ed8d80808254c87d9393cf51?pvs=21).*

After creating the Service Specification, you should mark this Service as a Bundle. Then, go to “Service Specification Relationships” and add the RFS Service.

Regarding the LCM Rules for the CFS Service, you should configure the following ones:


**[Pre-Provision Rule]**

![](./Documentation/Pictures/DummyOperatorService-Pre-Provision-Rule.png)


```java
{
java.util.HashMap<String,String> charvals = new java.util.HashMap<>();
charvals.put("_CR_SPEC",String.format("""
apiVersion: org.etsi.osl/v1
kind: DummyOperatorService
metadata:
  name: _to_be_replaced_by_osl
  namespace: default
spec:
  status: "%s"
"""
, "RUNNING"));
setServiceRefCharacteristicsValues("Dummy Operator Service - RFS", charvals);
}
```


**[Supervision Rule]**

![](./Documentation/Pictures/DummyOperatorService-Supervision-Rule.png)

```java
{
java.util.HashMap<String,String> charvals = new java.util.HashMap<>();
charvals.put("_CR_SPEC",String.format("""
apiVersion: org.etsi.osl/v1
kind: DummyOperatorService
metadata:
  name: _to_be_replaced_by_osl
  namespace: default
spec:
  qodProv:
    operation: "%s"
    provisioningId: "%s"
    device:
      phoneNumber: "%s"
      networkAccessIdentifier: "%s"
      ipv4Address:
        publicAddress: "%s"
        privateAddress: "%s"
        publicPort: %d
      ipv6Address: "%s"
    qosProfile: "%s"
    sink: "%s"
    sinkCredential:
      credentialType: "%s"
"""
, getCharValAsString("qodProv.operation"), getCharValAsString("qodProv.provisioningId"), getCharValAsString("qodProv.device.phoneNumber"), getCharValAsString("qodProv.device.networkAccessIdentifier"), getCharValAsString("qodProv.device.ipv4Address.publicAddress"), getCharValAsString("qodProv.device.ipv4Address.privateAddress"), getCharValNumber("qodProv.device.ipv4Address.publicPort"), getCharValAsString("qodProv.device.ipv6Address"), getCharValAsString("qodProv.qosProfile"), getCharValAsString("qodProv.sink"), getCharValAsString("qodProv.sinkCredential.credentialType")));
setServiceRefCharacteristicsValues("Dummy Operator Service - RFS", charvals);
}
setCharValFromStringType("camaraResults", getServiceRefPropValue("Dummy Operator Service - RFS", "serviceCharacteristicValue", "spec.camaraResults"));

```

You can find the  `_CR_SPEC` template used for the pre-provision rule at `/DummyOperatorService/OSLArtifacts/cr-template-pre-provision.yaml` . The `_CR_SPEC` template used for the supervision rule is available at `/DummyOperatorService/OSLArtifacts/cr-template-supervision.yaml` 

After that, you may expose this service via OSL’s Service Catalog, and order it. You do not need to configure any characteristics when ordering this Service. Confirm that the service order was completed, both RFS and CFS Services are active, and a Custom Resource of type *DummyOperatorService* was created in your Kubernetes Cluster. See images below.

![DummyOperatorService-ServiceOrder.png](./Documentation/Pictures/DummyOperatorService-ServiceOrder.png)

![DummyOperatorService-ServiceOrder.png](./Documentation/Pictures/DummyOperatorService-CustomResource.png)

### 2. CAMARA QoD Provisioning API

Then, we can proceed to design the CAMARAaaS QoD Provisioning API. To this end, OSL’s team has implemented CAMARA’s QoD Provisioning API, created a CRD to offer it, and developed a Kubernetes Operator to deal with its internal logic. Start by packaging the API in a docker image and pushing it to a docker repository.

Open the file `Makefile` and update the repository to where you will push the docker image. Update the variable `REPOSITORY_HOST` . You may also choose to update the other variables, but it is not required. After this, run  `make build-api-docker-image`. This command will build, tag, and push the API docker image to the repository you chose.


### 3. CAMARA QoD Provisioning API - Kubernetes Operator

The previous docker image shall make available the CAMARA QoD Provisioning API. However, these APIs will be made available through Custom Resources of Type `CAMARAaaS-QoDProvisiongAPI` . Therefore, we also need a Kubernetes Operator to manage these resources. The Operator’s code can be found under `/QoDProvisioningAPI/Operator` . There, you have the source code of the Operator, as well as an Helm Chart to install it. 

Start by building and pushing the Operator’s docker image to your repository. Run `make operator-docker-image`. Then, install the operator in your Kubernetes cluster. This action will result in the creation of the Custom Resource Definition for the CAMARA QoD Provisioning API  Resources, and deploy the operator in the cluster. It is this operator that will manage the  CAMARA QoD Provisioning API  Resources, being responsible, for instance, for the deployment a Kubernetes pod and service that expose the CAMARA QoD Provisioning API. To install the Operator, run the following command: 

```bash
helm install camaraaas-qod-prov-operator ./QoDProvisioningAPI/Operator/chart --set operator.image=<your_operator_image> --set camaraQoDAPI.image=<your_api_image>  --namespace <namespace_where_the_operator_shall_be_deployed> --create-namespace
```

To simplify things, you may also run: `make install-operator` . After this, check if the operator is running through the `make get-operator-logs` command. 

If everything went ok, you should have a new CRD in your Kubernetes cluster. Run this command to verify if it was created:  `kubectl describe crd camaraaas-qod-provisioning-apis.org.etsi.osl`.

Before designing the service in OSL, let us first create a Custom Resource of type `CAMARAaaS-QoDProvisiongAPI` to validate that the operator is behaving according to what is expected. To this end, you may use the test custom resource available at `/QoDProvisioningAPI/Operator/test-cr.yaml` . Before creating the resource, you need to update the fields: *spec.messageBroker.address*, *spec.messageBroker.port, spec.messageBroker.username, spec.messageBroker.password*, with the values that relate with your OSL instance. Most likely, the values will be the following ones:

- *spec.messageBroker.address: <your OSL address>*
- s*pec.messageBroker.port*: 61613
- *spec.messageBroker.username: artemis*
- *spec.messageBroker.password: artemis*

For now, you do not need to update the field serviceUnderControl.uuid. You may leave it as is.

After these updates, create the Custom Resource by running the command: `make create-operator-test-cr`. 

When the Custom Resource is created, its operator will deploy the CAMARA QoD API in a pod and expose it via a K8s Node Port. The URL where the API is available is published under the CR field `spec.camaraAPI.url` (e.g.[http://10.255.28.73:32630](http://10.255.28.73:32630/)). Check this field by running `make describe-operator-test-cr`. To confirm the API is running, access *<URL>/docs*. You should see the following:

![](./Documentation/Pictures/QoDProvisioningAPI-Docs.png)

If you see this page, the CAMARA QoD Provisioning API Custom Resources and their operator is working. You may delete the Custom Resource you created. Run the following command: `make delete-operator-test-cr`.

### 4. CAMARA QoD Provisioning API - Service Design

Now we can proceed to create a Service Specification that maps the CAMARA QoD Provisioning APICustom Resource.

The next step is to create a RFS Service to expose this resource. To do so, you may read the [Exposing Kubernetes Operators as a Service : Offering "Calculator as a Service" through OpenSlice](https://osl.etsi.org/documentation/latest/service_design/examples/ExposingCRDs_aaS_Example_Calculator/ExposingCRDs_aaS_Example_Calculator/) documentation page. 

Regarding the RFS Service, you must set the following characteristics:

- _CR_CHECKVAL_AVAILABLE = RUNNING
- _CR_CHECK_FIELD = spec.camaraAPI.status

By setting this characteristics, you will rely on the value of `spec.camaraAPI.status` to set the service as `active`. The previous operator, when it deploys the CAMARA QoD Provisioning API will set that CR field to `RUNNING`.

Then, you can proceed to create a CFS Service, which will incorporate the just created RFS Service. More information is available at: [Exposing Kubernetes Operators as a Service : Offering "Calculator as a Service" through OpenSlice](https://osl.etsi.org/documentation/latest/service_design/examples/ExposingCRDs_aaS_Example_Calculator/ExposingCRDs_aaS_Example_Calculator/). To create the CFS Service characteristics, you may use the Service Specification available at `/QoDProvisioningAPI/OSLArtifacts/CAMARAaaS-QoD-Provisioning-API-CFS-Specification.json` . You may manually create the CFS Service, or you may onboard this Service Specification by making a POST request to *[{{url}}/tmf-api/serviceCatalogManagement/v4/serviceSpecification](https://www.notion.so/CAMARAaaS-OSL-15e11fa2ed8d80808254c87d9393cf51?pvs=21).*

After creating the Service Specification, you should mark this Service as a Bundle. Then, go to “Service Specification Relationships” and add the RFS Service.

Regarding the LCM Rules for the CFS Service, you should configure the following ones:

**[Pre-Provision Rule]**

![](./Documentation/Pictures/CAMARAaaS-QoD-Prov-API-Pre-Provision-Rule.png)

```java
{
java.util.HashMap<String,String> charvals = new java.util.HashMap<>();
charvals.put("_CR_SPEC",String.format("""
apiVersion: org.etsi.osl/v1
kind: CAMARAaaS-QoDProvisiongAPI
metadata:
  name: _to_be_replaced_by_osl_
spec:
  messageBroker:
    address: "%s"
    port: %d
    username: "%s"
    password: "%s"
  serviceUnderControl:
    uuid: "%s"
"""
, getCharValAsString("messageBroker.address"), getCharValNumber("messageBroker.port"), getCharValAsString("messageBroker.username"), getCharValAsString("messageBroker.password"), getCharValAsString("serviceUnderControl.uuid")));
setServiceRefCharacteristicsValues("CAMARAaaS - QoD Provisioning API - RFS", charvals);
}

```

**[Supervision Rule]**

![CAMARAaaS-QoD-Prov-API-Supervision-Rule.png](./Documentation/Pictures/CAMARAaaS-QoD-Prov-API-Supervision-Rule.png)

```java
{
java.util.HashMap<String,String> charvals = new java.util.HashMap<>();
charvals.put("_CR_SPEC",String.format("""
apiVersion: org.etsi.osl/v1
kind: CAMARAaaS-QoDProvisiongAPI
metadata:
  name: _to_be_replaced_by_osl_
spec:
  messageBroker:
    address: "%s"
    port: %d
    username: "%s"
    password: "%s"
  serviceUnderControl:
    uuid: "%s"
"""
, getCharValAsString("messageBroker.address"), getCharValNumber("messageBroker.port"), getCharValAsString("messageBroker.username"), getCharValAsString("messageBroker.password"), getCharValAsString("serviceUnderControl.uuid")));
setServiceRefCharacteristicsValues("CAMARAaaS - QoD Provisioning API - RFS", charvals);
}
setCharValFromStringType("camaraAPI.status", getServiceRefPropValue("CAMARAaaS - QoD Provisioning API - RFS", "serviceCharacteristicValue", "spec.camaraAPI.status"));
setCharValFromStringType("camaraAPI.url", getServiceRefPropValue("CAMARAaaS - QoD Provisioning API - RFS", "serviceCharacteristicValue", "spec.camaraAPI.url"));
setCharValFromStringType("camaraAPI.username", getServiceRefPropValue("CAMARAaaS - QoD Provisioning API - RFS", "serviceCharacteristicValue", "spec.camaraAPI.username"));
setCharValFromStringType("camaraAPI.password", getServiceRefPropValue("CAMARAaaS - QoD Provisioning API - RFS", "serviceCharacteristicValue", "spec.camaraAPI.password"));
setCharValFromStringType("camaraAPI.results", getServiceRefPropValue("CAMARAaaS - QoD Provisioning API - RFS", "serviceCharacteristicValue", "spec.camaraAPI.results"));

```

You can find the  `_CR_SPEC` template used for both rules at `/QoDProvisioningAPI/OSLArtifacts/cr-template.yaml` . 

After that, you may expose this service via OSL’s Service Catalog, and order it. When you order it, you will be prompted to configure some characteristics:

- messageBroker.address
- messageBroker.port
- messageBroker.username
- messageBroker.password
- serviceUnderControl.uuid

In `serviceUnderControl.uuid` you should input the UUID of the Service (in the Service Inventory) that you ordered before: the one that relates with the Dummy Operator. For this tutorial, we have used the following characteristic values:


![CAMARAaaS-QoD-Prov-API-Ordering.png](./Documentation/Pictures/CAMARAaaS-QoD-Prov-API-Ordering.png)

Confirm that the service order was completed, both RFS and CFS Services are active, and a Custom Resource of type CAMARAaaS-QoDProvisiongAPI was created in your Kubernetes Cluster. See images below (`kubectl describe camaraaas-qod-provisioning-apis <name> -n <namespace>`)

![](./Documentation/Pictures/CAMARAaaS-QoD-Prov-API-Active-Services.png)

![](./Documentation/Pictures/CAMARAaaS-QoD-Prov-API-CR.png)


Additionally, in OSL, you may see the URL where the QoD Provisioning API is exposed. To do so, please see the characteristics of the CAMARAaaS QoD Provisioning API CFS. See image below.

![](./Documentation/Pictures/CAMARAaaS-QoD-Prov-API-Characteristics.png)

### 5. Validation

Now we can test if the two services are communicating. To do so, you should create a QoD Provisioning via the API that was just deployed. You may do that, using this command:

```bash
# You must update the url to correspond to your API instance.
curl --location 'http://10.255.28.73:31637/device-qos' \
--header 'Content-Type: application/json' \
--data-raw '{
  "device": {
    "phoneNumber": "+987654321",
    "networkAccessIdentifier": "987654321@example.org",
    "ipv4Address": {
      "publicAddress": "203.0.112.12",
      "publicPort": 59765
    },
    "ipv6Address": "2001:db8:85a3:8d3:1319:8a2e:370:7344"
  },
  "qosProfile": "QOS_PROFILE_A",
  "sink": "https://endpoint.example.com/"
}'
```

You should have received a response similar to this one: 

```bash
{"device":{"phoneNumber":"+987654321","networkAccessIdentifier":"987654321@example.org","ipv4Address":{"publicAddress":"203.0.112.12","privateAddress":null,"publicPort":59765},"ipv6Address":"2001:db8:85a3:8d3:1319:8a2e:370:7344"},"qosProfile":"QOS_PROFILE_A","sink":"https://endpoint.example.com/","sinkCredential":{"credentialType":null},"provisioningId":"cb55f9e9-802e-4898-95f5-d1a5a2552483","startedAt":"2024-12-17T15:49:21.995399","status":"REQUESTED","statusInfo":null}
```

Now, if everything is working properly, the characteristics of the Dummy Operator Service you referenced should have been update. You should now see these characteristics:

![DummyOperatorService-Characteristics-After-CAMARA-Invoking.png](./Documentation/Pictures/DummyOperatorService-Characteristics-After-CAMARA-Invoking.png)

You may also query the  QoD Provisioning API to check the status of your provisioning.

```bash
curl --location 'http://10.255.28.73:31637/device-qos/cb55f9e9-802e-4898-95f5-d1a5a2552483'
# notice the "provisioningId":"cb55f9e9-802e-4898-95f5-d1a5a2552483" above
```

If you do so, you will receive the following response:

```bash
{"device":{"phoneNumber":"+987654321","networkAccessIdentifier":"987654321@example.org","ipv4Address":{"publicAddress":"203.0.112.12","privateAddress":null,"publicPort":59765},"ipv6Address":"2001:db8:85a3:8d3:1319:8a2e:370:7344"},"qosProfile":"QOS_PROFILE_A","sink":"https://endpoint.example.com/","sinkCredential":{"credentialType":null},"provisioningId":"cb55f9e9-802e-4898-95f5-d1a5a2552483","startedAt":"2024-12-17T15:49:21.962746","status":"REQUESTED","statusInfo":null}
```

As there is no logic behind the Dummy Operator Service, the provisioning will remain with the status “REQUESTED”. However, we can simulate that Dummy Operator Service enforced a QOS enforcement, by patching its Custom Resource:

```bash
kubectl patch dummy-operator-services <name> -n <namespace> \
--type='json' -p='[{"op": "replace", "path": "/spec/camaraResults", "value": "[{\"device\": {\"ipv4Address\": {\"publicAddress\": \"203.0.112.12\", \"publicPort\": 59765}, \"ipv6Address\": \"2001:db8:85a3:8d3:1319:8a2e:370:7344\", \"networkAccessIdentifier\": \"987654321@example.org\", \"phoneNumber\": \"+987654321\"}, \"provisioningId\": \"cb55f9e9-802e-4898-95f5-d1a5a2552483\", \"qosProfile\": \"QOS_PROFILE_A\", \"sink\": \"https://endpoint.example.com/\", \"sinkCredential\": {}, \"status\": \"AVAILABLE\", \"startedAt\": \"2024-12-15T11:00:00Z\"}]"}]'
```

When you do this, the `camaraResults` characteristic in the Dummy Operator Service will be updated to:

![](./Documentation/Pictures/DummyOperatorService-Characteristics-After-CAMARA-Invoking.png)

After a while, if you check the characteristics of the CAMARAaaS QoD Provisioning API CFS, you will also see that the characteristic `camaraAPI.results` was updated.

![](./Documentation/Pictures/CAMARAaaS-QoD-Prov-API-Characteristics-After-CAMARA-Invoking.png)

Finally, execute this request again:

```bash
curl --location 'http://10.255.28.73:31637/device-qos/cb55f9e9-802e-4898-95f5-d1a5a2552483'
# notice the "provisioningId":"cb55f9e9-802e-4898-95f5-d1a5a2552483" above
```

You should receive the following response.

```bash
{"device":{"phoneNumber":"+987654321","networkAccessIdentifier":"987654321@example.org","ipv4Address":{"publicAddress":"203.0.112.12","privateAddress":null,"publicPort":59765},"ipv6Address":"2001:db8:85a3:8d3:1319:8a2e:370:7344"},"qosProfile":"QOS_PROFILE_A","sink":"https://endpoint.example.com/","sinkCredential":{"credentialType":null},"provisioningId":"cb55f9e9-802e-4898-95f5-d1a5a2552483","startedAt":"2024-12-15T11:00:00","status":"AVAILABLE","statusInfo":null}
```

Notice the `"status":"AVAILABLE"` . This means the 2 services are communicating. Now, you just have to implement your own Operator Service Kubernetes Operator, and you may use OSL’s CAMARAaaS Add-on to expose it through a CAMARA API.