Assessment updates

If automateable vulnerability scanners are available the product shall satisfy the following with respect to the most comprehensive of such scanners.

- **[REQ-EXPLOIT-0a]** The product shall have no known exploitable vulnerabilities discovered by scans.

- **[REQ-EXPLOIT-0c]** For each detected exploitable vulnerability, the product shall have the risk mitigated.

- **[REQ-EXPLOIT-0c]** For each identified exploitable vulnerability, the product shall have the risk mitigated.

- **[REQ-EXPLOIT-0d]** The used vulnerability scanner shall be fit for the purpose in detail, method and depth.

Recognising that there may be vulnerabilities discovered between the time that a product is placed on the market and the time of that product's first use, and that the product should be free from known exploitable vulnerabilities both when first made available and when first used by the system user.

### 6.2.0.1 REQ-TECH-1

**Objective:** Agreed Cryptographic Mechanisms specification is followed and the product shows it does that.<br/>

**Objective:** The implementation and operation follows the Agreed Cryptographic Mechanisms specification and the Annex K.<br/>

**Preparation:**

1.  Have the product initialised and available with the default configuration and required credentials.

**Verdict:**

1. Pass if the documentation describes with enough details how and where the encryption is used,

2. and handling of PII and privileged information is identified and protected with [5.2.4 State-of-the-art cryptographic libraries].

1. Pass if the documentation describes with enough details how and where the cryptographic features and functions are used,

2. and handling of data is identified and protected with cryptogaphy as defined in Annex K.

3. Fail otherwise.

**Supporting Evidence:**

**Activities:**

1. Study the technical documentation.

2. Identify the structures where administrative, PII or otherwise privileged information is transferred.

2. Identify the structures where data is transferred.

3. Study the deployment guidance.

4. Study the implementation from the product and from the technical documentation.

**Verdict:**

1. Pass, if the flow of privileged information is identifiable from the documentation,

1. Pass, if the data flow and transmission is identifiable from the technical documentation

2. and testing the implementation of interfaces matches the documentation

3. and the interfaces transfering confidential data are protected with encryption.

4. Fail otherwise.

### 6.2.0.4 REQ-TECH-4

**Objective:**  Ensure product that allows and encourages users to change keys when necessary for product security reasons, such as when employees and administrators roles change..<br/>

**Objective:** When employees and administrators roles change, the related keys shall change accordingly.<br/>

**Preparation:** None<br/>

**Activities:**

1. Dump all clock from all participating systems and computational nodes.

2. Ensure that the clock deviation is within limits specified in the techinal documentation.

3. If unsure, verify that the clock synchronisation mechanism is in use and functioning in the default installation.

3. Verify that time synchronisation mechanism is in use and functioning already with the default configuration.

**Verdict:**

1. Study the technical documentation.

2. Study the SBOM.

3. Cross reference to update instructions.

3. Check that the SBOM considers the update of components, its dependencies and that it allows to identify the source.

**Verdict:**

1. Pass if instructions are operatively consistent

1. Pass, if the listed SBOM entries is consistent with the corresponding real NMS components

2. and the update frequency has a possibility to match the expected changes in the deployment complexity.

3. Fail otherwise.

#### 6.3.6.1 REQ-METRICS-1

**Objective:** Prevent compromised managed element to hide its behaviour.<br/>

**Objective:** Stored data wiping or overwriting can always be recognized.<br/>

**Preparation:**

1. Have the product initialised and available with the default configuration.

2. Create required authentication credentials for the test.

3.  Prepare an import data set, that represents natural collected values from the target.

4.  Create a copy of the import data set and modify the values.

3. Prepare an import data set that represents normal operational metric data from a managed device.

4. Create a copy of the import data set and modify the data to change also related integrity protection values.

**Activities:**

1.  Begin timeseries data collection.

1.  Restart the collection of timeseries data with the modified data set.

1. Begin a timeseries data collection.

2. Restart the collection of timeseries data with the modified data set.

**Verdict:** Pass if systems detects the modified data set.<br/>

**Supporting Evidence:**

1. Collect output showing the whether the current metrics data is being handled by the normal flow as expected.

1. Collect output showing how the modified data set was accepted or discarded.

2. Collect output showing how the modified data set was accepted or discarded.

#### 6.3.6.2 REQ-METRICS-2

**Objective:** Understanding what is collected helps user to undrestand what happens, but also is needed for data minimisation validation.<br/>

**Objective:** The user has explanations about the meaning of each metric and is enabled to interpret it. Furthermore, it clarifies the relevance of collected metric data.<br/>

**Preparation:**

1.  Have the product initialised and available with the default configuration and required credentials.

**Verdict:**

1. Pass if there are no unknown metrics displayed or collected.

1. Pass if the metric well-known, like CPU usage, but undocumented.

2. Pass if the metric is recognised and pointer to the documentation is provided (the product's reference documentation).

3. Fail otherwise.

2. Pass if there are commonly well-known metrics displayed or collected, like CPU usage, which can remain without explanation.

3. Pass if a metric is recognised and a pointer to the documentation is provided (managed element manufacturer’s reference documentation e.g.).

4. Fail otherwise.

**Supporting Evidence:**

1. The technical documentation.

1. Screenshot of the GUI displaying how the data is displayed.

1. Collect output showing whether the normal operation metrics data is operated as expected.

2. Collect output showing whether the modified data set was detected and then notified or logged

> NOTE: Whether a modified data set gets accepted or discarded might not be an NMS but an administrator decision.

#### 6.3.6.3 REQ-METRICS-3

**Objective:** Metrics storage consumes alot of storage, and also affects persons privacy as rarely the data is deleted on demand.<br/>

**Objective:** The product provides the information on the meaning of the metrics data and about their memory and storage consumption.<br/>

**Preparation:**

1.  Have the product initialised and available with the default configuration and required credentials.

#### 6.3.6.5 REQ-METRICS-5

**Objective:** Crashes are used to modify the program state. Abnormal crashes can be an indication of upcoming compromise.<br/>

**Objective:** A NMS crash and restart is analyzable and can allow to be forecasted based on the recorded data and the event of a crashed or restarted managed element is detected and recorded by the NMS.<br/>

**Preparation:**

1.  Have the product initialised and available with the default configuration and required credentials.

1.  Have the NMS product initialised and available with the default configuration and required credentials.

2.  Have the managed element initialised and available with the default configuration and required credentials.

**Activities:**

1. Purposefully crash a process within the product.

2. Purposefully crash a managed element hosted process if funcitonality is available in the product.

3. Study the metrics output.

1. Purposefully crash an NMS process.

2. Study the NMS behaviour in terms of process recovery or restarting the NMS.

3. Purposefully crash a process or restart a connected element.

4. Study the metrics output.

**Verdict:**

1. Pass if process or device specific counter is iterated.

1. Fail otherwise.

1. Pass if the NMS process or the NMS recovers to normal operation

2. and if the event can be identifed in the NMS logging

3. and if the NMS detects and displays or reports the event on the managed device.

4. Fail otherwise.

**Supporting Evidence:** Log or and metrics output showing detected system or managed element crash or restart with the reported cause.<br/>

**Activities:**

1. Study the monitoring data GUI.

1. Study the provided documentation.

1. Study the provided documentation to interpret the monitoring data.

2. Intensionally disconnect the NMS from a provided service, for example the timestamp server.

**Verdict:**

1. Pass if expected operation availability is defined per connected element and relevant system processes.

2. Fail otherwise.

1. Pass, if the monitoring data the expected operations and their statuses per connected element and relevant system processes

2. and if the monitoring data and notifies the disconnection from the provided service

3. and if the disconnected service can be identified in the logging records.

4. Fail otherwise.

**Supporting Evidence:**

#### 6.3.6.7 REQ-METRICS-7

**Objective:** Bad service quality can be a indication of compromise.<br/>

**Objective:** Out of the normal metrics indicate failures or compromise of both, the managed elements and the NMS.<br/>

**Preparation:**

1.  Have the product initialised and available with the default configuration and required credentials.

1.  Have the NMS initialised and available with the default configuration and required credentials.

2.  Have the managed element initialised and available with the default configuration and required credentials.

**Activities:**

1. Study the monitoring data GUI.

1. Study the provided documentation.

1. Study the provided product documentation on metrics data.

2. Study the provided product documentation on metrics data that are received from managed elements.

**Verdict:**

1. Pass if Service Level Indicator defines and implements expected operation

1. Pass if the NMS monitoring data shows the expected operation statuses for both, the NMS and for the managed devices:

    - success rate

    - operation failure rate

    - query execution time

    - response size where applicable.

1. Fail otherwise.

2. Fail otherwise.

**Supporting Evidence:**

1. The technical documentation.

1. Screenshot of the GUI displaying how the data is displayed.

1. The technical documentation on metrics data of NMS and managed elements.

2. Sample or a screenshot how the data is displayed for both, the NMS and the managed elements.

#### 6.3.6.8 REQ-METRICS-8

**Activities:** <br/>

1.  Send a correct API request towards each system API endpoint.

1.  Send a correct API request from the product towards each managed element API endpoint.

2.  Send an incorrect API request towards each system API endpoint.

3.  Send a momentary spike of request that exceeds the reasonable and expected load of incoming requests.

#### 6.3.8.1 REQ-HA-1

**Objective:** <br/>

**Objective:** Events by changes of the product itself that impact the product availability do not render the product behaviour into the unpredicatable. The product keeps the availability time definitions.<br/>

**Preparation:**

1.  Have the product initialised and available with the default configuration and required credentials.

**Activities:**

1.  Kill a random process, processing node or simulate a loss of a datacenter.

1.  Repeat first step enough of times with varying scopes to demonstrate conformance.

1. Intensionally terminate randomly an NMS-internal process, a processing node or simulate a loss of a datacenter.

2. Repeat the previous step 1 enough often with varying scopes to demonstrate conformance.

**Verdict:**

1. Pass if loss of a resource matching the availability and distribution description doesn't exceed the availability definition.

1. Pass, if the effect of the loss of a chosen resource or process termination matches the availability and service description, and that the NMS meets the availability time period definitions.

1. Fail otherwise.

**Supporting Evidence:**

1. Structured log output or other documentation that shows made actions and perceived operative response.

**Objective:** System updates and changes are included in the availability definition.<br/>

**Preparation:** None <br/>

**Activities:**

1.  Study the availability definition and the presented metrics.

**Verdict:**

1. Pass if system changes, updates and upgrades are not considered to be outside of the availability definition and tracking.

1. Fail otherwise.

**Supporting Evidence:**

1. Pointers to the documentation.

#### 6.3.8.2 REQ-HA-2

**Objective:** The customer understands how the sytem behaves undre different conditions and can make a disaster recovery plan for the operation.<br/>

**Objective:** The user understands how the sytem behaves under different conditions and can make a disaster recovery plan for the operation.<br/>

**Preparation:**

1.  Have the product initialised and available with the default configuration and required credentials.

**Activities:**

1. Study the availability definition from the documentation.

2.  Delete or make unavailable: service, compute node, rack, or datacenter based on the what the system design tolerates.

2. Disconnect or make unavailable a service, a compute node, a rack, or a datacenter based on the tolerable definitions provided in the technical documentation.

3. Wait for stability.

4.  Return the resrouce taken away in previous step.

4. Reconnect or make available the previously removed resource.

5. Wait for stability.

**Verdict:**

4. and the defined high availability is hold.

5. Fail otherwise.

> NOTE: The selection of the resource to be removed should meet the principal of the most needed or having the highest priority resource for the normal NMS operations.

**Supporting Evidence:**

1. Pointers to the documentation.

2. Description of the test procedure and relevant bits of log showing the phases.

#### 6.3.8.3 REQ-HA-3

**Objective:** The customer understands how the sytem behaves undre different conditions and can make a disaster recovery plan for the operation.<br/>

**Objective:** The technical documentation provides explanations on how the sytem behaves under different conditions, enabling the user to develop a disaster recovery plan for the operation.<br/>

**Preparation:** None <br/>

**Activities:**