@@ -755,34 +770,276 @@ Manfacturer shall implement monitoring system features as listed in the table be
| [REQ-LOG-6a] | SIEM event for anomalities in clock accuracy is included in the technical documentation. |
| [REQ-LOG-6b] | SIEM event is emitted when clock anomalies occur. |
#### 6.3.x Monitoring tests
### 6.3.x Monitoring tests
**[REQ-MON-0]** Collected and stored metrics data can not be altered.
#### 6.3.x.0 REQ-MON-0
**Test**: Review the documentation of all components between the target and the collected and stored metrics data looking for any step that may allow alteration of the metrics data after it has left the target.
**Objective**: Collected and stored metrics data can not be altered.<br/>
**Preparation**:
**Activities**: Review the documentation of all components between the target and the collected and stored metrics data looking for any step that may allow alteration of the metrics data after it has left the target.<br/>
**Verdict**: Pass if no process step allows the alteration before ingestion of collected metrics data after it has left the target.<br/>
**Supporting Evidence**: The technical documentation.<br/>
**Result**: No process step allows the alteration of collected metrics data after it has left the target.
#### 6.3.x.1 REQ-MON-1
**[REQ-MON-1]** Historical metrics data re-imports and rewrites are noticed.
**Objective**: Historical metrics data import overwriting an existing data point is noticed.<br/>
**Preparation**:
**Test**: Set up the product to collect timeseries data in the normal manner from a client and begin collection. Collect output showing the whether the current metrics data is being handled by the normal flow. Re-import or resend the same set of data, but with modified values.
1. Have the NMS product initialized and available with the default configuration.
1. Create required authentication credentials for the test.
1. Prepare an import data set, that represents natural collected values from the target.
1. Create a copy of the import data set and modify the values.
**Result**: System output shows that the re-import of the timeseries was overwriting an existing set of values and if the new value was accepted or discarded.
**Activities**:
<mark>To be converted to above format:</mark>
1. Begin timeseries data collection.
1. Restart the collection of timeseries data with the modified data set.
| [REQ-MON-2] | National MSAs are able to validate the system design comformity without a deployment. |
| [REQ-MON-3] | National MSAs are able to validate the system design comformity without a deployment. |
| [REQ-MON-4] | <mark>How to define business continuity metrics? A new chapter?</mark> |
| [REQ-MON-5] | Relevant metrics are documented and presented to the user in the GUI. |
| [REQ-MON-6] | Relevant metrics are documented and presented to the user in the GUI. |
| [REQ-MON-7] | Relevant metrics are documented and presented to the user in the GUI. |
| [REQ-MON-8] | Relevant metrics are documented and presented to the user in the GUI. |
| [REQ-MON-9] | Relevant metrics are documented and presented to the user in the GUI. |
| [REQ-MON-10] | Relevant metrics are documented and presented to the user in the GUI. |
| [REQ-MON-11] | Relevant metrics are documented and presented to the user in the GUI. |
**Verdict**: Pass if systems detects the modified data set.<br/>
**Supporting Evidence**:
1. Collect output showing the whether the current metrics data is being handled by the normal flow as expected.
1. Collect output showing how the modfied data set was accepted or discarded.
#### 6.3.x.2 REQ-MON-2
**Objective**: Metric name, purpose, and value interpretation are described for the user.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
**Activities**:
1. Study the monitoring data GUI.
1. Study the provided documentation.
1. Investigate the metrics storage.
**Verdict**:
1. Pass if there are no uknown metrics displayed or collected.
1. Pass if the metric well-known, like CPU usage, but undocumented.
1. Pass if the metric is recognized and pointer to the documentation is provided (managed element manufacturer's reference documentation e.g.).
1. Fail otherwise.
**Supporting Evidence**:
1. The technical documentation.
1. Screenshot of the GUI displayging how the data is displayed.
#### 6.3.x.3 REQ-MON-3
**Objective**: Metrics cadence, accuracy and storage time are described for the user.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
**Activities**:
1. Study the monitoring data GUI.
1. Study the provided documentation.
1. Investigate the metrics storage.
**Verdict**:
1. Pass if the metrics collection cadence, accuracy and storage time matches the described use.
1. Fail otherwise.
**Supporting Evidence**:
1. The technical documentation.
1. Metrics storage plan.
#### 6.3.x.4 REQ-MON-4
**Objective**: System does not collect metrics that are not used in operative purposes.<br/>
**Preparation**:
1. Evaluate **[REQ-MON-2](#63x2-req-mon-2)** test
1. Evaluate **[REQ-MON-3](#63x3-req-mon-3)** test
**Activities**:
1. Study the monitoring data GUI.
1. Study the provided documentation.
1. Study the referred tests.
**Verdict**:
1. Pass if the metrics collection cadence, accuracy and storage time is justified by comformity to requlation or business related criteria.
**Supporting Evidence**:
1. Metrics storage plan.
1. Metrics comformity assesment.
1. Product position in relation to GDPR.
#### 6.3.x.5 REQ-MON-5
**Objective**: Relevant system and connected element metrics like CPU, memory, disk utilisation are tracked and reported.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
**Activities**:
1. Study the monitoring data GUI.
1. Study the provided documentation.
**Verdict**:
1. Pass if basic administrative metrics are tracked and displayed.
1. Fail otherwise.
**Supporting Evidence**:
1. The technical documentation.
1. Screenshot of the GUI displayging how the data is displayed.
#### 6.3.x.6 REQ-MON-6
**Reference**: **[REQ-MON-6a]** and **[REQ-MON-6b]**<br/>
**Objective a**: System process and service crashes and restarts are tracked and reported.<br/>
**Objective b**: Managed element process and service crashes and restarts are tracked and reported.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
1. Have the managed element initialized and available with the default configuration and required credentials.
**Activities**:
1. Purposfully crash a process or restart a connected element.
1. Study the metrics output.
**Verdict**:
1. Pass if process or device specific counter is iterated.
1. Fail otherwise.
**Supporting Evidence**: Log or and metrics output showing detected system or managed element crash or restart with the reported cause.<br/>
#### 6.3.x.7 REQ-MON-7
**Objective**: Managed elements and system nodes and provided services availabilities and statuses are tracked and reported.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
1. Have the managed element initialized and available with the default configuration and required credentials.
**Activities**:
1. Study the monitoring data GUI.
1. Study the provided documentation.
**Verdict**:
1. Pass if Service Level Indicator defines and implements expected operation availability per connected element and relevant system processes.
1. Fail otherwise.
**Supporting Evidence**:
1. The technical documentation.
1. Screenshot of the GUI displayging how the data is displayed.
#### 6.3.x.8 REQ-MON-8
**Reference**: **[REQ-MON-8a]** and **[REQ-MON-8b]**<br/>
**Objective**: Relevant system database and storage health metrics like queries per second, latency and throughput are tracked and reported.<br/>
**Objective**: Relevant managed element database and storage health metrics like queries per second, latency and throughput are tracked and reported.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
1. Have the managed element initialized and available with the default configuration and required credentials.
**Activities**:
1. Study the monitoring data GUI.
1. Study the provided documentation.
**Verdict**:
1. Pass if Service Level Indicator defines and implements expected operation
- success rate
- operation failure rate
- query execution time
- response size where applicable.
1. Fail otherwise.
**Supporting Evidence**:
1. The technical documentation.
1. Screenshot of the GUI displayging how the data is displayed.
#### 6.3.x.9 REQ-MON-9
**Objective**: Relevant networking metrics like throughput and protocol errros are tracked and reported.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
1. Have the managed element initialized and available with the default configuration and required credentials.
**Activities**:
1. Study the monitoring data GUI.
1. Study the provided documentation.
1. Generate large ammount of traffic in random location.
**Verdict**:
1. Pass if corresponding metrics matches the generated traffic.
1. Fail otherwise.
**Supporting Evidence**:
1. The technical documentation.
1. Screenshot of the GUI displayging how the data is displayed.
#### 6.3.x.10 REQ-MON-10
**Objective**: GUI and API latencies are tracked and reported.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
1. Have the managed element initialized and available with the default configuration and required credentials.
**Activities**: <br/>
1. Send a correct API request towards each system API endpoint.
**Verdict**: <br/>
1. Pass if all API endpoints responds as expected
1. and the response latency is recorded in defined accuracy
1. and tracks the number of successfull requests.
1. Fail otherwise.
**Supporting Evidence**:
1. Relevant metrics described in the technical documentation.
#### 6.3.x.11 REQ-MON-11
**Objective**: GUI and API error rates are tracked and reported.<br/>
**Preparation**:
1. Have the NMS product initialized and available with the default configuration and required credentials.
1. Have the managed element initialized and available with the default configuration and required credentials.
**Activities**:
1. Send a mallformed API request towards each system API endpoint.
**Verdict**:
1. Pass if all API endpoints notices the mallformed request.
1. and reports relevant data into the logs.
1. Fail otherwise.
**Supporting Evidence**:
1. System log output.
1. Relevant metrics described in the technical documentation.
# Annex A (informative): Mapping with essential requirements of the CRA
@@ -819,20 +1076,29 @@ Manfacturer shall implement monitoring system features as listed in the table be
# Annex C (informative): Risk acceptance criteria and risk management methodology
> Ref. (PT1 6.3)
> Table mapping the identified risks to requirements
## C.1 Risk acceptance and risk management methodology
> Describe how to decide if residual risks are tolerable.
## C.2 Risk Assessment
## C 2.1 ID Asset
> For each threat identified above, use likelihood and magnitude of the threat to assess its risk in the context of use cases. The results should be consistent with the mapping of use cases to security profiles.
## C 2.2 ID Threat
> Guidance from latest PT1 draft:
>
> An analysis in terms of likelihood and magnitude of a product’s threats is required to be able to determine the product’s risks.
## C 2.3 Estimate Risks (Risk factors)
> NOTE 1 This document does not require a specific methodology for a cybersecurity risk analysis as long as the cybersecurity risk estimation is based on the likelihood of occurrence and magnitude of loss or disruption of cybersecurity risks. Thus, different approaches and models such as the fishbone model, event tree analysis or fault tree models can be used within the analysis of cybersecurity risks.
## C 2.4 Evaluate Risks
> NOTE 2 A qualitative estimation of the cybersecurity risks can be performed using risk matrices that map qualitative categories of the likelihood of occurrence and qualitative categories of magnitude of loss or disruption to cybersecurity risk categories.
## C.1 Assets
> NOTE 3 A quantitative estimation of the cybersecurity risks can be performed using scoring systems that map qualitative categories of the likelihood of occurrence and qualitative categories of magnitude of loss or disruption to certain values.
<mark>FIXME asses the risks</mark>
## C 2.1 Assets
NMS protects systems that are relying on network connectivity to perform its daily operations.
@@ -864,7 +1130,7 @@ NMS protects systems that are relying on network connectivity to perform its dai
- CORBA access, grcp
- keys can be generated or imported through the keymanagement modules
### C.1.1 Data
### C 2.1.1 Data assets
The stored data depends on what functions the NMS has available and what the intend use is.
The stored data can be, but is not limited to:
@@ -883,11 +1149,7 @@ The stored data can be, but is not limited to:
The manufacturer shall follow the CRAs pricibles of implementing high level of cybersecurity of products [CRA Resictal 11] <ahref="#_ref_i.1">[i.1]</a>. The protection duty extends to third party integrations regardless of the market status of the component. [CRA Resictal 34] <ahref="#_ref_i.1">[i.1]</a>.
### C.1.2 Product functions
See the functions in [Section 4.7 Essential functions](#47-essential-functions).
## C.2 Threats
## C 2.2 Threats
> Based on the assets, what are the threats during:
>
@@ -920,7 +1182,7 @@ See the functions in [Section 4.7 Essential functions](#47-essential-functions).
> If any risks are not treated by the normative requirements, describe non-normative suggestions to mitigate them.
### C 2.4.2 Residual risk
> Describe how to treat any residual risks, for example by documenting them or informing the user.
## C 3 Assumptions
> List assumptions that are relevant to the risk analysis for these threats. Everything is hackable if you try hard enough, but what risks can this product mitigate, and what must it delegate to other components or the operational environment? Some potential examples:
>
@@ -960,22 +1234,6 @@ See the functions in [Section 4.7 Essential functions](#47-essential-functions).
<mark>FIXME list more assumptions</mark>
## C.4 Risk assessments of threats
> For each threat identified above, use likelihood and magnitude of the threat to assess its risk in the context of use cases. The results should be consistent with the mapping of use cases to security profiles.
> Guidance from latest PT1 draft:
>
> An analysis in terms of likelihood and magnitude of a product’s threats is required to be able to determine the product’s risks.
> NOTE 1 This document does not require a specific methodology for a cybersecurity risk analysis as long as the cybersecurity risk estimation is based on the likelihood of occurrence and magnitude of loss or disruption of cybersecurity risks. Thus, different approaches and models such as the fishbone model, event tree analysis or fault tree models can be used within the analysis of cybersecurity risks.
> NOTE 2 A qualitative estimation of the cybersecurity risks can be performed using risk matrices that map qualitative categories of the likelihood of occurrence and qualitative categories of magnitude of loss or disruption to cybersecurity risk categories.
> NOTE 3 A quantitative estimation of the cybersecurity risks can be performed using scoring systems that map qualitative categories of the likelihood of occurrence and qualitative categories of magnitude of loss or disruption to certain values.
<mark>FIXME asses the risks</mark>
# Annex D (informative): Risk evaluation guidance
For each network management system placed on the market, the manufacturer shall develop a threat model and risk profile of the foreseeable use of the system, and shall consider the interplay between:
@@ -1007,22 +1265,6 @@ Refer to normative standards:
- Device driver attack vectors
- Physical interface specific attack vectors?
## D.1 Mapping of risks to requirements
> Table mapping the identified risks to requirements
## D.2 Risks not treated by the requirements
> If any risks are not treated by the normative requirements, describe non-normative suggestions to mitigate them.
## D.3 Risk acceptance criteria
> Describe how to decide if residual risks are tolerable.
## D.4 Residual risks
> Describe how to treat any residual risks, for example by documenting them or informing the user.
# Annex L (informative): Relationship between the present document and the requirements of EU Regulation 2024/2847
