@@ -226,100 +226,8 @@ This section can include topic specific requirements.
### 5.3.5 Logging
In many product deployments, privileged users manage monitoring tasks also with the analysis of product logging records.
Especially when there are forensic demands, comprehensive and detailed logging becomes a larger challenge.
The logging requirements in this subclause define baseline event recording and additional protections for retention, integrity, backup, and external forwarding according to the applicable risk tier.
When evaluating the applicability of these requirements, the highest of following risk factors define the category to follow: [CENT]
For low risk:
***[REQ-LOG-0a]:** The log file of events shall be protected from unauthorised access.
***[REQ-LOG-0b]:** The log data of events shall be protected from modification including their deletion.
***[REQ-LOG-0c]:** The log data of events shall be confidentiality protected.
***[REQ-LOG-0d]:** The log shall include event time, actor identity, action type, and affected non-sensitive scope and object identifiers.
***[REQ-LOG-0e]:** The product shall monitor and log all received relevant events from its managed elements, including, not conclusive, incidents, alarms, time and clock shift alarms.
***[REQ-LOG-0f]:** The product shall define what is logged, and how long that record is kept.
The following requirements apply where the corresponding function exists:
***[REQ-LOG-1a]:** The product shall generate auditable events for successful and failed authentication events.
***[REQ-LOG-1b]:** The product shall generate auditable events for session establishment attempts with source details.
***[REQ-LOG-1c]:** The product shall generate auditable events for session termination events with reason.
***[REQ-LOG-1d]:** The product shall generate auditable events for session validation checks like number of concurrent sessions.
***[REQ-LOG-1e]:** The product shall generate auditable events for privilege and role changes.
***[REQ-LOG-1f]:** The product shall generate auditable events for configuration changes.
***[REQ-LOG-1g]:** The product shall generate auditable events for device enrollment and unenrollment.
***[REQ-LOG-1h]:** The product shall generate auditable events for trust-anchor changes.
***[REQ-LOG-1i]:** The product shall generate auditable events for policy changes.
***[REQ-LOG-1j]:** The product shall generate auditable events for credential changes.
***[REQ-LOG-2a]:** The product shall log boot or initialisation events including timestamped boot stage progression.
***[REQ-LOG-2b]:** The product shall log boot or initialisation events including software component verification and initialisation actions.
***[REQ-LOG-2c]:** The product shall log boot or initialisation events including recovery mode activations if in use.
***[REQ-LOG-3c]:** The product shall log events described by [5.3.4 Secure updates].
***[REQ-LOG-3d]:** The product shall log installation successes and failures in the managed devices and the product itself if that information can be extracted from the targets.
For medium risk:
***[REQ-LOG-4]:** The log information shall have an active backup scheduled.
***[REQ-LOG-5]:** Administrative log records, traces and events shall be forwarded to or stored in a service that prevents unauthorized modification or deletion of recorded entries.
> Clarification: write only service can be append only or even idempotent upsert system that does not let the received message to be altered later.
> The aim is to prevent a possible attacker to clear its traces by deleting the actions done in the system by distorting the history.
For high risk:
***[REQ-LOG-6]:** The product shall support forwarding of relevant administrative events to an external logging or SIEM system.
***[REQ-LOG-7]:** SIEM transfer format, field attributes and event descriptions shall made available in a machine readable format.
***[REQ-LOG-8]:** Exported artifacts shall preserve essential fields at least, but not limited to: time, actor, action type, affected scope, result.
***[REQ-LOG-9]:** The product shall record provenance sufficient to attribute the change to an actor and context information related to at least, but not limited to: user, automated workflow, policy or rule identifier, and triggering event reference.
### 5.3.6 Metrics
The metrics requirements in this subclause support security monitoring, operational visibility, fault detection, and verification of system behaviour.
Fulfilment of these metrics is essential for all products in all use cases and all risk levels.
Breaches can not be detected, if an attacker can hide it's existense.
These requirements are generally binding, and there is no low-medium-high tiering available.
General requirements:
***[REQ-METRICS-0]** The product shall be designed in a way that collected and stored metrics data can not be altered.
***[REQ-METRICS-1]** Historical metrics data import overwriting an existing data point shall be noticed.
***[REQ-METRICS-2]** Metrics name, purpose, and value interpretation shall be described for the user.
***[REQ-METRICS-3]** Metrics cadence, accuracy and storage time shall be described for the user.
***[REQ-METRICS-9]** The product shall not collect data unrelated to the purpose of the product operation.
> NOTE: [REQ-METRICS-1], [REQ-METRICS-2] and [REQ-METRICS-3] requirements apply to all collected metrics.
Availability and uptime requirements:
***[REQ-METRICS-4]** Relevant system and connected element metrics like CPU, memory, disk utilisation shall be tracked and reported.
***[REQ-METRICS-5a]** System incidents, such as process and service crashes and restarts, shall be tracked and reported.
***[REQ-METRICS-5b]** Managed element incidents, such as process and service crashes and restarts shall be tracked and reported.
***[REQ-METRICS-6a]** Managed elements availabilities and statuses shall be tracked and reported.
***[REQ-METRICS-6b]** System and provided service availabilities and statuses shall be tracked and reported.
***[REQ-METRICS-7a]** Relevant databases in the product and storage health metrics like queries per second, latency and throughput shall be tracked and reported.
***[REQ-METRICS-7b]** Relevant managed element database and storage health metrics like queries per second, latency and throughput shall be tracked and reported.
***[REQ-METRICS-8]** Relevant networking metrics like throughput and protocol errors shall be tracked and reported.
The **[REQ-METRICS-5]** is vague by design.
The product can host an ecosystem that exposes managed elements capacity for 3rd. party applications.
The product then operates as a runtime for the workload, and needs to inherit operating system responsibilies.
The **[REQ-METRICS-7\*]** is defined expecting that the product hosts these services.
When RDPS is used for these functions, the relevant metrics are different and "like queris per second" equivivalent can be used.
The **[REQ-METRICS-8]** and **[REQ-METRICS-9]** below can refer either the product or the connected devices.
Relevancy of the metric depends on the protocols and the product usage scenario.
Application metrics requirements:
***[REQ-METRICS-9]** Relevant application metrics, like GUI and API latencies and error rates, shall be tracked and reported.
Matching tests for these requirements are listed in [6.3.6 Metrics tests].
### 5.3.7 Data minimisation
### 5.3.8 High Availability
@@ -487,282 +395,6 @@ Matching tests for these requirements are listed in [6.3.6 Metrics tests].
### 6.3.5 Logging tests
### 6.3.5.8 REQ-LOG-8
**Objective:** Reduces vendor lock-in and supports incident response and CRA evidence portability.
**Preparation:** None<br/>
**Activities:** None<br/>
**Verdict:**
1. Pass if export is available, documented, and preserves essential fields.
2. Fail if evidence cannot be exported or loses critical context.
**Supporting Evidence:**
1. Pointers to the documentation.
### 6.3.5.9 REQ-LOG-9
**Objective:** Enables audit replay and accountability for automated control planes. Reduces ambiguity in incident investigations.
**Preparation:** None<br/>
**Activities:** None<br/>
**Verdict:**
1. Pass if each configuration change can be attributed to actor and context with retrievable references.
2. Fail if changes cannot be deterministically attributed.
**Supporting Evidence:**
1. Pointers to the documentation.
### 6.3.6 Metrics tests
#### 6.3.6.0 REQ-METRICS-0
**Objective:** Verifies the ingestion pipline design upholds integrity and confidentiality requirements.<br/>
**Preparation:** None<br/>
**Activities:**
1. Study the technical documentation.
**Verdict:**
1. Pass if no unauthorised process before ingestion of collected metrics data can alter the metrics before storage,
2. and the storage can not be altered outside of reqular cleaning cycles.
3. Fail otherwise.
**Supporting Evidence:**
1. References to to documentation sections.
#### 6.3.6.1 REQ-METRICS-1
**Objective:** Stored data wiping or overwriting can always be recognized.<br/>
**Preparation:**
1. Have the product initialised and available with the default configuration.
2. Create required authentication credentials for the test.
3. Prepare an import data set that represents normal operational metric data from a managed device.
4. Create a copy of the import data set and modify the data to change also related integrity protection values.
**Activities:**
1. Begin a timeseries data collection.
2. Restart the collection of timeseries data with the modified data set.
**Verdict:** Pass if systems detects the modified data set.<br/>
**Supporting Evidence:**
1. Collect output showing the whether the current metrics data is being handled by the normal flow as expected.
2. Collect output showing how the modified data set was accepted or discarded.
#### 6.3.6.2 REQ-METRICS-2
**Objective:** The user has explanations about the meaning of each metric and is enabled to interpret it. Furthermore, it clarifies the relevance of collected metric data.<br/>
**Preparation:**
1. Have the product initialised and available with the default configuration and required credentials.
**Activities:**
1. Study the monitoring data GUI.
1. Study the provided documentation.
1. Investigate the metrics storage.
**Verdict:**
1. Pass if there are no unknown metrics displayed or collected.
2. Pass if there are commonly well-known metrics displayed or collected, like CPU usage, which can remain without explanation.
3. Pass if a metric is recognised and a pointer to the documentation is provided (managed element manufacturer’s reference documentation e.g.).
4. Fail otherwise.
**Supporting Evidence:**
1. Collect output showing whether the normal operation metrics data is operated as expected.
2. Collect output showing whether the modified data set was detected and then notified or logged
> NOTE: Whether a modified data set gets accepted or discarded might not be an NMS but an administrator decision.
#### 6.3.6.3 REQ-METRICS-3
**Objective:** The product provides the information on the meaning of the metrics data and about their memory and storage consumption.<br/>
**Preparation:**
1. Have the product initialised and available with the default configuration and required credentials.
**Activities:**
1. Study the monitoring data GUI.
1. Study the provided documentation.
1. Investigate the metrics storage.
**Verdict:**
1. Pass if the metrics collection cadence, accuracy and storage time matches the described use.
1. Fail otherwise.
**Supporting Evidence:**
1. The technical documentation.
1. Metrics storage plan.
#### 6.3.6.4 REQ-METRICS-4
**Objective:** Support users or administrators ability to detect compromised, misconfigured, or harmful connected elements through unusual, excessive, or risky patterns of use.<br/>
**Preparation:**
1. Have the product initialised and available with the default configuration and required credentials.
2. Have at lest one managed element as part of the system the product operates
**Activities:**
1. Study the provided technical documentation to interpret the monitoring data GUI.
1. Simulate unnormal behaviour by intensionally cutting the connection with a managed element and observe the monitoring data.
1. Restart the managed element.
**Verdict:**
1. Pass if basic administrative metrics are tracked and displayed.
1. Pass if the connection loss to the managed element is recognized.
1. Pass if by observing the metrics a baseline can be established.
1. Pass if anomalities like load spikes after a restart can be observed.
1. Fail otherwise.
**Supporting Evidence:**
1. The technical documentation of the monitoring data.
1. Screenshot of the GUI displaying how the data is displayed the normal operation and the detection of the unnormal behaviour.
#### 6.3.6.5 REQ-METRICS-5
**Objective:** A NMS crash and restart is analyzable and can allow to be forecasted based on the recorded data and the event of a crashed or restarted managed element is detected and recorded by the NMS.<br/>
**Preparation:**
1. Have the NMS product initialised and available with the default configuration and required credentials.
2. Have the managed element initialised and available with the default configuration and required credentials.
**Activities:**
1. Purposefully crash an NMS process.
2. Study the NMS behaviour in terms of process recovery or restarting the NMS.
3. Purposefully crash a process or restart a connected element.
4. Study the metrics output.
**Verdict:**
1. Pass if the NMS process or the NMS recovers to normal operation
2. and if the event can be identifed in the NMS logging
3. and if the NMS detects and displays or reports the event on the managed device.
4. Fail otherwise.
**Supporting Evidence:** Log or and metrics output showing detected system or managed element crash or restart with the reported cause.<br/>
#### 6.3.6.6 REQ-METRICS-6
**Objective:** Bad availability can be a indication of compromise.<br/>
**Preparation:**
1. Have the product initialised and available with the default configuration and required credentials.
2. Have the managed element initialised and available with the default configuration and required credentials.
**Activities:**
1. Study the provided documentation to interpret the monitoring data.
2. Intensionally disconnect the NMS from a provided service, for example the timestamp server.
**Verdict:**
1. Pass, if the monitoring data the expected operations and their statuses per connected element and relevant system processes
2. and if the monitoring data and notifies the disconnection from the provided service
3. and if the disconnected service can be identified in the logging records.
4. Fail otherwise.
**Supporting Evidence:**
1. The technical documentation.
1. Screenshot of the GUI displaying how the data is displayed.
#### 6.3.6.7 REQ-METRICS-7
**Objective:** Out of the normal metrics indicate failures or compromise of both, the managed elements and the NMS.<br/>
**Preparation:**
1. Have the NMS initialised and available with the default configuration and required credentials.
2. Have the managed element initialised and available with the default configuration and required credentials.
**Activities:**
1. Study the provided product documentation on metrics data.
2. Study the provided product documentation on metrics data that are received from managed elements.
**Verdict:**
1. Pass if the NMS monitoring data shows the expected operation statuses for both, the NMS and for the managed devices:
- success rate
- operation failure rate
- query execution time
- response size where applicable.
2. Fail otherwise.
**Supporting Evidence:**
1. The technical documentation on metrics data of NMS and managed elements.
2. Sample or a screenshot how the data is displayed for both, the NMS and the managed elements.
#### 6.3.6.8 REQ-METRICS-8
**Objective:** The error logging records enable for tracing back and serve as entry point for the analysis and detection of abnormal behavior in the managed element and in the system.<br/>
**Preparation:**
1. Have the product initialised and available with the default configuration and required credentials.
2. Have the managed element initialised and available with the default configuration and required credentials.
**Activities:**
1. Study the provided documentation to interpret the monitoring data GUI with respect to errors.
1. Study the provided documentation.
1. Generate large amount of traffic in random location.
**Verdict:**
1. Pass if corresponding metrics matches the generated traffic.
1. Fail otherwise.
**Supporting Evidence:**
1. The technical documentation.
1. Screenshot of the GUI displaying how the data is displayed.
#### 6.3.6.9 REQ-METRICS-9
**Objective:** Response latencies on the product queries or protocol errors on incoming data from managed elements enable to detect defective or compromised managed elements.<br/>
**Preparation:**
1. Have the product initialised and available with the default configuration and required credentials.
2. Have the managed element initialised and available with the default configuration and required credentials.
3. Study the deployment instrcutions about sizing of the operative environment.
**Activities:**<br/>
1. Send a correct API request from the product towards each managed element API endpoint.
2. Send an incorrect API request towards each system API endpoint.
3. Send a momentary spike of request that exceeds the reasonable and expected load of incoming requests.
**Verdict:**<br/>
1. Pass if all API endpoints responds as expected
2. and the response latency is recorded in defined accuracy
3. and tracks the number of successfull requests
4. and the incorrect API requests are identified and logged
5. and the spike of high volume requests is handled gracefully.
6. Fail otherwise.
**Supporting Evidence:**
1. Relevant metrics described in the technical documentation.
### 6.3.8 High availability tests
# Annex A (informative): Mapping with essential requirements of the CRA
