Merge in XRCA/teraflow from vhallivu/xr_pytest_with_http_mock to xr_development

Squashed commit of the following:

commit d0688826c369c9320027cc6b3eef34d82674f1ec
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 30 07:29:30 2022 +0300

    XR Teraflow driver logging cleanup

commit 89922d37c7709b4fe5acb3b122f5d6aea023c956
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 30 07:04:15 2022 +0300

    Improved HTTP request logging

commit c50420a5358686dbafa1cf61acf35ac753a2dc5f
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 29 14:33:28 2022 +0300

    POST refactoring

commit a133ed135816fd252bdc026e25598e8c2e65e8ec
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 29 13:24:15 2022 +0300

    More refactoring

commit 202c9827f7373dabc98ac2e72464fca104d6ba64
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 29 08:16:07 2022 +0300

    Start of HTTP refactoring

commit 8595e63c2cc6fef51476315134c12f6f4e700a46
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 24 08:06:58 2022 +0300

    Add missing mutexes to XrDriver

commit d250437104322de12232320aa55e371f9777875a
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 23 17:54:48 2022 +0300

    Connection update unit test

commit 5240a96941b43e2cc2488443bb927475fc96ac81
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 22 14:34:56 2022 +0300

    Improved HTTP get error handling

commit 10fa5132aa47db2a69345c5f69a60dd9e2c25c74
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 22 11:51:12 2022 +0300

    Higher level unit test for XR

commit 242f44363b8bdb63e02ec92c034aa3208abd7219
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Sat Aug 20 16:59:18 2022 +0300

    Improved test data for constellation pytests

commit 3f834ba6f7e9448cab77869789ed0ded6d838e70
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Sat Aug 20 16:32:48 2022 +0300

    Initial prototyping of mock based testing for xr

Merge in XRCA/teraflow from vhallivu/vti_mode_prototyping to xr_development

Squashed commit of the following:

commit 6127856df3f3e4c9131f40972d799a6de32a38ef
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Thu Aug 18 09:15:59 2022 +0300

    Fix lint errors

commit dbea279626606f62553b1d6e1a80ae12748cdfd5
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Thu Aug 18 08:48:57 2022 +0300

    pytests improvements

commit e7218f6369df648c73782d7b41b062bc50670490
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 17 17:54:04 2022 +0300

    Some tests for TC

commit 8327392bfa22d06d334349fcf872579414141277
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 17 15:59:01 2022 +0300

    Unit test for connection

commit 5148a84534b332ee82458b080fbc89d19bec3c63
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 17 13:26:23 2022 +0300

    Incremental connection endpoint modifications support

commit 10f3e1e43de483c4dc84a6dcd7b0f8bba37d97a2
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 17 11:32:05 2022 +0300

    Refactored TF to use the new backend for create

commit 1e678b0222b9ef7dbdaf09b83696f561c6889c08
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 17 10:33:34 2022 +0300

    More code refactoring

commit 991d951485fb7fa02692c08da7fe6261cc7f4ce7
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 16 18:25:34 2022 +0300

    More refactoring

commit 62f0f407ca78cadd5c6c581d98e47c8688909318
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 16 17:33:03 2022 +0300

    Transport capacity testing and cm-cli support

commit a8d0bd8797d6754e7adbfacf292831eaf69c83f6
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 16 16:35:14 2022 +0300

    TC delete, more refactoring

commit ec70cefccdf4dd94c70a9bc31d4f9c841177db9c
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 16 14:48:23 2022 +0300

    Support for creating transport capacities

commit 3148f2c472452adcef78f1a80540091b6fcf505d
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 16 11:19:53 2022 +0300

    Code refactoring

commit 6e980925e3605ade0e5b6e6d0447c29fc3badce5
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 15 08:42:43 2022 +0300

    Refactoring + tuning gitignore + updating version controlled generated files

Merge in XRCA/teraflow from vhallivu/update_cm_authentication to xr_development

Squashed commit of the following:

commit 6a7069ded9b899afe4c9a218f1062da5c69782e4
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Fri Aug 12 07:51:32 2022 +0300

    review + lint fixes

commit f02d6f4ce42f909067a2570daff136e1ed0c7bbd
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Thu Aug 11 14:39:12 2022 +0300

    XR connection delete support

commit 81c3aae5ba105f6650f5392f438200c4b505eea1
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Thu Aug 11 12:39:29 2022 +0300

    Create services to CM

commit 56b3fb3acfc3281b3b3136269163697ca679913c
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Thu Aug 11 11:17:28 2022 +0300

    Improved connection management code, code refactoring

commit 46ef072679ffd37c22c4bf959bb8ba009cf4a082
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 10 17:30:27 2022 +0300

    More support for connection management

commit 49a2762abd7c1c8adafd666624b31976281d3860
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 10 15:52:17 2022 +0300

    Refactored CM connection, now using CM if database

commit 1b404f2c10befeecb48a7a2278f5e9ec60c25d9a
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Wed Aug 10 14:14:03 2022 +0300

    Start of refactoring of CM connectivity

commit 8ec8251bafbba559a4881e50be0aa068340fe536
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Tue Aug 9 16:37:49 2022 +0300

    Improved CM authentication

Merge in XRCA/teraflow from vhallivu/XR_DRIVER_EMULATE_ENDPOINTS to xr_development

Squashed commit of the following:

commit da5b46937d8f0dd87f535e3d736dc1d9e2043e94
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 8 15:21:16 2022 +0300

    Updates to emulation

commit b9d9e566f4622f85767c85b72494fbb33896de60
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 8 13:49:55 2022 +0300

    Add XR constellation icon for topology

commit 6d24a643f11ecb684043332500c2ee8b2768e900
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 8 12:56:16 2022 +0300

    Improved XR service creation stubs

commit 7d31fd2ecc67848368e2d0eddb616faa6115128d
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Mon Aug 8 08:07:10 2022 +0300

    Eulation for XrDriver endpoints

Merge in XRCA/teraflow from vhallivu/XR_OPTICS_DRIVER_LOGIN_TO_CM to xr_development

Squashed commit of the following:

commit ba53d51d94c52491c4f70587187bf6746a9613da
Author: Ville Hallivuori <VHallivuori@infinera.com>
Date:   Fri Aug 5 13:26:40 2022 +0300

    Initial code for XR Device Driver, can connect to CM and acquire access token. No other functionality

- fixed parsing of custom rules in unitary tests

- Added missing log dump in unitary test

Adapted L3_AttackMitigator/L3_CentralizedAttackDetector/L3_DistributedAttackDetector to common proto folder
- Moved imports from *.proto folder to common.proto folder
- Removed genproto.sh scripts
- Arranged requirements to accelerate Docker image creation
- Minor code and linting fixes
- Updated Dockerfile
- Arranged GitLab CI/CD pipeline

Common:
- Corrected Python package versions to prevent collisions with p4runtime

Context:
- Minor corrections in GitLab CI/CD pipeline

Device:
- Corrected Python package versions to prevent collisions with p4runtime

Monitoring:
- Arranged requirements to accelerate Docker image creation
- Corrected Python package versions to prevent collisions with p4runtime
- Updated Dockerfile
- Arranged GitLab CI/CD pipeline

Device:
- Removed obsoleted run_tests_locally-device.sh script
- Moved imports from *.proto folder to common.proto folder
- Removed device's genproto.sh script
- Minor code and linting fixing
- Arranged requirements to accelerate Docker image creation
- Updated Dockerfile
- Arranged GitLab CI/CD pipeline
- Corrected usage of ConfigRules to use new custom ConfigRules

Monitoring:
- Moved imports from *.proto folder to common.proto folder
- Removed monitoring's genproto.sh script

- Corrected unitary test to execute pipeline

- Migrated to use new generic gRPC servicer
- Migrated to use new generic Rest servicer
- Migrated to use new settings framework
- Migrated tests to use new generic servicers and mock's
- Solved bug with endpoint monitors' population (dupplicated sample types due to wrong key generation)
- Use Emulated driver when both driver and device type are not specified
- Separated unitary tests and execution scripts per device driver
- Extracted common unitary test functionalities into separate code files
- Minor code styling/formatting

- code styling and cleanup

- Corrected default values in unitary test config template
- Minor code styling

- solved minor bug with default openconfig operations.

- OpenConfigDriver: code cleanup and added default values

- Updated default OpenConfig unitary test template config
- Added minor code improvements in OpenConfig Driver

- OpenConfigDriver migrate from netconf_client to ncclient library
- Clean integration of Candidate datastore
- OpenConfigDriver code cleanup

- Implemented support for ACLs
- Added suport for Candidate datastore

new development of microwave device - implemented the Api to get configurations, create and delete services

Signed-off-by: Georgios Katsikas <katsikas.gp@gmail.com>

OpenConfig Driver:
- updated delete operation for interfaces to just remove its description

- Added Routing Policies as special resource for interrogation during initialization
- Added flags in unitary tests to control which tests to be executed in OpenConfig.
- Added run-test-locally specific for device component.

OpenConfig Driver:
- Implemented routing policy templates, parsing and configuration.
- Implemented retry decorator for NetConf client.

- updated OpenConfig templetes
- improved monitoring to report delta values per time period instead of absolute values
- improved unitary tests to select which OpenConfig operation should be tested

Device OpenConfig Driver:
- add missing template
- implement filter to avoid sending empty edit-config requests

Service L3NM-Emulated ServiceHandler:
- implement missing rule to configure routing tables in packet devices

Service L3NM-OpenConfig ServiceHandler:
- implement missing rule to configure routing tables in packet devices

Monitoring:
Reduced default sampling rate to adapt to Infinera devices