Skip to content
Snippets Groups Projects
  1. Jan 13, 2025
    • Waleed Akbar's avatar
      Changes in Telemetry backend Analytics · 8c1d7155
      Waleed Akbar authored
      - Added AnalyzerHelper class
      - Added AnalyzerHandler class
      - Added updated streamer class
      - Update test scripts for analytics backend
      - Updated run script format
      8c1d7155
  2. Nov 04, 2024
  3. Oct 07, 2024
  4. Sep 26, 2024
  5. Sep 25, 2024
  6. Sep 24, 2024
    • Waleed Akbar's avatar
      Updates to Streamer Classes in Backend Analytics: · dafc8e87
      Waleed Akbar authored
      - Replaced `SparkStreamer` class with `DaskStreamer` class.
      - Updated methods to call the `DaskStreamer` class.
      - Improved messages and test files.
      - Added new package requirements.
      - Made minor changes to the frontend `requirements` file.
      dafc8e87
  7. Sep 22, 2024
    • Waleed Akbar's avatar
      Minor Changes in New Monitoring Services to Improve Overall Functionality · c0c517f0
      Waleed Akbar authored
      - Added new tests in the `scripts` folder.
      - Improved messages and test files across all services.
      Telemetry and Analytics Backend:
      - Added a `try` statement to handle Kafka message decoding errors.
      Telemetry Frontend and KPI Value API:
      - Downgraded `apscheduler` from `3.10.4` to `3.10.1` in the requirements file.
      Kafka:
      - Added a method to retrieve `KAFKA_SERVER_ADDRESS` from environment variables.
      Telemetry Backend:
      - Added a condition to allow the collector backend to run indefinitely.
      - Changed the extracted `KpiValue` timestamp format to `"%Y-%m-%dT%H:%M:%SZ"` to meet Apache Spark requirements.
      - Removed some unused packages from the `requirements.in` file.
      Deployment Script:
      - Added `"telemetry"` and `"analytics"` component names to the `TFS_COMPONENTS` variable in the `my_deploy.sh` file.
      c0c517f0
  8. Sep 19, 2024
    • Waleed Akbar's avatar
      Changes to Analytics Backend: · 821e14f8
      Waleed Akbar authored
      - Corrected the name of `install_servicers()`.
      - Removed `stop_event` from the `RequestedListener` call.
      - Spark Streamer now writes to the `ALARMS` topic.
      - Improved messages and test files.
      821e14f8
  9. Sep 13, 2024
    • Waleed Akbar's avatar
      Changes in Analytics · c1ee28db
      Waleed Akbar authored
      Backend:
      - Updated the `StartSparkStreamer` function call to send `Analyzer_uuid` as the key for messages produced on the Kafka topic.
      - Updated the `SparkStream` definition to receive the key and added the key to the `streamwriter` object.
      
      Frontend:
      - Integrated APScheduler to manage `StreamListener`.
      - Added `ResponseListener` to consume messages from the `analytics_response_topic` and process them.
      - Added APScheduler to manage StreamListener.
      - Added "ResponseListener" to consumer messages from analytics response_topic and process it.
      c1ee28db
  10. Sep 09, 2024
    • Waleed Akbar's avatar
      Changes in Analytics · 8ac130c1
      Waleed Akbar authored
      Proto:
      - Added `Duration_s` field to the proto file.
      
      Frontend:
      - Added `SelectAnalyzer` logic.
      - Improved message formatting in the `create_analyzer_filter()` function.
      - Added a test case: `test_SelectAnalytics`.
      
      Backend:
      - Renamed the `RunSparkStreamer` method to `StartSparkStreamer`.
      - Updated the `StartRequestListener` method to return `(thread, stop_event)`.
      - Added a `StopRequestListener` method to stop the listener.
      
      Database:
      - Added the `select_with_filter` method with actual logic implementation.
      - Updated the `ConvertRowToAnalyzer` method to correctly read the `operation_mode` ENUM value.
      8ac130c1
  11. Sep 05, 2024
    • Waleed Akbar's avatar
      Changes in Analytics Backend. · 21832189
      Waleed Akbar authored
      - In the BackendService.py:
           + Updated SparkStreamer call with new parameters.
      - In SparkStreaming.py:
         + Added 'GetAggregerations' and 'ApplyThresholds' methods
         + Added 'window_size', 'win_slide_duration', 'time_stamp_col' and 'thresholds' parameters.
      - Added new messages.
      - Updated the 'RunSparkStreamer' call wth new parameters.
      21832189
  12. Sep 03, 2024
    • Waleed Akbar's avatar
      Changes in Telemetry · 067ace16
      Waleed Akbar authored
      - Added Kafka Request Listener to listen for requests from the Analytics Frontend.
      - Updated `SparkStreamer.py` to consume and process streams from VALUE topics, and further filter rows based on KPIs in the `input_kpis` list.
      - Updated the frontend TOPIC from `VALUE` to `ANALYTICS_REQUEST`.
      - Updated messages to keep the KPI UUID consistent with the one generated by the Telemetry Backend service.
      067ace16
    • Waleed Akbar's avatar
      Analytics Backend Initial Version · 0c5d2856
      Waleed Akbar authored
      - Added CRDB secret.
      - Added test script.
      - Added `main` and `__init__.py` files.
      - Added Backend Service file.
      - Added Spark Streamer file (successfully consuming streams from Kafka Topics).
      - Added Tests folder and files.
      - Added requirements file.
      - Added ANALYTICS Kafka topics in `common.tools.kafka.variables` file.
      0c5d2856
  13. Aug 27, 2024
    • Waleed Akbar's avatar
      Minor Changes in Telemetry test file. · 162e29e1
      Waleed Akbar authored
      - src folder refernce is removed from header
      162e29e1
    • Waleed Akbar's avatar
      Changes in Telemetry. · 622fe067
      Waleed Akbar authored
      - tests file name is corrected.
      - Telmetry frontend and backend requriements.in is updated
      622fe067
    • Waleed Akbar's avatar
      Changes to telemetry service · dbb3690f
      Waleed Akbar authored
      - TelemetryBackendService name and port is added in constants
      - get_kafka_address() call added in TelemetryBackendService
      - kafk topics test added in telemetrybackend test file
      - get_kafka_address() call added in TelemetryBackendServiceImpl
      - kafk topics test added in telemetryfrontend test file
      - improvements in Telemetry gitlab-ci file
         + unit_test for telemetry backend and frontend added.
         + many other small changes
      dbb3690f
    • Waleed Akbar's avatar
      Telemetry Backend Re-structuring · ce7fbb40
      Waleed Akbar authored
      - BackendService is restructured according to the design in report.
      - ResponseListener is added in Frontend
      - Improvements in test and messages files
      ce7fbb40
  14. Aug 11, 2024
  15. Aug 09, 2024
    • Waleed Akbar's avatar
      Changes to telemetry service · 6ac3c400
      Waleed Akbar authored
      - TelemetryBackendService name and port is added in constants
      - get_kafka_address() call added in TelemetryBackendService
      - kafk topics test added in telemetrybackend test file
      - get_kafka_address() call added in TelemetryBackendServiceImpl
      - kafk topics test added in telemetryfrontend test file
      - improvements in Telemetry gitlab-ci file
         + unit_test for telemetry backend and frontend added.
         + many other small changes
      6ac3c400
  16. Aug 08, 2024
    • Waleed Akbar's avatar
      Telemetry Backend Re-structuring · 1d01e32c
      Waleed Akbar authored
      - BackendService is restructured according to the design in report.
      - ResponseListener is added in Frontend
      - Improvements in test and messages files
      1d01e32c
  17. Jul 24, 2024
  18. Jul 09, 2024
  19. Jun 10, 2024
  20. Jun 05, 2024
  21. Jun 03, 2024
  22. May 30, 2024
  23. May 24, 2024
  24. May 10, 2024
  25. May 09, 2024
  26. May 08, 2024
  27. May 07, 2024
Loading