Skip to content
Snippets Groups Projects
  1. Sep 13, 2024
    • Waleed Akbar's avatar
      Changes in Analytics · c1ee28db
      Waleed Akbar authored
      Backend:
      - Updated the `StartSparkStreamer` function call to send `Analyzer_uuid` as the key for messages produced on the Kafka topic.
      - Updated the `SparkStream` definition to receive the key and added the key to the `streamwriter` object.
      
      Frontend:
      - Integrated APScheduler to manage `StreamListener`.
      - Added `ResponseListener` to consume messages from the `analytics_response_topic` and process them.
      - Added APScheduler to manage StreamListener.
      - Added "ResponseListener" to consumer messages from analytics response_topic and process it.
      c1ee28db
  2. Sep 09, 2024
    • Waleed Akbar's avatar
      Changes in Analytics · 8ac130c1
      Waleed Akbar authored
      Proto:
      - Added `Duration_s` field to the proto file.
      
      Frontend:
      - Added `SelectAnalyzer` logic.
      - Improved message formatting in the `create_analyzer_filter()` function.
      - Added a test case: `test_SelectAnalytics`.
      
      Backend:
      - Renamed the `RunSparkStreamer` method to `StartSparkStreamer`.
      - Updated the `StartRequestListener` method to return `(thread, stop_event)`.
      - Added a `StopRequestListener` method to stop the listener.
      
      Database:
      - Added the `select_with_filter` method with actual logic implementation.
      - Updated the `ConvertRowToAnalyzer` method to correctly read the `operation_mode` ENUM value.
      8ac130c1
  3. Sep 07, 2024
    • Waleed Akbar's avatar
      Changes in Analytics Backend · d6dc91c2
      Waleed Akbar authored
      - Updated the position of the `stop_event` parameter.
      - Added confirmation for pySpark checkpoint deletion.
      - Added a PySpark termination script to handle the `StopCollector` event.
      d6dc91c2
    • Waleed Akbar's avatar
      Changes in Analytics · 88caa1cf
      Waleed Akbar authored
      **Backend:**
      - Added a dictionary to manage running analyzers.
      - Implemented logic to manage running analyzers.
      - Added the `TerminateAnalyzerBackend` method to handle analyzer termination.
      
      **Frontend:**
      - Modified and invoked the `PublishStopRequestOnKafka` method.
      88caa1cf
    • Waleed Akbar's avatar
      Changes in Analytics Service. · 88d7a200
      Waleed Akbar authored
      - Thresholds, window_size, window_slider is added in Frontend and Backend.
      88d7a200
  4. Sep 05, 2024
    • Waleed Akbar's avatar
      Changes in Analytics Backend. · 21832189
      Waleed Akbar authored
      - In the BackendService.py:
           + Updated SparkStreamer call with new parameters.
      - In SparkStreaming.py:
         + Added 'GetAggregerations' and 'ApplyThresholds' methods
         + Added 'window_size', 'win_slide_duration', 'time_stamp_col' and 'thresholds' parameters.
      - Added new messages.
      - Updated the 'RunSparkStreamer' call wth new parameters.
      21832189
  5. Sep 03, 2024
    • Waleed Akbar's avatar
      Changes in Telemetry · 067ace16
      Waleed Akbar authored
      - Added Kafka Request Listener to listen for requests from the Analytics Frontend.
      - Updated `SparkStreamer.py` to consume and process streams from VALUE topics, and further filter rows based on KPIs in the `input_kpis` list.
      - Updated the frontend TOPIC from `VALUE` to `ANALYTICS_REQUEST`.
      - Updated messages to keep the KPI UUID consistent with the one generated by the Telemetry Backend service.
      067ace16
    • Waleed Akbar's avatar
      Analytics Backend Initial Version · 0c5d2856
      Waleed Akbar authored
      - Added CRDB secret.
      - Added test script.
      - Added `main` and `__init__.py` files.
      - Added Backend Service file.
      - Added Spark Streamer file (successfully consuming streams from Kafka Topics).
      - Added Tests folder and files.
      - Added requirements file.
      - Added ANALYTICS Kafka topics in `common.tools.kafka.variables` file.
      0c5d2856
  6. Apr 18, 2024
  7. Feb 01, 2023
  8. Jul 26, 2022
    • Lluis Gifre Renom's avatar
      PathComp component: · ef420f23
      Lluis Gifre Renom authored
      Common:
      - added helper scripts
      
      FrontEnd:
      - corrected import paths
      - added missing parameters in test
      - refined creation of backend request
      ef420f23
  9. May 20, 2022
  10. Mar 08, 2022
  11. Feb 17, 2022
  12. Feb 16, 2022
  13. Nov 04, 2021
  14. Sep 23, 2021
  15. Jul 16, 2021
    • Lluis Gifre Renom's avatar
      Added testunit example: · 98fc013f
      Lluis Gifre Renom authored
      - monitoring/test_monitoring.py: provides a basic testing skeleton
      - monitoring/requirements.in/.txt: updated with pytest
      - monitoring/genproto.py: added instructions to import generated Python modules using "from . import ..." instead of "import ..."; otherwise, pytest and other possible modules doing imports might complain.
      - monitoring/monitoring_server.py: packed start_server and stop_server functionaity to simplify test definition, minor formattings, bug resolutions and improvements in package import.
      - monitoring/monitoring_client.py: minor formattings, bug resolutions, and improvements in package import.
      98fc013f
  16. Jun 14, 2021
  17. Jun 11, 2021
Loading