Skip to content
  1. Sep 13, 2024
    • Lluis Gifre Renom's avatar
      Pre-merge commit · 05902708
      Lluis Gifre Renom authored
      05902708
    • Lluis Gifre Renom's avatar
      Analytics component: · 8e0f0601
      Lluis Gifre Renom authored
      - Corrected Liveness/Readiness probes
      8e0f0601
    • Waleed Akbar's avatar
      Changes in Analytic, manifest files and deployment script · 2c1fc9c8
      Waleed Akbar authored
      Deployment Script
      - Analytics is added in new montioring component to be deployed with TFS.
      - condition is added in analytics component TFS deployment script.
      - Analytics show logs files are added.
      Manifest
      - modifed KAFKA_ADVERTISED_LISTENERS with internal kafka service address in kafka.yml
      - analyticsservice.yml added.
      Analytics
      - grpc service is added in backend
      - Docker file for backend and frontend is added.
      - main file is updaed
      - small changes in requirements.in files
      Kafka Variables
      - get_kafka_address method logic is updated.
      2c1fc9c8
    • Waleed Akbar's avatar
      Changes in Analytics · c1ee28db
      Waleed Akbar authored
      Backend:
      - Updated the `StartSparkStreamer` function call to send `Analyzer_uuid` as the key for messages produced on the Kafka topic.
      - Updated the `SparkStream` definition to receive the key and added the key to the `streamwriter` object.
      
      Frontend:
      - Integrated APScheduler to manage `StreamListener`.
      - Added `ResponseListener` to consume messages from the `analytics_response_topic` and process them.
      - Added APScheduler to manage StreamListener.
      - Added "ResponseListener" to consumer messages from analytics response_topic and process it.
      c1ee28db
  2. Sep 09, 2024
    • Waleed Akbar's avatar
      Changes in Analytics · 8ac130c1
      Waleed Akbar authored
      Proto:
      - Added `Duration_s` field to the proto file.
      
      Frontend:
      - Added `SelectAnalyzer` logic.
      - Improved message formatting in the `create_analyzer_filter()` function.
      - Added a test case: `test_SelectAnalytics`.
      
      Backend:
      - Renamed the `RunSparkStreamer` method to `StartSparkStreamer`.
      - Updated the `StartRequestListener` method to return `(thread, stop_event)`.
      - Added a `StopRequestListener` method to stop the listener.
      
      Database:
      - Added the `select_with_filter` method with actual logic implementation.
      - Updated the `ConvertRowToAnalyzer` method to correctly read the `operation_mode` ENUM value.
      8ac130c1
  3. Sep 07, 2024
    • Waleed Akbar's avatar
      This Commit is for Error Correction · 54e0014b
      Waleed Akbar authored
      - Reverted an unintentional change made in the `tfs.sh` file.
      54e0014b
    • Waleed Akbar's avatar
      Changes in Analytics Backend · d6dc91c2
      Waleed Akbar authored
      - Updated the position of the `stop_event` parameter.
      - Added confirmation for pySpark checkpoint deletion.
      - Added a PySpark termination script to handle the `StopCollector` event.
      d6dc91c2
    • Waleed Akbar's avatar
      Changes in Telemetry Service · 9e226625
      Waleed Akbar authored
      **Frontend:**
      - Deleted irrelevant lines.
      
      **Backend:**
      - Added `delete_db_row_by_id` in the Stop Collector method.
      9e226625
    • Waleed Akbar's avatar
      Changes in Analytics · 88caa1cf
      Waleed Akbar authored
      **Backend:**
      - Added a dictionary to manage running analyzers.
      - Implemented logic to manage running analyzers.
      - Added the `TerminateAnalyzerBackend` method to handle analyzer termination.
      
      **Frontend:**
      - Modified and invoked the `PublishStopRequestOnKafka` method.
      88caa1cf
    • Waleed Akbar's avatar
      Changes in Analytics Service. · 88d7a200
      Waleed Akbar authored
      - Thresholds, window_size, window_slider is added in Frontend and Backend.
      88d7a200
  4. Sep 06, 2024
    • Waleed Akbar's avatar
      Changes in Analytics. · 5eed4743
      Waleed Akbar authored
      - UNSPECIFIED option added in the "AnalyzerOperationMode" enum as a best practice.
      - In SparkStreamer, changed the thresholds parameter from optional to compulsory.
      5eed4743
    • Waleed Akbar's avatar
      Changes in Analytics DB. · 990395f4
      Waleed Akbar authored
      - parameters col added in DB
      - parameters field added in Analytics.proto
      - AnalyzerModel class methods changes
      - changes in messages file
      990395f4
  5. Sep 05, 2024
  6. Sep 03, 2024
    • Waleed Akbar's avatar
      Changes in Analytics Backend · e7e36f7d
      Waleed Akbar authored
      - SparkStreamer write Stream to Kafka topic ANALYTICS_RESPONSE.
      e7e36f7d
    • Waleed Akbar's avatar
      Changes in Telemetry · 067ace16
      Waleed Akbar authored
      - Added Kafka Request Listener to listen for requests from the Analytics Frontend.
      - Updated `SparkStreamer.py` to consume and process streams from VALUE topics, and further filter rows based on KPIs in the `input_kpis` list.
      - Updated the frontend TOPIC from `VALUE` to `ANALYTICS_REQUEST`.
      - Updated messages to keep the KPI UUID consistent with the one generated by the Telemetry Backend service.
      067ace16
    • Waleed Akbar's avatar
      **Changes in Frontend** · c8ad3a5c
      Waleed Akbar authored
      - Added the missing `map<string, string>` in the `analytics_frontend.proto` file.
      - Removed `pyspark` and `java` from the `requirements.in` file, as they are not required for the Frontend.
      c8ad3a5c
    • Waleed Akbar's avatar
      Analytics Backend Initial Version · 0c5d2856
      Waleed Akbar authored
      - Added CRDB secret.
      - Added test script.
      - Added `main` and `__init__.py` files.
      - Added Backend Service file.
      - Added Spark Streamer file (successfully consuming streams from Kafka Topics).
      - Added Tests folder and files.
      - Added requirements file.
      - Added ANALYTICS Kafka topics in `common.tools.kafka.variables` file.
      0c5d2856
  7. Aug 28, 2024
    • Waleed Akbar's avatar
      Changes in Analytic DB and Frontend · 28bf80d3
      Waleed Akbar authored
      - Created the Analytic Engine, Model, and DB files.
      - Added the DB connection.
      - Added `add_row_to_db` in `StartCollector`.
      - Added `delete_db_row_by_id` in `StopCollector`.
      - Improved message formatting.
      - Added a DB test file.
      28bf80d3
    • Waleed Akbar's avatar
      Changes in Analytics Frontend. · e0a77d5f
      Waleed Akbar authored
      - Actul logic is added in StartAnalyzer method.
      - Actual logic is added in StopAnalyzer method.
      - Improvements in message formats.
      e0a77d5f
  8. Aug 27, 2024