- Sep 13, 2024
-
-
Waleed Akbar authored
Backend: - Updated the `StartSparkStreamer` function call to send `Analyzer_uuid` as the key for messages produced on the Kafka topic. - Updated the `SparkStream` definition to receive the key and added the key to the `streamwriter` object. Frontend: - Integrated APScheduler to manage `StreamListener`. - Added `ResponseListener` to consume messages from the `analytics_response_topic` and process them. - Added APScheduler to manage StreamListener. - Added "ResponseListener" to consumer messages from analytics response_topic and process it.
-
- Sep 09, 2024
-
-
Waleed Akbar authored
Proto: - Added `Duration_s` field to the proto file. Frontend: - Added `SelectAnalyzer` logic. - Improved message formatting in the `create_analyzer_filter()` function. - Added a test case: `test_SelectAnalytics`. Backend: - Renamed the `RunSparkStreamer` method to `StartSparkStreamer`. - Updated the `StartRequestListener` method to return `(thread, stop_event)`. - Added a `StopRequestListener` method to stop the listener. Database: - Added the `select_with_filter` method with actual logic implementation. - Updated the `ConvertRowToAnalyzer` method to correctly read the `operation_mode` ENUM value.
-
- Sep 07, 2024
-
-
Waleed Akbar authored
- Reverted an unintentional change made in the `tfs.sh` file.
-
Waleed Akbar authored
- Updated the position of the `stop_event` parameter. - Added confirmation for pySpark checkpoint deletion. - Added a PySpark termination script to handle the `StopCollector` event.
-
Waleed Akbar authored
**Frontend:** - Deleted irrelevant lines. **Backend:** - Added `delete_db_row_by_id` in the Stop Collector method.
-
Waleed Akbar authored
**Backend:** - Added a dictionary to manage running analyzers. - Implemented logic to manage running analyzers. - Added the `TerminateAnalyzerBackend` method to handle analyzer termination. **Frontend:** - Modified and invoked the `PublishStopRequestOnKafka` method.
-
Waleed Akbar authored
- Thresholds, window_size, window_slider is added in Frontend and Backend.
-
- Sep 06, 2024
-
-
Waleed Akbar authored
- UNSPECIFIED option added in the "AnalyzerOperationMode" enum as a best practice. - In SparkStreamer, changed the thresholds parameter from optional to compulsory.
-
Waleed Akbar authored
- parameters col added in DB - parameters field added in Analytics.proto - AnalyzerModel class methods changes - changes in messages file
-
- Sep 05, 2024
-
-
Waleed Akbar authored
-
Waleed Akbar authored
- Updated function names.
-
Waleed Akbar authored
- In the BackendService.py: + Updated SparkStreamer call with new parameters. - In SparkStreaming.py: + Added 'GetAggregerations' and 'ApplyThresholds' methods + Added 'window_size', 'win_slide_duration', 'time_stamp_col' and 'thresholds' parameters. - Added new messages. - Updated the 'RunSparkStreamer' call wth new parameters.
-
- Sep 03, 2024
-
-
Waleed Akbar authored
- SparkStreamer write Stream to Kafka topic ANALYTICS_RESPONSE.
-
Waleed Akbar authored
- Added Kafka Request Listener to listen for requests from the Analytics Frontend. - Updated `SparkStreamer.py` to consume and process streams from VALUE topics, and further filter rows based on KPIs in the `input_kpis` list. - Updated the frontend TOPIC from `VALUE` to `ANALYTICS_REQUEST`. - Updated messages to keep the KPI UUID consistent with the one generated by the Telemetry Backend service.
-
Waleed Akbar authored
- Added the missing `map<string, string>` in the `analytics_frontend.proto` file. - Removed `pyspark` and `java` from the `requirements.in` file, as they are not required for the Frontend.
-
Waleed Akbar authored
- Added CRDB secret. - Added test script. - Added `main` and `__init__.py` files. - Added Backend Service file. - Added Spark Streamer file (successfully consuming streams from Kafka Topics). - Added Tests folder and files. - Added requirements file. - Added ANALYTICS Kafka topics in `common.tools.kafka.variables` file.
-
- Aug 28, 2024
-
-
Waleed Akbar authored
- Created the Analytic Engine, Model, and DB files. - Added the DB connection. - Added `add_row_to_db` in `StartCollector`. - Added `delete_db_row_by_id` in `StopCollector`. - Improved message formatting. - Added a DB test file.
-
Waleed Akbar authored
- Actul logic is added in StartAnalyzer method. - Actual logic is added in StopAnalyzer method. - Improvements in message formats.
-
- Aug 27, 2024
-
-
Waleed Akbar authored
- Assignement number was incorrect, 5 was assigned twice.
-
Waleed Akbar authored
- AnalyzerId is added in "Analyzer" message type.
-
Waleed Akbar authored
- Unnecessary echo statements are removed from .gitlab-ci.yml file. - "-e ALLOW_ANONYMOUS_LOGIN=yes" flag is added to allow unauthorized connection with the zookeeper.
-
Waleed Akbar authored
- Created the CRDB secret (`crdb-telemetry`). - Updated the IF conditions in the `$COMPONENT` deployment loop within the `tfs.sh` file according to telemetry service requirements. - Added the `Telemetryservice.yaml` file to the manifest folder. - Updated the CRDB URL in `TelemetryEngine.py`. - Made a minor formatting change in `TelemetryModel.py`.
-
Waleed Akbar authored
- src folder refernce is removed from header
-
Waleed Akbar authored
- updated cmd in test file of KPI manager - move kafka producer object to __init__ function. - write JSON object to Kafka - Read JSON object from Kafka - Slight to manage the affect of JSON object. - Static methods are removed.
-
Waleed Akbar authored
- tests file name is corrected. - Telmetry frontend and backend requriements.in is updated
-
Waleed Akbar authored
-
Waleed Akbar authored
- Renamed the method to "KafkaKpiConsumer" to avoid conflict with the "KafkaConsumer" import in KpiApiWriter. - Removed unnecessary imports in KpiWriterToProm. - Added `get_service_port_grpc` call and imports in the Telemetry backend service. - Added new libraries to `requirements.in` for Telemetry.
-
Waleed Akbar authored
- GenericGrpcService inheritance is added in Telemetry backend class. - Database folder were missing in working directory in DockerFile of Telemetry Frontend.
-
Waleed Akbar authored
- start_http_server() call is move to main - CollectorRegistory variable is removed
-
Waleed Akbar authored
- src/$IMAGE_NAME/tests/Dockerfile is removed from API. - -f is added in "docker rm -f zookeeper" and "docker rm -f kafka" in Writer file.
-
Waleed Akbar authored
- docker run --name ... cmd missing (backend/frontend) after image name.
-
Waleed Akbar authored
- docker build ... -target builder is removed
-
Waleed Akbar authored
-
Waleed Akbar authored
-
Waleed Akbar authored
- TelemetryBackendService name and port is added in constants - get_kafka_address() call added in TelemetryBackendService - kafk topics test added in telemetrybackend test file - get_kafka_address() call added in TelemetryBackendServiceImpl - kafk topics test added in telemetryfrontend test file - improvements in Telemetry gitlab-ci file + unit_test for telemetry backend and frontend added. + many other small changes
-
Waleed Akbar authored
- Kafka deployment script is added in .gitlab-ci file - call is changed to "get_kafka_address()" in KpiValueWriter.py
-
Waleed Akbar authored
- Improvements in Kafka.variables files. - In KpiValueApiServiceImpl corrected the call from "admin_client()" to "kafka_address()"
-
Waleed Akbar authored
- In Kafka.variables files: get_kafka_address() and get_admin_client() is added. - In KpiValueApiServerImpl Kafka Admin Client call is updated. - Kafka deployment script is added.
-
Waleed Akbar authored
- __main__ is added. - DockerFile added. - .gitlab-ci.yml file added.
-
Waleed Akbar authored
- GenerateRawMetrics() add
-