- 05 Sep, 2024 1 commit
-
-
Waleed Akbar authored
- In the BackendService.py: + Updated SparkStreamer call with new parameters. - In SparkStreaming.py: + Added 'GetAggregerations' and 'ApplyThresholds' methods + Added 'window_size', 'win_slide_duration', 'time_stamp_col' and 'thresholds' parameters. - Added new messages. - Updated the 'RunSparkStreamer' call wth new parameters.
-
- 03 Sep, 2024 4 commits
-
-
Waleed Akbar authored
- SparkStreamer write Stream to Kafka topic ANALYTICS_RESPONSE.
-
Waleed Akbar authored
- Added Kafka Request Listener to listen for requests from the Analytics Frontend. - Updated `SparkStreamer.py` to consume and process streams from VALUE topics, and further filter rows based on KPIs in the `input_kpis` list. - Updated the frontend TOPIC from `VALUE` to `ANALYTICS_REQUEST`. - Updated messages to keep the KPI UUID consistent with the one generated by the Telemetry Backend service.
-
Waleed Akbar authored
- Added the missing `map<string, string>` in the `analytics_frontend.proto` file. - Removed `pyspark` and `java` from the `requirements.in` file, as they are not required for the Frontend.
-
Waleed Akbar authored
- Added CRDB secret. - Added test script. - Added `main` and `__init__.py` files. - Added Backend Service file. - Added Spark Streamer file (successfully consuming streams from Kafka Topics). - Added Tests folder and files. - Added requirements file. - Added ANALYTICS Kafka topics in `common.tools.kafka.variables` file.
-
- 28 Aug, 2024 2 commits
-
-
Waleed Akbar authored
- Created the Analytic Engine, Model, and DB files. - Added the DB connection. - Added `add_row_to_db` in `StartCollector`. - Added `delete_db_row_by_id` in `StopCollector`. - Improved message formatting. - Added a DB test file.
-
Waleed Akbar authored
- Actul logic is added in StartAnalyzer method. - Actual logic is added in StopAnalyzer method. - Improvements in message formats.
-
- 27 Aug, 2024 33 commits
-
-
Waleed Akbar authored
- Assignement number was incorrect, 5 was assigned twice.
-
Waleed Akbar authored
- AnalyzerId is added in "Analyzer" message type.
-
Waleed Akbar authored
- Unnecessary echo statements are removed from .gitlab-ci.yml file. - "-e ALLOW_ANONYMOUS_LOGIN=yes" flag is added to allow unauthorized connection with the zookeeper.
-
Waleed Akbar authored
- Created the CRDB secret (`crdb-telemetry`). - Updated the IF conditions in the `$COMPONENT` deployment loop within the `tfs.sh` file according to telemetry service requirements. - Added the `Telemetryservice.yaml` file to the manifest folder. - Updated the CRDB URL in `TelemetryEngine.py`. - Made a minor formatting change in `TelemetryModel.py`.
-
Waleed Akbar authored
- src folder refernce is removed from header
-
Waleed Akbar authored
- updated cmd in test file of KPI manager - move kafka producer object to __init__ function. - write JSON object to Kafka - Read JSON object from Kafka - Slight to manage the affect of JSON object. - Static methods are removed.
-
Waleed Akbar authored
- tests file name is corrected. - Telmetry frontend and backend requriements.in is updated
-
Waleed Akbar authored
-
Waleed Akbar authored
- Renamed the method to "KafkaKpiConsumer" to avoid conflict with the "KafkaConsumer" import in KpiApiWriter. - Removed unnecessary imports in KpiWriterToProm. - Added `get_service_port_grpc` call and imports in the Telemetry backend service. - Added new libraries to `requirements.in` for Telemetry.
-
Waleed Akbar authored
- GenericGrpcService inheritance is added in Telemetry backend class. - Database folder were missing in working directory in DockerFile of Telemetry Frontend.
-
Waleed Akbar authored
- start_http_server() call is move to main - CollectorRegistory variable is removed
-
Waleed Akbar authored
- src/$IMAGE_NAME/tests/Dockerfile is removed from API. - -f is added in "docker rm -f zookeeper" and "docker rm -f kafka" in Writer file.
-
Waleed Akbar authored
- docker run --name ... cmd missing (backend/frontend) after image name.
-
Waleed Akbar authored
- docker build ... -target builder is removed
-
Waleed Akbar authored
-
Waleed Akbar authored
-
Waleed Akbar authored
- TelemetryBackendService name and port is added in constants - get_kafka_address() call added in TelemetryBackendService - kafk topics test added in telemetrybackend test file - get_kafka_address() call added in TelemetryBackendServiceImpl - kafk topics test added in telemetryfrontend test file - improvements in Telemetry gitlab-ci file + unit_test for telemetry backend and frontend added. + many other small changes
-
Waleed Akbar authored
- Kafka deployment script is added in .gitlab-ci file - call is changed to "get_kafka_address()" in KpiValueWriter.py
-
Waleed Akbar authored
- Improvements in Kafka.variables files. - In KpiValueApiServiceImpl corrected the call from "admin_client()" to "kafka_address()"
-
Waleed Akbar authored
- In Kafka.variables files: get_kafka_address() and get_admin_client() is added. - In KpiValueApiServerImpl Kafka Admin Client call is updated. - Kafka deployment script is added.
-
Waleed Akbar authored
- __main__ is added. - DockerFile added. - .gitlab-ci.yml file added.
-
Waleed Akbar authored
- GenerateRawMetrics() add
-
Waleed Akbar authored
- BackendService is restructured according to the design in report. - ResponseListener is added in Frontend - Improvements in test and messages files
-
Waleed Akbar authored
- Changed the column type of the start and end timestamps to Float. - Added ConvertRowToCollector() in TelemetryModel. - Renamed the class variable from "DBobj" to "tele_db_obj". - Renamed the local variable from "collector_id" to "collector_uuid". - Added PublishStopRequestOnKafka() to publish the stop collector request on Kafka. - Improved the test files.
-
Waleed Akbar authored
- pytest logging is changed to DEBUG. - ManagemetDB test is removed. - Irrelevent imports and methods are removed from the test file. -
-
Waleed Akbar authored
- collector description is removed from TelemetryModel. - "ConvertCollectorToRow" is added in Telemetry Model class. - NameMapping is removed from service client and service. - TelemetryDB object name and import is updated with correct class name. - StartCollector is restructured. - "PublishRequestOnKafka" is restructured.
-
Waleed Akbar authored
- added start_time and end_time in proto.
-
Waleed Akbar authored
-
-
Waleed Akbar authored
- README files of kpi manager/value writer/api are updated to reflect new changes.
-
Waleed Akbar authored
- PROM_URL variable is updated with FQDN of Prometheus.
-
Waleed Akbar authored
- unused imports and functions are removed
-
Waleed Akbar authored
- refine Kpi_DB.py methods. - improve description of messages. - imporve the text description.
-
Waleed Akbar authored
- KFK_NAMESPACE and KFK_PORT
-