Real-time monitoring and data warehouse for energy management
Advizeo wanted to implement a comprehensive real-time energy monitoring solution capable of collecting, processing, and analyzing data from thousands of connected IoT devices.
Project objectives
- Ensure continuous monitoring of energy data to optimize consumption
- Build a reliable and scalable data warehouse in the cloud.
- Automate deployments to ensure stability and rapid updates.
Key Steps
Development of data pipelines with Kafka and KSQL
enabling continuous processing and real-time analysis of collected data.
Implementation of a cloud warehouse
based on Amazon Redshift, with transformations orchestrated via DBT and automated ingestion by Airbyte.
Automation of CI/CD processes
using GitLab CI, orchestrated by Kubernetes and ArgoCD, ensuring efficient continuous integration and deployment.
Project Details
Execution Date:
April 2021 – November 2022
Client
Advizeo
Location
Paris, France
English (UK) 
Français