Marketing data and ERP integration platform
The Creative Club managed data scattered across multiple marketing platforms and an internal ERP system, which complicated analysis and slowed down decision-making. The need was to centralize this data to obtain a unified and reliable view. This project aimed to create a high-performance, secure, and scalable platform to automate data processing and facilitate its use.
Project objectives
- Centralize data from multiple heterogeneous sources.
- Build reliable and high-performance pipelines for data processing and analysis.
- Automate deployments to ensure the quality and speed of updates.
Key Steps
Multi-source data ingestion
including Amazon S3, Salesforce, SharePoint, Facebook Ads, and on-premise ERP, for complete coverage of information flows.
Creation of robust ELT pipelines
on Redshift, then migration to Snowflake, thereby optimizing analytical performance.
Deployment of Dagster and Airbyte
on an EKS (Amazon Elastic Kubernetes Service) cluster, with continuous integration and continuous deployment (CI/CD) orchestrated via GitHub Actions.
Project Details
Execution Date:
December, 2023
Client
The Creative Club
English (UK) 
Français