Migration to Google Cloud Platform resulted in cost savings for a global leader in CPG industry
Challenge
We needed to develop a Data Platform that could effectively monitor interactions across various touchpoints and provide real-time insights into customer behaviours and product usage. The platform was specifically designed to cater to the company’s Machine Learning and Business Intelligence requirements. It had to efficiently gather data from different sources such as web-clicks, mobile app clickstream, CIAM events, and interactions from the Loyalty Management System.
Our approach
To facilitate the migration of parquet files to the Google Cloud Platform from another cloud storage provider, we relied on Google Cloud Storage buckets and the STS service. For the ingestion and transformation of the flat files, we employed pySpark in Dataproc and loaded them into BigQuery. To handle the loading of events from SaaS CDP, we utilized batch Dataflow jobs. Real-time data sources were ingested through PubSub with BigQuery serving as the sink. The orchestration of ETL processes and downstream SQL processing was implemented as well.
The outcome
The Consumer Data Platform within the Google Cloud Platform serves as the ultimate source of accurate information for all business intelligence (BI) and machine learning (ML) models. The platform possesses the capability to efficiently handle the ingestion of hundreds of millions of customer events on a daily basis. By utilizing the toolset provided by the Google Cloud Platform, the marketing technologists were able to successfully move away from the isolated data storage approach, resulting in the ability to gain a comprehensive, real-time understanding of each individual customer.