E-Commerce Customer & Product Analytics Platform

An online store produces data in many shapes: catalogue updates, orders and payments, stock levels and a constant stream of website interactions. Each answers different questions, and they rarely line up on their own.

I built this platform to ingest all of it, transform it at scale and model it for business intelligence and customer analytics.

• Website interaction data is high-volume and event-driven, unlike the batch extracts from order and inventory systems.
• Late or duplicate records must not distort sales and inventory figures.
• Analysts need fast, predictable queries over a model they can understand.

• Python and SQL ETL/ELT pipelines for customer, product, order, payment and inventory data, with Cloud Storage holding raw and processed datasets.
• Event-driven ingestion of website interactions using Pub/Sub.
• PySpark transformations on Databricks with Delta Lake tables across Bronze, Silver and Gold layers.
• A star-schema model with fact and dimension tables, loaded into BigQuery for reporting.
• Incremental loading and data-quality checks, orchestrated with Apache Airflow and Cloud Composer.
• Optimized SQL and processing jobs for analytical performance.
• Exploratory GenAI work on product recommendations and natural-language questions over the sales data.

Running two engines is more to operate than one. Delta Lake handles large PySpark transformations and late-arriving data well, while BigQuery gives analysts fast SQL without managing clusters, so each layer uses the tool that suits it.

Analytics-ready datasets for customer behaviour, product performance, sales and revenue trends, and inventory management.