My DE Zoomcamp Journey :Week 3 – Data Warehousing & BigQuery!

Diving into the World of OLAP, OLTP, and BigQuery This week was all about data at scale. We explored data warehousing, OLAP vs. OLTP, and Google BigQuery, diving deep into costs, best practices, and optimization techniques. If you're working with large datasets, these concepts are crucial for efficient querying, cost savings, and performance optimization. OLTP vs. OLAP: Understanding the Difference Before jumping into BigQuery, we first differentiated OLTP (Online Transaction Processing) and OLAP (Online Analytical Processing): Feature OLTP OLAP Purpose Runs essential business operations in real-time Supports decision-making, problem-solving, and analytics Data updates Short, fast user-initiated updates Scheduled, long-running batch jobs Database design Normalized for efficiency Denormalized for analysis Space requirements Smaller (historical data archived) Larger (aggregates vast amounts of data) Backup & Recovery Essential for business continuity Can reload data from OLTP if needed Users Clerks, customer-facing staff, online shoppers Data analysts, executives, business intelligence teams

Feb 12, 2025 - 11:33

My DE Zoomcamp Journey :Week 3 – Data Warehousing & BigQuery!

Diving into the World of OLAP, OLTP, and BigQuery

This week was all about data at scale. We explored data warehousing, OLAP vs. OLTP, and Google BigQuery, diving deep into costs, best practices, and optimization techniques. If you're working with large datasets, these concepts are crucial for efficient querying, cost savings, and performance optimization.

OLTP vs. OLAP: Understanding the Difference

Before jumping into BigQuery, we first differentiated OLTP (Online Transaction Processing) and OLAP (Online Analytical Processing):

Feature	OLTP	OLAP
Purpose	Runs essential business operations in real-time	Supports decision-making, problem-solving, and analytics
Data updates	Short, fast user-initiated updates	Scheduled, long-running batch jobs
Database design	Normalized for efficiency	Denormalized for analysis
Space requirements	Smaller (historical data archived)	Larger (aggregates vast amounts of data)
Backup & Recovery	Essential for business continuity	Can reload data from OLTP if needed
Users	Clerks, customer-facing staff, online shoppers	Data analysts, executives, business intelligence teams