Latest blog posts by Michal Milosz

Optimize large datasets in Databricks with Partitioning, Z-Ordering, Auto Optimize, Delta Lake Vacuum, Caching, and Cost Monitoring for better performance.

Configure the Celery Kubernetes Executor in Airflow 2.0 to combine Celery's scalability with Kubernetes' resource optimization for efficient workflows.

Discover the top data engineering trends for 2025, including AI-driven automation, Lakehouse architecture, serverless and edge computing, and sustainable data practices. Embracing these innovations will help organisations optimise data operations and stay ahead in a rapidly changing landscape.

The article compares three key data architectures: data warehouses for structured analytics, data lakes for flexible raw data storage, and lakehouse as a unified model. It highlights their strengths, limitations, and when each is the best fit for modern organizations.

Explore stream vs batch processing, their benefits, use cases, and technologies. Learn how to choose the right data strategy for your business needs.

Discover how microservices revolutionize data engineering by replacing monolithic systems, boosting scalability, flexibility, and deployment efficiency.

Explore the shift from Excel to data lakes, modern storage solutions, and best practices for scalable, efficient, and innovative data management strategies.

Lakehouse Federation in Databricks: query external data sources without moving data. See how it works and when to use it in your data platform.