Tag: Delta Lake

All the articles with the tag "Delta Lake".

• Partitioning Strategy in Data Lakes: When It Hurts Performance

  8 Oct, 2026

  Partitioning speeds up queries only when it matches filters and data volume. When over-partitioning backfires, and what to use instead.

• Delta Lake Schema Evolution Explained with Real Examples

  8 Oct, 2026

  Schema enforcement vs. evolution in Delta Lake: mergeSchema, overwriteSchema, MERGE evolution, column mapping for renames and drops, and type widening.

• Small Files Problem in Delta Lake: Detection and Compaction Strategy

  8 Oct, 2026

  How to detect small files in a Delta table from the transaction log, and a compaction strategy using OPTIMIZE, optimized writes and auto compaction.

• S3 Eventual Consistency Edge Cases in Data Pipelines

  8 Oct, 2026

  S3 has been strongly consistent since 2020. The edge cases that still break data pipelines: no atomic rename, concurrent writers, LastModified, events.

• Modern Table Formats: Iceberg, Delta Lake, and Hudi

  1 Oct, 2025

  Data Lakes made it possible to store raw data at scale, but they lacked the reliability and governance of data warehouses. Files could be dropped into storage (S3, HDFS, MinIO), but analysts struggled

• How Transactions Work in Databricks Using Delta Lake

  22 Jul, 2025

  Databricks is a powerful platform for big data analytics and machine learning. One of its key features is the ability to run transactional workloads over large-scale data lakes using Delta Lake . This

• Delta Lake vs. Traditional Data Lakes: Key Differences and Vendor Options

  30 Mar, 2025

  Introduction As data-driven organizations scale their analytics and machine learning workloads, the limitations of traditional data lakes become more apparent. Delta Lake is an open-source storage