Posts
All the articles I've posted.
-
Small Files Problem in Delta Lake: Detection and Compaction Strategy
How to detect small files in a Delta table from the transaction log, and a compaction strategy using OPTIMIZE, optimized writes and auto compaction.
-
Comparing Parquet Compression Codecs: Snappy vs ZSTD Benchmark
A reproducible PyArrow benchmark of Parquet codecs on 5M rows: Snappy, LZ4, GZIP and ZSTD levels 1–19. File size, write and read time.
-
S3 Eventual Consistency Edge Cases in Data Pipelines
S3 has been strongly consistent since 2020. The edge cases that still break data pipelines: no atomic rename, concurrent writers, LastModified, events.
-
Managing OAuth2 Token Rotation in Postman Without Overwriting Environments
Keep OAuth2 access and refresh tokens out of your Postman environments: collection-scoped storage, a pre-request refresh script, and Newman-safe exports.
-
Designing Event Schemas for Product Analytics in ClickHouse
Fixed columns vs Map vs JSON, ORDER BY for funnels, dedup, TTL and materialized views: a practical event schema for product analytics in ClickHouse.
-
Why ClickHouse Is Fast: Understanding MergeTree Internals
Parts, granules, sparse primary indexes, marks and background merges: a hands-on look at how MergeTree lets ClickHouse skip most of your data.
-
How to Get the Status of an AWS Glue Workflow Using boto3
A Glue workflow can report COMPLETED while its jobs failed. How to poll workflow runs with boto3 and read per-node status from the run graph.
-
AWS Glue Job Fails Due to Dependency Version Conflict: How to Fix It
Your Glue job works locally but fails with ImportError or numpy binary incompatibility. Why it happens and how to pin dependencies against the Glue runtime.