Tag: Data Engineering
All the articles with the tag "Data Engineering".
-
COALESCE doesn't care what your columns mean
COALESCE checks types, not meaning. How merging columns with different semantics silently breaks data, and the dbt guardrails that catch it.
-
Triggering AWS Glue Jobs from EventBridge: Workflows, Scheduler, and the IAM You Actually Need
A Glue job isn't a native EventBridge rule target. Three working patterns (workflow, Scheduler, Step Functions) with the exact IAM each one needs.
-
Debugging AWS Glue Job Bookmarks Not Working as Expected
Glue job bookmarks fail silently: reprocessed data, skipped files, or no state at all. A checklist of the root causes and how to verify each one.
-
Where Agents Schema fits in a dbt → Snowflake → Looker pipeline
Agents Schema doesn't replace dbt, semantic views or LookML. It indexes them inside the warehouse for agents. Where it fits, what it costs, when to skip it.
-
Why Athena Returns No Rows on Partitioned Tables (and How to Fix It)
Your data is in S3, the table exists, and Athena returns zero rows. Six root causes behind partition metadata desync and how to diagnose each one.
-
Tuva's encounter grouper only groups claims. Here's what happens to your ADT encounters
Tuva's encounter grouper only runs on claims. Clinical ADT/EHR encounters are standardized and stacked in core.encounter, never matched or deduped.
-
What's new in cms_hcc v1.0.0: the payer grain fix nobody's docs warned you about
cms_hcc v1.0.0 fixes a silent grain mismatch between int_members and int_disease_factors that dropped every disease factor for members who switched payers mid-year.
-
DuckDB concurrency in 2026: why you can't run dbt and DBeaver at the same time
DuckDB's single-writer model means that opening a .duckdb file in DBeaver blocks dbt from acquiring its write connection. Here's the exact error, why it happens, and the right fix.