Tag: Data Engineering

All the articles with the tag "Data Engineering".

• HL7 v2 ADT feeds: from pipe-delimited events to an encounters table

  1 Nov, 2026

  Parse HL7 v2 ADT messages in plain Python, ACK them over MLLP, and fold admits, discharges and cancellations into one row per hospital visit.

• Migrating from SFTP to a roster API: dual-run and reconciliation

  31 Oct, 2026

  Cut over from CSV-over-SFTP to a FHIR roster API without a big bang: phased dual-run, DuckDB reconciliation of CSV vs NDJSON, and exit criteria.

• Why flat-file rosters break: CSV over SFTP in healthcare data pipelines

  25 Oct, 2026

  CSV rosters over SFTP still run value-based care. Here are the failure modes: full replace, fuzzy identity, no acknowledgements, no deltas.

• Snowflake SSO from the terminal: CLI, dbt Core and DBeaver

  20 Oct, 2026

  One Entra ID SSO login, cached for 4 hours and shared by the Snowflake CLI, dbt Core and DBeaver. The errors on the way and the config that fixed them.

• Connecting Clinical and Operational Data to Tuva with dbt

  16 Oct, 2026

  How to map EHR and operational sources into the Tuva Input Layer with dbt, and where to join what Tuva does not model.

• Tuva column extensions: carry a custom column from eligibility to core

  16 Oct, 2026

  How to carry a custom column like contract_type from Tuva's eligibility input layer to core__eligibility using x_ extension columns, without forking.

• The limit_zero macro bug: how dbt resolves macro names across packages

  28 May, 2026 · Updated 15 Oct, 2026

  Why a Tuva demo project failed with 'limit_zero' is undefined on DuckDB: bare macro names don't cross packages, dispatch is a second lookup, and the fix.

• Partitioning Strategy in Data Lakes: When It Hurts Performance

  8 Oct, 2026

  Partitioning speeds up queries only when it matches filters and data volume. When over-partitioning backfires, and what to use instead.