Synthea vs SynPUF: picking synthetic healthcare data for pipeline tests

// healthcare · synthetic data

TL;DR

Synthea simulates whole patient lives and exports FHIR, including Bulk Data NDJSON. Each patient is coherent, but population statistics are only as good as its disease modules. CMS’s DE-SynPUF is perturbed 2008–2010 Medicare claims: realistic file layouts and volumes, but no identities and ICD-9 codes. Use Synthea to test FHIR and identity logic, a claims dataset to test claims logic. Neither one tests your data quality, so inject the defects yourself.

The problem

The Roster API series needed test data at a realistic scale: tens of thousands of patients with names, birth dates and MRNs, so the converter, $member-add, bulk export and the dual-run reconciliation all had something to work on. Production data was out of the question. It is PHI, and a dev environment is not where PHI goes.

What I tried

Faker

Faker gives you names and dates in two lines. But nothing stands behind them: no encounters, no conditions, no coverage history. It worked for testing the CSV parser, and nothing past that.

CMS DE-SynPUF

The Data Entrepreneurs’ Synthetic Public Use File is CMS’s synthetic Medicare data: 2008–2010 claims, released in 20 subsamples, built from a real beneficiary sample with the values perturbed. The files mirror real CMS research files:

It is great for one thing: exercising claims-shaped pipelines at volume. For the roster API it failed fast:

• No identities. A synthetic DESYNPUF_ID, a birth date, sex and state. No names, no MRNs, so nothing to test identity matching with.
• ICD-9. Every diagnosis predates the October 2015 switch to ICD-10, so any current grouper or HCC logic needs a crosswalk first.
• Incoherent at the patient level. The synthesis keeps file-level distributions and breaks the links between variables, and CMS says plainly that the files are not meant for inference. A beneficiary can have claims that don’t make clinical sense together. That is fine for testing joins and badly misleading for analytics.

For claims logic I had already been using a synthetic Medicare dataset with Tuva (see what 167k synthetic Medicare claims taught me). The roster needed people.

What actually worked

Synthea, exported as Bulk FHIR

Synthea simulates each person from birth to the reference date. Disease modules are state machines (diabetes, COPD, pregnancy, …) that drive encounters, conditions, medications, observations and claims. The output is one coherent record per person.

# needs a JDK (17+); pin a tagged release for reproducibility
curl -LO https://github.com/synthetichealth/synthea/releases/download/master-branch-latest/synthea-with-dependencies.jar

java -jar synthea-with-dependencies.jar \
  -p 1000 -s 42 -cs 42 -r 20261001 \
  --exporter.fhir.bulk_data true \
  --exporter.years_of_history 5 \
  Massachusetts

With bulk_data on, output/fhir/ holds Patient.ndjson, Encounter.ndjson, Condition.ndjson and so on. That is the same shape the bulk export in part 4 produces, so the consumer side can be tested against Synthea output directly.

master-branch-latest moves. Seeds only reproduce a population on the same Synthea version, so for anything checked into CI, download a tagged release.

Reading it with DuckDB

SELECT
    list_filter(identifier, lambda i: i.type.coding[1].code = 'MR')[1].value AS mrn,
    regexp_replace(name[1].given[1], '\d+', '', 'g')                     AS first_name,
    regexp_replace(name[1].family,   '\d+', '', 'g')                     AS last_name,
    birthDate                                                            AS dob,
    address[1].state                                                     AS state
FROM read_json_auto('output/fhir/Patient.ndjson')
WHERE deceasedDateTime IS NULL;

┌───────────────┬────────────┬───────────┬────────────┬─────────┐
│      mrn      │ first_name │ last_name │    dob     │  state  │
├───────────────┼────────────┼───────────┼────────────┼─────────┤
│ b8f0c1d2-0001 │ Ana        │ Rivera    │ 1952-03-14 │ MA      │
└───────────────┴────────────┴───────────┴────────────┴─────────┘

Two Synthea quirks show up right away:

• Names have numbers. Synthea appends digits to names (Ana847 Rivera512) to make them distinguishable. Strip them, or every fuzzy-match test passes for the wrong reason.
• The MRN is the resource id. The MR identifier carries the same UUID as Patient.id. Real MRNs are short and facility-scoped, so truncate or re-map them.

On DuckDB 1.3+ use lambda i: ... for list lambdas. The old i -> ... arrow syntax is deprecated and DuckDB 2.0 rejects it by default.

Making it as messy as a real roster

Synthea’s data is too clean for the failure modes from part 1. No missing MRNs, no duplicates, no swapped dates. So the fixture generator injects them on purpose, with a fixed seed so tests are stable:

# synthea_to_roster.py
import csv
import random
import sys

import duckdb

SEED = 42
NPIS = ["1234567893", "1245319599", "1922082684"]

rows = duckdb.sql("""
    SELECT
        list_filter(identifier, lambda i: i.type.coding[1].code = 'MR')[1].value AS mrn,
        regexp_replace(name[1].given[1], '\\d+', '', 'g') AS first_name,
        regexp_replace(name[1].family,   '\\d+', '', 'g') AS last_name,
        birthDate::VARCHAR                                AS dob,
        date_diff('year', birthDate, DATE '2026-01-01')   AS age
    FROM read_json_auto(?)
    WHERE deceasedDateTime IS NULL
    ORDER BY mrn
""", params=[sys.argv[1]]).fetchall()

rng = random.Random(SEED)
out = []
for mrn, first, last, dob, age in rows:
    out.append({
        "mrn": mrn[:8],
        "first_name": first,
        "last_name": last,
        "dob": dob,
        "plan_type": "MA" if age >= 65 else rng.choice(["ACO", "COMMERCIAL", "MEDICAID"]),
        "provider_npi": rng.choice(NPIS),
        "period_start": "2026-01-01",
        "period_end": "2026-12-31",
    })

# Synthea data is too clean. Inject the defects real roster files have.
for row in rng.sample(out, k=max(1, len(out) // 50)):
    row["mrn"] = ""                                    # identity by name + DOB only
for row in rng.sample(out, k=max(1, len(out) // 100)):
    dup = dict(row, first_name=row["first_name"].upper(), mrn="")
    out.append(dup)                                    # same person, second row
for row in rng.sample(out, k=max(1, len(out) // 200)):
    y, m, d = row["dob"].split("-")
    if int(d) <= 12:
        row["dob"] = f"{y}-{d}-{m}"                    # day/month swapped upstream

writer = csv.DictWriter(sys.stdout, fieldnames=list(out[0]))
writer.writeheader()
writer.writerows(out)

python synthea_to_roster.py output/fhir/Patient.ndjson > roster.csv

The columns match the CSV from part 2, so the converter runs on it unchanged. Because the injected defects are known, the tests can assert exactly how many rows should fall back to $match, how many duplicates the dedupe should catch, and that the swapped dates surface as mismatches instead of silently becoming new patients.

plan_type is assigned from age, not taken from Synthea’s payer history. For a roster fixture that is a feature: the rule is obvious and easy to assert on.

Which one when

The last row is the important one. Synthetic data tests whether the pipeline does what you designed. It says nothing about whether the design survives your actual sources. Inject the defects you have seen in production, on purpose, with a seed.

Key takeaways

• Synthea for FHIR, identity and roster pipelines: coherent patients, Bulk NDJSON output, reproducible with seeds on a pinned version.
• DE-SynPUF for claims-shaped volume tests only: no identities, ICD-9 codes, and links between variables broken by design.
• Synthetic data is clean. Generate fixtures that inject your real-world defects with a fixed seed, and assert on the counts.

References

• Synthea (GitHub)
• Synthea wiki — Basic setup and running
• Synthea wiki — Common configuration
• CMS — Medicare Claims Synthetic Public Use Files (SynPUFs)
• DuckDB — lambda functions
• DuckDB — reading JSON

02 Nov 2026 · #healthcare #fhir #duckdb #testing

GLB_