De-identifying FHIR data: HIPAA Safe Harbor, limited data sets and leaks

// healthcare · compliance

TL;DR

HIPAA gives you two ways to de-identify: Safe Harbor (remove 18 kinds of identifiers and have no actual knowledge that someone can be re-identified) or Expert Determination (a qualified expert certifies the risk is very small). In FHIR, the identifiers hide in more places than Patient.name. Write the transform as an allowlist per resource type, re-key every id with random surrogates, and run a leak check against the source before anything leaves. If the analysis needs real dates, Safe Harbor is the wrong tool: you want a limited data set under a data use agreement.

The problem

After the Roster API series, two requests showed up in the same week:

1. A copy of the FHIR bulk export for the dev environment, “so we can debug with real data.”
2. An extract for an analytics vendor to study readmissions.

Both were phrased as “just remove the names and SSNs.” Neither is that simple. Real FHIR data is PHI, and under HIPAA, data is either de-identified according to 45 CFR §164.514 or it is still PHI. Nothing counts as “mostly anonymous.”

What I tried

A denylist on Patient

The first version removed the obvious fields:

for k in ("name", "telecom", "address", "identifier"):
    patient.pop(k, None)

To check it, I wrote a leak scanner (below) that takes identifier values from the source Patients and searches the output for them. On a Synthea sample (why Synthea) it found:

LEAK encounter.ndjson: 'Rivera'
LEAK encounter.ndjson: 'b8f0c1d2-0001'
LEAK patient.ndjson: 'Rivera'
LEAK patient.ndjson: 'b8f0c1d2-0001'
LEAK patient.ndjson: 'b8f0c1d2-0002'

Where they came from:

FHIR is built to link things together, and most of that linking carries identifying data: references with display names, narratives, extensions, identifiers on every resource (Encounter.identifier, Claim.identifier), contained resources and attachments. A denylist means finding every one of those, in every resource type, in every profile version. Miss one and the data is still PHI.

Hashing the MRN

The next idea was sha256(mrn) as the patient key: stable across runs, no lookup table. But Safe Harbor’s re-identification code rule (§164.514(c)) requires the code not be derived from information about the individual. A hash of the MRN is derived from the MRN. A salted hash is one leaked salt away from a lookup table, because the space of MRNs is small.

What actually worked

Pick the method first

The readmission study needs admit and discharge dates. Under Safe Harbor those become years, and the 30-day window disappears. So the vendor extract became a limited data set under a DUA (still PHI, with obligations), and the dev copy became Safe Harbor. Date shifting (one random offset per patient, keeping intervals) is a common way to keep readmission windows, but it is an Expert Determination technique: shifted dates are still “elements of dates” under Safe Harbor.

The Safe Harbor list, mapped to FHIR

The 18 identifier types apply to the individual and their relatives, employers and household members:

Plus the condition that is easy to forget: the covered entity must have no actual knowledge that the remaining information could identify someone.

Allowlist, flat output, surrogate keys

Instead of cleaning FHIR in place, the transform emits only the fields the consumer needs, as flat records. Everything not named is dropped, including fields that future profile versions add.

# deid.py — Safe Harbor-style extract from FHIR NDJSON
import json
import secrets
import sys
from datetime import date
from pathlib import Path

# 3-digit ZIPs with 20,000 or fewer people (HHS guidance, 2000 Census).
RESTRICTED_ZIP3 = {
    "036", "059", "063", "102", "203", "556", "692", "790", "821",
    "823", "830", "831", "878", "879", "884", "890", "893",
}
AS_OF = date(2026, 10, 31)


class Crosswalk:
    """Random surrogate keys. Kept by the covered entity, never shipped."""

    def __init__(self, path: Path):
        self.path = path
        self.map = json.loads(path.read_text()) if path.exists() else {}

    def key(self, reference: str) -> str:
        if reference not in self.map:
            self.map[reference] = secrets.token_hex(16)
        return self.map[reference]

    def save(self) -> None:
        self.path.write_text(json.dumps(self.map))


def year(value: str | None) -> int | None:
    return int(value[:4]) if value else None


def zip3(postal_code: str | None) -> str | None:
    if not postal_code:
        return None
    z = postal_code[:3]
    return "000" if z in RESTRICTED_ZIP3 else z


def codes(concept: dict | None) -> list[str]:
    # coding system + code only; `text` and `display` can carry free text
    return [f"{c.get('system')}|{c.get('code')}" for c in (concept or {}).get("coding", [])]


def patient(r: dict, xw: Crosswalk) -> dict:
    birth_year = year(r.get("birthDate"))
    age = AS_OF.year - birth_year if birth_year else None
    addr = (r.get("address") or [{}])[0]
    return {
        "patient_key": xw.key(f"Patient/{r['id']}"),
        "gender": r.get("gender"),
        "birth_year": birth_year if age is not None and age < 90 else None,
        "age_90_plus": age is not None and age >= 90,
        "death_year": None if age is not None and age >= 90 else year(r.get("deceasedDateTime")),
        "state": addr.get("state"),
        "zip3": zip3(addr.get("postalCode")),
    }


def encounter(r: dict, xw: Crosswalk) -> dict:
    return {
        "encounter_key": xw.key(f"Encounter/{r['id']}"),
        "patient_key": xw.key(r["subject"]["reference"]),
        "class": r.get("class", {}).get("code"),
        "type": [c for t in r.get("type", []) for c in codes(t)],
        "start_year": year(r.get("period", {}).get("start")),
    }


def condition(r: dict, xw: Crosswalk) -> dict:
    enc = r.get("encounter", {}).get("reference")
    return {
        "patient_key": xw.key(r["subject"]["reference"]),
        "encounter_key": xw.key(enc) if enc else None,
        "code": codes(r.get("code")),
        "onset_year": year(r.get("onsetDateTime")),
    }


TRANSFORMS = {"Patient": patient, "Encounter": encounter, "Condition": condition}


def main(src: Path, dst: Path, crosswalk: Path) -> None:
    xw = Crosswalk(crosswalk)
    dst.mkdir(exist_ok=True)
    for resource_type, transform in TRANSFORMS.items():
        with open(src / f"{resource_type}.ndjson") as fin, \
             open(dst / f"{resource_type.lower()}.ndjson", "w") as fout:
            for line in fin:
                fout.write(json.dumps(transform(json.loads(line), xw)) + "\n")
    xw.save()


if __name__ == "__main__":
    main(Path(sys.argv[1]), Path(sys.argv[2]), Path(sys.argv[3]))

python deid.py output/fhir deid/ /secure/crosswalk.json

{"patient_key": "89a5fad30bc69a41db2103e67a95ace6", "gender": "female", "birth_year": 1952, "age_90_plus": false, "death_year": null, "state": "MA", "zip3": "011"}
{"patient_key": "7d931c6ca2f5cacd01e7aa445fce0d2d", "gender": "male", "birth_year": null, "age_90_plus": true, "death_year": null, "state": "ID", "zip3": "834"}
{"encounter_key": "ba0aa776836dd18c408b802040b83e49", "patient_key": "89a5fad30bc69a41db2103e67a95ace6", "class": "IMP", "type": ["http://snomed.info/sct|183452005"], "start_year": 2026}

The decisions in that code:

• Surrogates are random and kept separately. The crosswalk lets the covered entity map results back if needed (for example, to send a care manager a list), which §164.514(c) allows as long as the code is not derived from the individual’s data and the mapping is not disclosed. Keep the crosswalk outside the extract, with the access controls of PHI.
• A 90+ birth year is suppressed, not just flagged. In 2026, a birth year of 1931 is an age over 89. The same goes for the death year of someone who would be over 89.
• Only system|code from codings. CodeableConcept.text and display are often free text typed by a human.
• References are re-keyed, never copied. Patient/b8f0c1d2-0001 is an identifier. Patient/89a5... is a surrogate.

The leak check runs every time

The allowlist is the design. The leak check is the test that it holds. It builds a set of identifier values from the source and fails if any of them appears anywhere in the output:

# leakcheck.py
import json
import re
import sys
from pathlib import Path


def tokens(src: Path) -> set[str]:
    """Identifier values from the source Patients: names, ids, phones, streets."""
    found = set()
    for line in open(src / "Patient.ndjson"):
        p = json.loads(line)
        found.add(p["id"])
        found.update(i["value"] for i in p.get("identifier", []))
        found.update(t["value"] for t in p.get("telecom", []))
        for n in p.get("name", []):
            found.update(re.sub(r"\d+", "", x) for x in [n.get("family", ""), *n.get("given", [])])
        for a in p.get("address", []):
            found.update(a.get("line", []))
            found.add(a.get("city", ""))
    return {t for t in found if len(t) >= 4}


def main(src: Path, dst: Path) -> None:
    needles = tokens(src)
    leaks = 0
    for f in dst.glob("*.ndjson"):
        text = f.read_text()
        for needle in needles:
            if needle in text:
                print(f"LEAK {f.name}: {needle!r}")
                leaks += 1
    sys.exit(1 if leaks else 0)


if __name__ == "__main__":
    main(Path(sys.argv[1]), Path(sys.argv[2]))

The denylist version produced the five leaks above. The allowlist version exits 0. Run the check in the same job that produces the extract and block the upload on a non-zero exit. It is a substring search, so it catches the obvious leaks, not every possible one. It does not replace review of new fields.

”No actual knowledge” means checking small groups

Safe Harbor can pass every field check and still fail on that condition. If one 1940-born woman in ZIP3 834 has a rare condition, anyone who knows her can find her row. HIPAA sets no minimum group size, but CMS’s data use policy for its own files (no cell under 11) is a reasonable bar:

from collections import Counter

groups = Counter((p["birth_year"], p["gender"], p["zip3"]) for p in patients)
small = {k: n for k, n in groups.items() if n < 11}

Generalize further (5-year birth bands, state instead of ZIP3) or suppress until small is empty, or until you can justify what remains.

For larger setups, Microsoft’s open source Tools for Health Data Anonymization applies FHIRPath-based rules (redact, date shift, crypto hash, generalize) to FHIR resources. Their defaults are a Safe Harbor-style starting point. The same caveats apply: crypto hashing and date shifting are outside strict Safe Harbor.

Key takeaways

• Decide the method before writing code. Safe Harbor reduces dates to years, so analytics that need intervals use a limited data set under a DUA or Expert Determination.
• In FHIR, identifiers live in narratives, references, extensions and resource ids, not just Patient.name. Emit an allowlist of fields and drop everything else.
• Re-key with random surrogates held in a separate crosswalk, not hashes, and block every extract on a leak check and a small-group check.

References

• 45 CFR §164.514 — Other requirements relating to uses and disclosures of PHI (eCFR)
• HHS — Guidance regarding methods for de-identification of PHI
• ResDAC — CMS cell size suppression policy
• Microsoft — Tools for Health Data Anonymization
• FHIR R4 — Patient
• FHIR R4 — References

03 Nov 2026 · #healthcare #fhir #security #python

GLB_