Predicting patient risk with scikit-learn on top of HCC suspecting data

The hcc_suspecting__summary table from the Tuva Project has one row per patient with their HCC gap count — the number of suspected conditions that haven’t appeared in current-year claims yet. The median on the 167k claim synthetic dataset is 3 gaps per patient.

That sets up a natural classification target: high_risk = 1 if suspecting_gaps >= median. The goal is to predict this before running the full HCC suspecting calculation — useful for early outreach prioritization where you want a risk signal faster than the full pipeline produces one.

Feature table construction

Query hcc_suspecting__summary directly from DuckDB:

import duckdb
import pandas as pd

con = duckdb.connect("tuva.duckdb", read_only=True)

df = con.execute("""
    SELECT
        patient_id,
        patient_age,
        CASE WHEN patient_sex = 'M' THEN 1 ELSE 0 END AS sex_male,
        total_paid_amount,
        condition_count,
        suspecting_gaps
    FROM hcc_suspecting__summary
    WHERE patient_age IS NOT NULL
      AND total_paid_amount IS NOT NULL
""").df()

con.close()

Four features:

• patient_age — age at end of measurement period
• sex_male — binary encoding of sex
• total_paid_amount — sum of all paid claim amounts for this patient in the measurement period
• condition_count — count of distinct HCC conditions in prior year claims

Target definition

median_gaps = df["suspecting_gaps"].median()
df["high_risk"] = (df["suspecting_gaps"] >= median_gaps).astype(int)

Using the dataset median produces a roughly balanced class split, which is useful for a demo and avoids needing class-weight adjustments. In production, the threshold would be defined clinically or by business need, not the data distribution.

Training

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

features = ["patient_age", "sex_male", "total_paid_amount", "condition_count"]
X = df[features]
y = df["high_risk"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

clf = RandomForestClassifier(n_estimators=100, random_state=42, n_jobs=-1)
clf.fit(X_train, y_train)

print(classification_report(y_test, clf.predict(X_test)))

On the synthetic dataset this produces accuracy around 0.80 depending on the split. condition_count is the most important feature by Gini impurity, followed by total_paid_amount. Age contributes. Sex contributes least.

This makes clinical sense: patients with more distinct conditions in prior year are more likely to have more HCC gaps, and paid amount is a proxy for overall clinical complexity (more utilization = more diagnoses = more HCC exposure).

Streamlit integration

The predictions page caches the trained model with @st.cache_resource and scores all patients:

import streamlit as st
import pandas as pd
import plotly.express as px

@st.cache_resource
def train_model():
    # ... load data, train, return clf and feature list
    return clf, features

clf, features = train_model()

df_scored = df.copy()
df_scored["risk_score"] = clf.predict_proba(df_scored[features])[:, 1]

st.subheader("Highest-risk patients")
st.dataframe(
    df_scored.sort_values("risk_score", ascending=False)[
        ["patient_id", "patient_age", "total_paid_amount", "condition_count", "risk_score"]
    ].head(50),
    use_container_width=True,
)

Feature importance is shown as a bar chart:

importance_df = pd.DataFrame({
    "feature": features,
    "importance": clf.feature_importances_,
}).sort_values("importance", ascending=True)

fig = px.bar(
    importance_df,
    x="importance",
    y="feature",
    orientation="h",
    title="Feature importances (Gini)",
)
st.plotly_chart(fig, use_container_width=True)

What this model is and isn’t

This is a demo-grade model. It predicts suspecting_gaps >= median from four basic features, trained and tested on synthetic data where the correlations are built in by construction. The 0.80 accuracy reflects genuine signal in the features, not a real-world validation.

For a production deployment, the requirements change substantially:

• Real historical claims with known outcomes
• More features: prior-year HCC code list, provider visit frequency, dual eligibility status
• Time-windowed train/test split (train on year N, predict year N+1)
• Clinical validation of the risk threshold and model outputs

The model here proves the pipeline from DuckDB mart to scored output works end to end. That’s the right scope for a local demo on synthetic data.