Explainable AI Models for Diabetes Risk Prediction

Scale:  Leading academic medical system, with model development and validation spanning two major hospital networks.

Product/system area impacted:  Clinical decision-support research: predictive modeling for diabetes management, built on real-world electronic health record (EHR) data.

Challenge

Main problem:  Clinical teams needed a way to flag, ahead of time, which diabetic patients were likely to fail early treatment or come back within 30 days of hospitalization. A prediction alone wasn’t enough. Clinicians also needed to see why the model flagged a given patient before they’d act on it.

Why it existed:  The risk factors live in two different kinds of data: static patient attributes (social and lifestyle history) and dense temporal records (procedures, diagnoses, lab results, vitals, devices). Sorting through that by hand is slow, and most machine-learning models that could do it automatically are black boxes clinicians have no reason to trust.

Impact before the project:  Without a validated tool, clinicians had no systematic way to flag at-risk patients early. That meant more preventable cases of poor glycemic control and more readmissions than there needed to be.

Goals / Success Criteria

Top goals:

• Build explainable AI models so clinicians can understand why a diabetes-related prediction was made
• Identify patients likely to fail early treatment and need additional support
• Identify patients at high risk of hospital readmission within 30 days

Key metrics:  Model accuracy and precision on real-world patient data; consistency of performance when tested across two independent hospital systems.

Must-have constraints:  The models had to run on real-world patient data, not synthetic data, get validated at a second institution, and be releasable publicly so other researchers could build on the work.

Our Role

FLS responsibility: Clinovera, First Line Software’s healthcare division, owned the architecture and development of the predictive models end-to-end: feature engineering, model design, training, and evaluation.

Scope & timeline:  We worked in phases, moving from data preparation and feature engineering through model training and, finally, multi-site validation.

Approach

Key delivery phases: Data preparation and feature engineering, then model architecture design for both TCN and RNN/LSTM, then internal training and validation at the primary site, then external validation at a second, independent hospital system.

Decisions that reduced risk or accelerated delivery: 

• The feature space was wide and mixed (static plus temporal), so we quantized numeric features into quintile-based binary groups to make it tractable
• We put a self-attention layer at the model’s entry point over the temporal feature vectors. It cuts dimensionality, and it also shows which sequences and features mattered most for a given prediction — that’s what makes the model explainable instead of a black box

Quality and stability:  We built two competing architectures, TCN and RNN/LSTM, and benchmarked them against each other rather than betting on one from the start. A second hospital system then validated our results independently, confirming the models hold up outside the data they were trained on.

Solution

Solution headline:  Two deep-learning models, one predicting critical A1C (blood sugar control) change and the other predicting 30-day hospital readmission, trained on real-world patient data and validated at two major hospital systems.

Core components:

• Two architectures we built and compared head-to-head: a Temporal Convolutional Network (TCN) and a Recurrent Neural Network (RNN, LSTM variant)
• A feature pipeline combining roughly 200 static binary features (social and lifestyle attributes) with temporal sequences covering procedures, diagnoses, lab measurements, vital signs, and devices
• 36 time points per sequence for the A1C model; 6-hour intervals for the readmission model
• A self-attention layer that compresses the large dynamic feature space and flags which sequences matter most for a given prediction
• Validation at two collaborating hospital systems, with public release planned so other researchers can use the work

What makes it different:  Most predictive models hand clinicians a number and nothing else. Ours shows its work: the self-attention mechanism lets a clinician see which part of a patient’s history actually drove the prediction. We also validated it at two independent institutions instead of one.

Technology

Key technologies:  Temporal Convolutional Networks (TCN) and Recurrent Neural Networks with LSTM hidden layers, self-attention for feature reduction and interpretability, and real-world EHR data covering both static and temporal patient records.

Tooling for reliability/visibility:  On-premise GPU compute for model training and evaluation.

Infrastructure: 

• 3 × NVIDIA A6000 GPUs (48 GB each)
• 512 GB RAM

Results

Measurable improvements:

• A1C (blood sugar control) prediction: TCN hit 76% accuracy and precision; RNN (LSTM) reached 85%
• 30-day readmission (RNN only, since sequence lengths varied too much for TCN): initial training reached 89% accuracy and precision
• Running both architectures side by side gave us real evidence for which model to trust for which task, instead of a guess

Strongest impact statements:

• RNN beat TCN on A1C prediction by 9 percentage points, and it’s the explainable version, so clinicians get both the better number and the reasoning behind it
• An 89% initial readmission score is a strong starting point, though the class imbalance in the data still needs to be worked out before it’s ready for discharge planning

What’s next: We still need to tune the readmission model to deal with class imbalance in the training data. Once external validation at the second site wraps up, we plan to release both models publicly for other researchers to use.

“The hard part was never getting a model to spit out a risk score. It was getting one we could actually stand behind in front of a patient. The self-attention layer is what changed that for us—instead of a black box, we can see which piece of a patient’s history the model actually weighted. We ran two architectures side by side and let the data pick the winner rather than committing to one upfront, and having a second hospital system validate the results independently mattered more to me than the accuracy numbers themselves.” — Alexander Turchin, Associate Professor of Medicine at Harvard Medical School.

AI Enablement

Is AI used, and how: Yes. This isn’t a project with an AI feature bolted on; the two predictive models are the product.

Specific AI components: RNN (LSTM) and TCN architectures for prediction; a self-attention mechanism that makes feature importance interpretable; quintile-based feature quantization as the data-engineering step that feeds both models.

Business problem solved: Care teams get an early signal for which diabetic patients are likely to lose glycemic control or land back in the hospital within 30 days, and they get to see why, so they can step in before a crisis instead of after one.

Measurable impact: Up to 85% accuracy and precision on A1C risk prediction, 89% on initial readmission-risk prediction, and an attention mechanism that turns the model’s reasoning into something a clinician can actually read.

Before / After

This was built from scratch; tracks clinical capability:

August 2026