Student Success & Retention Modeling

To support retention outreach, I built interpretable logistic-regression models predicting student persistence risk. The interpretable model reached ROC AUC ≈ 0.719; when validated against a subsequent outreach application, performance held at ROC AUC ≈ 0.679. Those are honest, modest, useful numbers — good enough to prioritize outreach far better than chance, and reported with their limitations attached.

0.719

ROC AUC — interpretable model

0.679

ROC AUC — outreach validation

Outreach capacity is finite. Advisors and retention staff can meaningfully contact only a fraction of students in a term, so the operative question isn't 'who might leave?' — it's 'given limited hours, who should we reach first?' That's a ranking problem, and it's exactly what a well-calibrated, interpretable model is for.

I chose logistic regression deliberately over more complex models. In a student-success context the model has to be explainable to the people acting on it — an advisor should be able to see why a student is flagged, and leadership should be able to interrogate the features. Feature engineering drew on enrollment intensity, course-taking patterns, and academic progress indicators; features were vetted for both predictive value and appropriateness for an equity-sensitive use.

Evaluation used held-out data with ROC AUC as the headline discrimination metric. The later outreach-validation check — how the model ranked students in actual application — is the number I consider most meaningful, and it's the lower one: 0.679. Models nearly always degrade from notebook to practice, and reporting that degradation is part of doing this work honestly.

A risk score is not an intervention. The model's output was translated into prioritized outreach lists with the drivers attached in plain language, so staff conversations started from context rather than from a number. Score thresholds were set by outreach capacity — the model ranks; humans decide.

• An AUC around 0.7 means meaningful but imperfect discrimination — the model is a triage aid, not an oracle, and was presented that way.
• Predictions were reviewed for the risk of encoding existing inequities; the intended use (more support, sooner) fails safe, but that framing was made explicit rather than assumed.
• Whether the outreach itself improved retention is a separate causal question that would require a designed comparison to answer; the validation reported here concerns the model's ranking quality, not intervention effect.

• Logistic regression, feature engineering, and model evaluation (ROC AUC) in Python/R
• Validation against real-world application, not just held-out data
• Interpretation and intervention design with non-technical staff
• Responsible-analytics framing: limitations stated, causal claims avoided