Comparative Evaluation of Multilayer Perceptron, Random Forest, and Support Vector Machine for Sepsis Classification Using Longitudinal Structured Clinical Data

Abstract

Sepsis classification from routinely collected clinical data is challenging because positive labels are uncommon, repeated observations from the same patient are correlated, and leakage-prone preprocessing can produce misleadingly high performance. This study re-evaluated multilayer perceptron (MLP), random forest (RF), and support vector machine (SVM) models using a leakage-controlled patient-level design. The source file contained 1,552,210 hourly observations from 40,336 patients; 2,932 patients (7.27%) were ever sepsis-positive. A reproducible stratified sample of 10,000 patients (382,387 hourly observations) was selected, preserving patient-level prevalence, and divided into 8,000 development and 2,000 independent hold-out patients with no patient overlap. Nine routinely available predictors were retained after a development-only missingness audit: heart rate, oxygen saturation, temperature, systolic blood pressure, mean arterial pressure, diastolic blood pressure, respiratory rate, age, and gender. Median imputation, missingness indicators, scaling, imbalance handling, hyperparameter tuning, and threshold selection were confined to development data and patient-grouped cross-validation. On the untouched hold-out cohort, RF achieved the highest ROC-AUC (0.671; 95% CI 0.635–0.706) and PR-AUC (0.041), with sensitivity 0.538 and specificity 0.716. RF significantly exceeded MLP in ROC-AUC (difference 0.0317, p=0.012), whereas RF and SVM did not differ significantly (p=0.054). Calibrated Brier scores were similar (0.0175–0.0176). Permutation importance and SHAP identified heart rate, respiratory rate, temperature, and blood-pressure measures as influential RF predictors. The models show moderate discrimination but low positive predictive value under severe class imbalance; therefore, they should be regarded as feasibility-level decision-support models requiring external, prospective, temporal, and fairness validation before clinical use.