Explainable Ensemble Detection of Mobile-Money Transaction Anomalies: An M-PESA Case Study

Abstract

the unauthorized transactions that follow may leave observable signatures in transaction records. This study implements and evaluates an Explainable Stacking Ensemble Machine Learning (XSEML) model for detecting transaction anomalies informed by documented social-engineering behaviour. The model combines Isolation Forest, XGBoost and Random Forest as base learners with a Logistic Regression meta-learner trained on out-of-fold predictions, and SHAP for instance-level explanation. An exploratory sequential mixed-methods design was used: a structured document review of cybersecurity reporting (2022–2025) identified four behavioural themes informing feature engineering. The model was trained and tested on a publicly available synthetic M-PESA dataset of 120,000 transactions (2.93% fraudulent). On an untouched 30% test set (36,000 transactions; 1,053 fraudulent) the ensemble achieved 98.8% accuracy, 89.8% precision, 65.5% recall, F1 = 0.758 and ROC-AUC = 0.831. Comparison against seven baselines with McNemar tests and bootstrap confidence intervals shows the ensemble significantly outperforming a 2σ rule (ΔF1 = +0.639, 95% CI [0.615, 0.664]) but significantly underperforming a tuned XGBoost classifier (ΔF1 = −0.026, 95% CI [−0.034, −0.019]; McNemar p < 0.001). Direct testing shows transaction month carries the highest SHAP attribution yet contributes almost nothing to discrimination (permutation AUC drop 0.0014; ablation ΔF1 = −0.002; χ² = 11.96, p = 0.367). More consequentially, a single hand-written threshold on the amount-to-balance ratio achieves perfect precision and F1 = 0.791, exceeding every model tested, because no legitimate transaction among 116,490 exceeds a ratio of 0.90. The dataset's fraud labels therefore appear rule-generated, and the evaluation is circular. We report this, rather than the performance figures, as the principal finding, and conclude that synthetic mobile-money datasets require explicit label-provenance auditing before they can support detection claims. The framework detects transaction anomalies consistent with social-engineering attack patterns; it does not detect social-engineering attacks.