Adaptive Drift Aware Statistical Learning via Decaying Memory Operators
Main Article Content
Abstract
Modern data-driven systems operate in environments where the underlying data distribution changes over time. These non-stationary conditions arise in domains such as energy markets, financial systems, online services, and large-scale cyber-physical infrastructures. Classical statistical learning assumes stationary distributions. Under concept drift this assumption breaks, and model performance degrades. Many adaptive techniques attempt to address this challenge through sliding windows, exponential forgetting, or drift detection. These strategies often rely on heuristic update rules rather than explicit statistical formulations. This study introduces a mathematically grounded framework for adaptive statistical learning based on decaying memory operators. The approach embeds temporal memory directly inside the empirical risk functional. Memory decay becomes part of the learning objective rather than an external algorithmic mechanism. This formulation unifies sliding window estimation, exponential forgetting, and adaptive reweighting within a single analytical structure. Theoretical analysis is conducted under bounded loss and bounded drift assumptions. The analysis provides interpretable guarantees on convergence behavior and excess risk in evolving environments. The results show how the decay parameter governs the balance between stability and responsiveness. Experimental evaluation uses both synthetic drift benchmarks and a real-world electricity demand dataset containing more than forty-five thousand observations. Results demonstrate improved predictive accuracy and smoother adaptation compared with static empirical risk minimization and sliding window baselines. Sensitivity analysis further illustrates the impact of the decay parameter on learning dynamics. The proposed framework offers a principled foundation for adaptive statistical learning in non-stationary data environments. By modeling temporal memory explicitly, the approach improves interpretability, theoretical clarity, and practical robustness in streaming analytics applications.
Article Details
Issue
Section
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
References
Widmer, G., & Kubat, M. (1996). Learning in the presence of concept drift and hidden contexts. Machine Learning, 23, 69–101.
Tsymbal, A. (2004). The problem of concept drift: Definitions and related work. Trinity College Dublin.
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), 44.
Koychev, I. (2000). Gradual forgetting for adaptation to concept drift. In Workshop on Spatio-Temporal Reasoning.
Kolter, J., & Maloof, M. (2007). Dynamic weighted majority. Journal of Machine Learning Research, 8, 2755–2790.
Bifet, A., Gavaldà, R., Holmes, G., & Pfahringer, B. (2018). Machine learning for data streams. MIT Press.
Bottou, L., Curtis, F., & Nocedal, J. (2018). Optimization methods for large-scale machine learning. SIAM Review, 60(2), 223–311.
Cesa-Bianchi, N., & Lugosi, G. (2006). Prediction, learning, and games. Cambridge University Press.
Chen, Y., Hu, Z., Han, J., Deng, S., & Yin, J. (2024). Toward principled adaptive learning in non-stationary environments. Journal of Applied Data Analysis.
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2020). Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering.
Webb, G., Hyde, R., Cao, H., Nguyen, H., & Petitjean, F. (2022). Characterizing concept drift. Data Mining and Knowledge Discovery.
Montiel, J., Halford, M., Mastelini, S., Bolmier, G., Sourty, R., Vaysse, R., & Bifet, A. (2021). River: Machine learning for streaming data. Journal of Machine Learning Research.
Gomes, H. M., Bifet, A., Read, J., Barddal, J., Enembreck, F., Pfahringer, B., Holmes, G., & Abdessalem, T. (2017). Adaptive random forests for evolving data streams. Machine Learning, 106, 1469–1495.
Losing, V., Hammer, B., & Wersing, H. (2018). Incremental online learning: A review. Neural Networks, 67, 126–140.
Hinder, F., Katzfuss, M., & O’Connor, L. (2021). Online learning under concept drift. Statistics and Computing, 31, 76.
Cano, A., Krawczyk, B., & Woźniak, M. (2020). Online ensemble learning for concept drift. Information Sciences, 520, 221–237.
Bouchachia, A., & Sayed-Mouchaweh, M. (2022). Concept drift adaptation in evolving data streams. Evolving Systems, 13, 1–14.
Haug, J., Kasneci, G., & Fischer, M. (2023). Drift detection in machine learning pipelines. Machine Learning, 112, 1915–1943.
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024). Continual learning in neural networks: A review. Artificial Intelligence Review, 57.
Parisi, G. I., Kemker, R., Part, J., Kanan, C., & Wermter, S. (2019). Continual lifelong learning with neural networks. Neural Networks, 113, 54–71.
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021). Continual learning: A comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A., Milan, K., Quan, J., Ramalho, T., & Grabska-Barwinska, A. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114, 3521–3526.
Aggarwal, C. C. (2021). Neural networks and deep learning. Springer.
Zhou, Z. H. (2021). Machine learning. Springer.
Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018). Foundations of machine learning. MIT Press.
Bousquet, O., & Elisseeff, A. (2002). Stability and generalization. Journal of Machine Learning Research, 2, 499–526.
Minku, L. L., White, A. P., & Yao, X. (2010). The impact of diversity on online ensemble learning in the presence of concept drift. IEEE Transactions on Knowledge and Data Engineering, 22, 730–742.
Alquier, P., & Guedj, B. (2018). Simplified PAC-Bayesian bounds for hostile data. Machine Learning, 107, 887–902.
Zhang, Q., Wang, Y., & Zhao, Z. (2023). Adaptive learning for streaming data with concept drift. Knowledge-Based Systems, 266, 110355.
Dong, F., Liu, A., Lu, J., & Zhang, G. (2021). Drift detection methods in evolving data streams. IEEE Transactions on Knowledge and Data Engineering.
Li, Y., Chen, H., Wang, X., & Zhou, J. (2025). Adaptive streaming machine learning under non-stationary environments. IEEE Transactions on Neural Networks and Learning Systems.
Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J., & Woźniak, M. (2017). Ensemble learning for data stream analysis: A survey. Information Fusion, 37, 132–156.
Žliobaitė, I. (2010). Learning under concept drift: An overview. Vilnius University.
Pesaranghader, A., Viktor, H. L., & Paquet, E. (2018). Fast Hoeffding drift detection method for evolving data streams. Machine Learning, 107, 857–881.
Gama, J., Medas, P., Castillo, G., & Rodrigues, P. (2004). Learning with drift detection. In Advances in artificial intelligence (pp. 286–295).
Liu, A., Lu, J., Dong, F., & Zhang, G. (2023). Online learning for evolving data streams: A survey. ACM Computing Surveys.
Dong, F., Zhang, G., Lu, J., & Liu, A. (2022). A framework for adaptive learning in non-stationary environments. Knowledge-Based Systems, 243, 108449.
Baena-García, M., del Campo, Á., Fidalgo, R., & Morales-Bueno, R. (2006). Early drift detection method. In Proceedings of the international workshop on knowledge discovery from data streams.
Bifet, A., & Gavaldà, R. (2007). Learning from time-changing data with adaptive windowing. In Proceedings of the SIAM international conference on data mining.