Deep Learning: Foundations, Architectures, and Applications

Mehrbakhsh Nilashi

Abstract


Deep learning is a specialized branch of machine learning characterized by neural networks with multiple layers of representation. By automatically learning hierarchical features from large datasets, deep learning models achieve state-of-the-art performance on tasks such as image recognition, speech understanding, and language processing. This paper provides an overview of deep learning concepts, including neural network structure and training, common model architectures, and their key differences from traditional machine learning. We discuss practical considerations in building deep models and highlight major application domains where deep learning has had significant impact. Overall, deep learning’s ability to capture complex patterns makes it a central technology in modern AI, enabling tasks previously out of reach for classical algorithms.


Keywords


Deep Learning, Convolutional and Recurrent Networks, Human-Level Accuracy, Artificial Neural Network

Full Text:

Abstract

References


Deng, L. and D. Yu, Deep learning: methods and applications. Foundations and trends® in signal processing, 2014. 7(3–4): p. 197-387.

Jena, B., G.K. Nayak, and S. Saxena, High-performance computing and its requirements in deep learning, in High-Performance Medical Image Processing. 2022, Apple Academic Press. p. 255-288.

Mienye, I.D. and T.G. Swart, A comprehensive review of deep learning: Architectures, recent advances, and applications. Information, 2024. 15(12): p. 755.

Akbari, E., et al., ANFIS modeling for bacteria detection based on GNR biosensor. Journal of Chemical Technology & Biotechnology, 2016. 91(6): p. 1728-1736.

Foroughi, B., et al., Determinants of intention to use autonomous vehicles: Findings from PLS-SEM and ANFIS. Journal of Retailing and Consumer Services, 2023. 70: p. 103158.

Iranmanesh, M., et al., Factors influencing attitude and intention to use autonomous vehicles in Vietnam: findings from PLS-SEM and ANFIS. Information Technology & People, 2024. 37(6): p. 2223-2246.

Kheirandish, A., et al., Using ANFIS technique for PEM fuel cell electric bicycle prediction model. International Journal of Environmental Science and Technology, 2019. 16(11): p. 7319-7326.

Nilashi, M., Evaluation of Security Pillars in the Industrial Internet of Things: A Fuzzy Logic Approach. Journal of Soft Computing and Decision Support Systems, 2024. 11(4): p. 1-7.

Nilashi, M., et al., Accuracy analysis of Type-2 fuzzy system in predicting parkinson’s disease using biomedical voice measures. International Journal of Fuzzy Systems, 2024. 26(4): p. 1261-1284.

Nilashi, M., et al., A hybrid method to solve data sparsity in travel recommendation agents using fuzzy logic approach. Mathematical Problems in Engineering, 2022. 2022(1): p. 7372849.

Nilashi, M., et al., Research Article A Hybrid Method to Solve Data Sparsity in Travel Recommendation Agents Using Fuzzy Logic Approach. 2022.

Nilashi, M., et al., Early diagnosis of Parkinson’s disease: A combined method using deep learning and neuro-fuzzy techniques. Computational biology and chemistry, 2023. 102: p. 107788.

Nilashi, M., et al., Preference learning for eco-friendly hotels recommendation: A multi-criteria collaborative filtering approach. Journal of Cleaner Production, 2019. 215: p. 767-783.

Nilashi, M., et al., A predictive method for hepatitis disease diagnosis using ensembles of neuro-fuzzy technique. Journal of infection and public health, 2019. 12(1): p. 13-20.

Nilashi, M., O. bin Ibrahim, and N. Ithnin, Hybrid recommendation approaches for multi-criteria collaborative filtering. Expert Systems with Applications, 2014. 41(8): p. 3879-3900.

Nilashi, M., O. bin Ibrahim, and N. Ithnin, Multi-criteria collaborative filtering with high accuracy using higher order singular value decomposition and Neuro-Fuzzy system. Knowledge-Based Systems, 2014. 60: p. 82-101.

Nilashi, M., et al., A multi-criteria collaborative filtering recommender system for the tourism domain using Expectation Maximization (EM) and PCA–ANFIS. Electronic Commerce Research and Applications, 2015. 14(6): p. 542-562.

Nilashi, M., et al., Measuring country sustainability performance using ensembles of neuro-fuzzy technique. Sustainability, 2018. 10(8): p. 2707.

Nilashi, M., et al., A soft computing method for the prediction of energy performance of residential buildings. Measurement, 2017. 109: p. 268-280.

Nilashi, M., et al., An analytical method for measuring the Parkinson’s disease progression: A case on a Parkinson’s telemonitoring dataset. Measurement, 2019. 136: p. 545-557.

Nilashi, M., et al., A multi-criteria recommendation system using dimensionality reduction and Neuro-Fuzzy techniques. Soft Computing, 2015. 19(11): p. 3173-3207.

Nilashi, M., et al., Analysis of travellers’ online reviews in social networking sites using fuzzy logic approach. International Journal of Fuzzy Systems, 2019. 21(5): p. 1367-1378.

Yadegaridehkordi, E., et al., Predicting determinants of hotel success and development using Structural Equation Modelling (SEM)-ANFIS method. Tourism Management, 2018. 66: p. 364-386.

Zogaan, W.A., et al., A combined method of optimized learning vector quantization and neuro-fuzzy techniques for predicting unified Parkinson's disease rating scale using vocal features. MethodsX, 2024. 12: p. 102553.

Nilashi, M., et al., Predicting parkinson’s disease progression: Evaluation of ensemble methods in machine learning. Journal of healthcare engineering, 2022. 2022(1): p. 2793361.

Ahmadi, N., et al., Eye State Identification Utilizing EEG Signals: A Combined Method Using Self?Organizing Map and Deep Belief Network. Scientific Programming, 2022. 2022(1): p. 4439189.

Nilashi, M., et al., The impact of multi-criteria ratings in social networking sites on the performance of online recommendation agents. Telematics and Informatics, 2023. 76: p. 101919.

Nilashi, M., et al., Customer satisfaction analysis and preference prediction in historic sites through electronic word of mouth. Neural Computing and Applications, 2022. 34(16): p. 13867-13881.

Nilashi, M., et al., Knowledge discovery for course choice decision in Massive Open Online Courses using machine learning approaches. Expert Systems with Applications, 2022. 199: p. 117092.

Nilashi, M., et al., An analytical method for diseases prediction using machine learning techniques. Computers & Chemical Engineering, 2017. 106: p. 212-223.

Nilashi, M., et al., Big social data analysis for impact of food quality on travelers’ satisfaction in eco-friendly hotels. ICT Express, 2023. 9(2): p. 182-188.

Nilashi, M., et al., Measuring sustainability through ecological sustainability and human sustainability: A machine learning approach. Journal of Cleaner Production, 2019. 240: p. 118162.

Nilashi, M., et al., Travellers decision making through preferences learning: A case on Malaysian spa hotels in TripAdvisor. Computers & Industrial Engineering, 2021. 158: p. 107348.

Nilashi, M., et al., A hybrid intelligent system for the prediction of Parkinson's Disease progression using machine learning techniques. Biocybernetics and Biomedical Engineering, 2018. 38(1): p. 1-15.

Agatonovic-Kustrin, S. and R. Beresford, Basic concepts of artificial neural network (ANN) modeling and its application in pharmaceutical research. Journal of pharmaceutical and biomedical analysis, 2000. 22(5): p. 717-727.

Rusk, N., Deep learning. Nature Methods, 2016. 13(1): p. 35-35.

Abumalloh, R.A., et al., Parkinson’s disease diagnosis using deep learning: A bibliometric analysis and literature review. Ageing research reviews, 2024. 96: p. 102285.

Nilashi, M., et al., A combined method for diabetes mellitus diagnosis using deep learning, singular value decomposition, and self-organizing map approaches. Diagnostics, 2023. 13(10): p. 1821.

Nilashi, M., et al., Online reviews analysis for customer segmentation through dimensionality reduction and deep learning techniques. Arabian Journal for Science and Engineering, 2021. 46(9): p. 8697-8709.

Safaei, M., et al., Deep learning algorithm for supervision process in production using acoustic signal. Applied Soft Computing, 2023. 146: p. 110682.

Nilashi, M., et al., Remote tracking of Parkinson's disease progression using ensembles of deep belief network and self-organizing map. Expert Systems with Applications, 2020. 159: p. 113562.

Madiajagan, M. and S.S. Raj, Parallel computing, graphics processing unit (GPU) and new hardware for deep learning in computational intelligence research, in Deep learning and parallel computing environment for bioengineering systems. 2019, Elsevier. p. 1-15.

Dhilleswararao, P., et al., Efficient hardware architectures for accelerating deep neural networks: Survey. IEEE access, 2022. 10: p. 131788-131828.

Huang, G., et al. Densely connected convolutional networks. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

Banerjee, C., T. Mukherjee, and E. Pasiliao Jr. An empirical study on generalizations of the ReLU activation function. in Proceedings of the 2019 ACM southeast conference. 2019.

Shen, S.-L., et al., Enhancement of neural networks with an alternative activation function tanhLU. Expert Systems with Applications, 2022. 199: p. 117181.

Pang, B., E. Nijkamp, and Y.N. Wu, Deep learning with tensorflow: A review. Journal of Educational and Behavioral Statistics, 2020. 45(2): p. 227-248.

Ketkar, N. and J. Moolayil, Introduction to pytorch, in Deep learning with python: learn best practices of deep learning models with PyTorch. 2021, Springer. p. 27-91.

Schmidt, R.M., Recurrent neural networks (rnns): A gentle introduction and overview. arXiv preprint arXiv:1912.05911, 2019.

Yuan, Z.-W. and J. Zhang. Feature extraction and image retrieval based on AlexNet. in Eighth International Conference on Digital Image Processing (ICDIP 2016). 2016. SPIE.

Mascarenhas, S. and M. Agarwal. A comparison between VGG16, VGG19 and ResNet50 architecture frameworks for Image Classification. in 2021 International conference on disruptive technologies for multi-disciplinary research and applications (CENTCON). 2021. IEEE.

Graves, A., Long short-term memory. Supervised sequence labelling with recurrent neural networks, 2012: p. 37-45.

Shewalkar, A., Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU. Journal of Artificial Intelligence and Soft Computing Research, 2019. 9.

Dai, X., H. Yin, and N.K. Jha, Grow and prune compact, fast, and accurate LSTMs. IEEE Transactions on Computers, 2019. 69(3): p. 441-452.

Chang, Y., et al., A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 2024. 15(3): p. 1-45.

Chen, Z., et al., Exploring the potential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter, 2024. 25(2): p. 42-61.

Corso, G., et al., Graph neural networks. Nature Reviews Methods Primers, 2024. 4(1): p. 17.

Wu, L., et al. Graph neural networks: foundation, frontiers and applications. in Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2022.

Mazhar, K. and P. Dwivedi. A survey on methods for explainability in deep learning models. in International Conference on Machine Intelligence, Tools, and Applications. 2024. Springer.

Jain, A., J.K. Sandhu, and P. Dwivedi, Understanding the Unseen: Explainability in Deep Learning for Computer Vision. Applied Computer Vision through Artificial Intelligence, 2025: p. 187-212.

Chel, Y.H. and L.L. Poh, Brain Tumor Classification in MRI: Insights from LIME and Grad-CAM Explainable AI Techniques. IEEE Access, 2025.

Narkhede, J. Comparative evaluation of post-hoc explainability methods in ai: Lime, shap, and grad-cam. in 2024 4th International Conference on Sustainable Expert Systems (ICSES). 2024. IEEE.

Akgündo?du, A. and ?. Çelikba?, Explainable deep learning framework for brain tumor detection: Integrating LIME, Grad-CAM, and SHAP for enhanced accuracy. Medical Engineering & Physics, 2025: p. 104405.

Nazim, S., et al., Advancing malware imagery classification with explainable deep learning: A state-of-the-art approach using SHAP, LIME and Grad-CAM. PLoS One, 2025. 20(5): p. e0318542.

Shah, M. and N. Sureja, A comprehensive review of bias in deep learning models: Methods, impacts, and future directions. Archives of Computational Methods in Engineering, 2025. 32(1): p. 255-267.


Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.