Calin O. Deep Learning Architectures. A Mathematical Approach 2020

Textbook in PDF format

This book describes how neural networks operate from the mathematical perspective, having in mind that the success of the neural networks methods should not be determined by trial-and-error or luck, but by a clear mathematical analysis. The main goal of the present work is to write the ideas and concepts of neural networks, which are used nowadays at an intuitive level, into a precise modern mathematical language. The book is a mixture of old good classical mathematics and modern concepts of Deep Learning. The main focus is on the mathematical side, since in today’s developing trend many mathematical aspects are kept silent and most papers underline only the computer science details and practical applications.
The multiple commercial applications of neural networks have been highly profitable. Neural networks are imbedded into novel technologies, which are now used successfully by top companies such as Google, Microsoft,IBM, Apple, Adobe, Netflix, NVIDIA, and Baidu. A neural network is a collection of computing units, which are connected together, called neurons, each producing a real-valued outcome, called activation. Input neurons get activated from the sensors that perceive the environment, while the other neurons get activated from the previous neuron activations. This structure allows neurons to send messages among themselves and consequently, to straighten those connections that lead to success in solving a problem and diminishing those which are leading to failure.
This book can be used in a graduate course in Deep Learning, with the first few parts being accessible to senior undergraduates. In addition, the book will be of wide interest to Machine Learning researchers who are interested in a theoretical understanding of the subject