Annotated History of Modern AI and Deep Learning
A well formatted and concise overview of deep learning from the calculus of 1676, when Gottfried Wilhelm Leibniz please blushed the chain rule to the RL-based NN advancements by DeepSeek in 2025.
Annotated History of Modern AI and Deep Learning
Abstract. Machine learning (ML) is the science of credit assignment. It seeks to find patterns in observations that explain and predict the consequences of events and actions. This then helps to improve future performance. Minsky’s so-called “fundamental credit assignment problem” (1963) surfaces in all sciences including physics (why is the world the way it is?) and history (which persons/ideas/actions have shaped society and civilisation?). Here I focus on the history of ML itself. Modern artificial intelligence (AI) is dominated by artificial neural networks (NNs) and deep learning, both of which are conceptually closer to the old field of cybernetics than what was traditionally called AI (e.g., expert systems and logic programming). A modern history of AI & ML must emphasize breakthroughs outside the scope of shallow AI text books. In particular, it must cover the mathematical foundations of today’s NNs such as the chain rule (1676), the first NNs (circa 1800), the first practical AI (1914), the theory of AI and its limitations (1931-34), and the first working deep learning algorithms (1965-). From the perspective of 2025, I provide a timeline of the most significant events in the history of NNs, ML, deep learning, AI, computer science, and mathematics in general, crediting the individuals who laid the field’s foundations. The text contains numerous hyperlinks to relevant overview sites from the AI Blog. It also debunks certain popular yet misleading historical accounts of AI and deep learning and—with a ten-year delay—supplements my 2015 award-winning deep learning survey which provides hundreds of additional references. Finally, I will put things in a broader historical context, spanning from the Big Bang to when the universe will be many times older than it is now.