Capacity development

AI is no longer a concept confined to research laboratories or science fiction novels. From smartphones that recognise faces to virtual assistants that understand speech and recommendation engines that predict what we want to watch next, AI has become embedded in everyday life.

Behind this transformation lies a set of core principles, or the fundamentals of AI, which explain how machines learn, adapt, and perform tasks once considered the exclusive domain of humans.

At the heart of modern AI are neural networks, mathematical structures inspired by the human brain. They organise computation into layers of interconnected nodes, or artificial neurones, which process information and learn from examples.

Unlike traditional programming, where every rule must be explicitly defined, neural networks can identify patterns in data autonomously. The ability to learn and improve with experience underpins the astonishing capabilities of today’s AI.

Multi-layer perceptron networks

A neural network consists of multiple layers of interconnected neurons, not just a simple input and output layer. Each layer processes the data it receives from the previous layer, gradually building hierarchical representations.

In image recognition, early layers detect simple features, such as edges or textures, middle layers combine these into shapes, and later layers identify full objects, like faces or cars. In natural language processing, lower layers capture letters or words, while higher layers recognise grammar, context, and meaning.

Without multiple layers, the network would be shallow, limited in its ability to learn, and unable to handle complex tasks. Multi-layer, or deep networks, are what enable AI to perform sophisticated functions like autonomous driving, medical diagnosis, and language translation.

How mathematics drives artificial intelligence

The foundation of AI is mathematics. Without linear algebra, calculus, probability, and optimisation, modern AI systems would not exist. These disciplines allow machines to represent, manipulate, and learn from vast quantities of data.

Linear algebra allows inputs and outputs to be represented as vectors and matrices. Each layer of a neural network transforms these data structures, performing calculations that detect patterns in data, such as shapes in images or relationships between words in a sentence.

Calculus, especially the study of derivatives, is used to measure how small changes in a network’s parameters, called weights, affect its predictions. This information is critical for optimisation, which is the process of adjusting these weights to improve the network’s accuracy.

The loss function measures the difference between the network’s prediction and the actual outcome. It essentially tells the network how wrong it is. For example, the mean squared error measures the average squared difference between the predicted and actual values, while cross-entropy is used in classification tasks to measure how well the predicted probabilities match the correct categories.

Gradient descent is an algorithm that uses the derivative of the loss function to determine the direction and magnitude of changes to each weight. By moving weights gradually in the direction that reduces the loss, the network learns over time to make more accurate predictions.

Backpropagation is a method that makes learning in multi-layer neural networks feasible. Before its introduction in the 1980s, training networks with more than one or two layers was extremely difficult, as it was hard to determine how errors in the output layer should influence the earlier weights. Backpropagation systematically propagates this error information backwards through the network.

At its core, it applies the chain rule of calculus to compute gradients, indicating how much each weight contributes to the overall error and the direction it should be adjusted. Combined with gradient descent, this iterative process allows networks to learn hierarchical patterns, from simple edges in images to complex objects, or from letters to complete sentences.

Backpropagation has transformed neural networks from shallow, limited models into deep, powerful tools capable of learning sophisticated patterns and making human-like predictions.

Why neural network architecture matters

The arrangement of layers in a network, or its architecture, determines its ability to solve specific problems.

Activation functions introduce non-linearity, giving networks the ability to map complex, high-dimensional data. ReLU (Rectified Linear Unit), one of the most widely used activation functions, addresses critical training issues and enables deep networks to learn efficiently.

Convolutional neural networks (CNNs) excel in image and video analysis. By applying filters across images, CNNs detect local patterns like edges and textures. Pooling layers reduce spatial dimensions, making computation faster while preserving essential features. Local connectivity ensures neurones process only relevant input regions, mimicking human vision.

Recurrent neural networks (RNNs) and their variants, such as LSTMs and GRUs, process sequential data like text or audio. They maintain a hidden state that acts as memory, capturing dependencies over time, a crucial feature for tasks such as speech recognition or predictive text.

Transformer revolution and attention mechanisms

In 2017, AI research took a major leap with the introduction of Transformer models. Unlike RNNs, which process sequences step by step, transformers use attention mechanisms to evaluate all parts of the input simultaneously.

The attention mechanism calculates which elements in a sequence are most relevant to each output. Using linear algebra, it compares query, key, and value vectors to assign weights, highlighting important information and suppressing irrelevant details.

That approach enabled the creation of large language models (LLMs) such as GPT and BERT, capable of generating coherent text, answering questions, and translating languages with unprecedented accuracy.

Transformers reshaped natural language processing and have since expanded into areas such as computer vision, multimodal AI, and reinforcement learning. Their ability to capture long-range context efficiently illustrates the power of combining deep learning fundamentals with innovative architectures.

How does AI learn and generalise?

One of the central challenges in AI is ensuring that networks learn meaningful patterns from data rather than simply memorising individual examples. The ability to generalise and apply knowledge learnt from one dataset to new, unseen situations is what allows AI to function reliably in the real world.

Supervised learning is the most widely used approach, where networks are trained on labelled datasets, with each input paired with a known output. The model learns to map inputs to outputs by minimising the difference between its predictions and the actual results.

Applications include image classification, where the system distinguishes cats from dogs, or speech recognition, where spoken words are mapped to text. The accuracy of supervised learning depends heavily on the quality and quantity of labelled data, making data curation critical for reliable performance.

Unsupervised learning, by contrast, works with unlabelled data and seeks to uncover hidden structures and patterns. Clustering algorithms, for instance, can group similar customer profiles in marketing, while dimensionality reduction techniques simplify complex datasets for analysis.

The paradigm enables organisations to detect anomalies, segment populations, and make informed decisions from raw data without explicit guidance.

Reinforcement learning allows machines to learn by interacting with an environment and receiving feedback in the form of rewards or penalties. Unlike supervised learning, the system is not told the correct action in advance; it discovers optimal strategies through trial and error.

That approach powers innovations in robotics, autonomous vehicles, and game-playing AI, enabling systems to learn long-term strategies rather than memorise specific moves.

A persistent challenge across all learning paradigms is overfitting, which occurs when a network performs exceptionally well on training data but fails to generalise to new examples. Techniques such as dropout, which temporarily deactivate random neurons during training, encourage the network to develop robust, redundant representations.

Similarly, weight decay penalises excessively large parameter values, preventing the model from relying too heavily on specific features. Achieving proper generalisation is crucial for real-world applications: self-driving cars must correctly interpret new road conditions, and medical AI systems must accurately assess patients with cases differing from the training dataset.

By learning patterns rather than memorising data, AI systems become adaptable, reliable, and capable of making informed decisions in dynamic environments.

The black box problem and explainable AI (XAI)

Deep learning and other advanced AI technologies rely on multi-layer neural networks that can process vast amounts of data. While these networks achieve remarkable accuracy in image recognition, language translation, and decision-making, their complexity often makes it extremely difficult to explain why a particular prediction was made. That phenomenon is known as the black box problem.

Though these systems are built on rigorous mathematical principles, the interactions between millions or billions of parameters create outputs that are not immediately interpretable. For instance, a healthcare AI might recommend a specific diagnosis, but without interpretability tools, doctors may not know what features influenced that decision.

Similarly, in finance or law, opaque models can inadvertently perpetuate biases or produce unfair outcomes.

Explainable AI (XAI) seeks to address this challenge. By combining the mathematical and structural fundamentals of AI with transparency techniques, XAI allows users to trace predictions back to input features, assess confidence, and identify potential errors or biases.

In practice, this means doctors can verify AI-assisted diagnoses, financial institutions can audit credit decisions, and policymakers can ensure fair and accountable deployment of AI.

Understanding the black box problem is therefore essential not only for developers but for society at large. It bridges the gap between cutting-edge AI capabilities and trustworthy, responsible applications, ensuring that as AI systems become more sophisticated, they remain interpretable, safe, and beneficial.

Data and computational power

Modern AI depends on two critical ingredients: large, high-quality datasets and powerful computational resources. Data provides the raw material for learning, allowing networks to identify patterns and generalise to new situations.

Image recognition systems, for example, require millions of annotated photographs to reliably distinguish objects, while language models like GPT are trained on billions of words from books, articles, and web content, enabling them to generate coherent, contextually aware text.

High-performance computation is equally essential. Training deep neural networks involves performing trillions of calculations, a task far beyond the capacity of conventional processors.

Graphics Processing Units (GPUs) and specialised AI accelerators enable parallel processing, reducing training times from months to days or even hours. This computational power enables real-time applications, such as self-driving cars interpreting sensor data instantly, recommendation engines adjusting content dynamically, and medical AI systems analysing thousands of scans within moments.

The combination of abundant data and fast computation also brings practical challenges. Collecting representative datasets requires significant effort and careful curation to avoid bias, while training large models consumes substantial energy.

Researchers are exploring more efficient architectures and optimisation techniques to reduce environmental impact without sacrificing performance.

The future of AI

The foundations of AI continue to evolve rapidly, driven by advances in algorithms, data availability, and computational power. Researchers are exploring more efficient architectures, capable of learning from smaller datasets while maintaining high performance.

For instance, self-supervised learning allows a model to learn from unlabelled data by predicting missing information within the data itself, while few-shot learning enables a system to understand a new task from just a handful of examples. These methods reduce the need for enormous annotated datasets and make AI development faster and more resource-efficient.

Transformer models, powered by attention mechanisms, remain central to natural language processing. The attention mechanism allows the network to focus on the most relevant parts of the input when making predictions.

For example, when translating a sentence, it helps the model determine which words are most important for understanding the meaning. Transformers have enabled the creation of large language models like GPT and BERT, capable of summarising documents, answering questions, and generating coherent text.

Beyond language, multimodal AI systems are emerging, combining text, images, and audio to understand context across multiple sources. For instance, a medical AI system might analyse a patient’s scan while simultaneously reading their clinical notes, providing more accurate and context-aware insights.

Ethics, transparency, and accountability remain critical. Explainable AI (XAI) techniques help humans understand why a model made a particular decision, which is essential in fields like healthcare, finance, and law. Detecting bias, evaluating fairness, and ensuring that models behave responsibly are becoming standard parts of AI development.

Energy efficiency and sustainability are also priorities, as training large models consumes significant computational resources.

Ultimately, the future of AI will be shaped by models that are not only more capable but also more efficient, interpretable, and responsible.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!