Why Simple Algorithms Are Not Enough for Complex Tasks
When One Step Was No Longer Enough
Take a simple task: determining if a number is even. A single rule is enough here. But what if the task is different: determining if there is a cat in a photo? There is no single rule for this. The condition «if the pixels show ears» doesn't work. «If there are whiskers» won't work on its own either. There are many features, they are diverse, they appear in different combinations, and none of them guarantees anything individually.
It is precisely for such tasks, where the result is determined not by a single condition but by a complex intertwining of many factors, that a fundamentally different approach was required. Neural networks became the answer not because they are «smarter» than other methods, but because their structure allows them to solve tasks of such complexity systematically, step by step.
How Neural Networks Process Numerical Data
Numbers In, Numbers Out
Let's start with the basics.
A neural network is a system that receives numbers at the input and produces numbers at the output. Everything that happens between the input and output consists of sequential mathematical data transformations.
There are no inherent meanings inside the system. There are only numbers and operations performed on them.
When we feed an image into a model, it is converted into a set of numbers – the brightness values of each pixel. When we input text, it is also encoded as numbers. Any type of data a neural network works with is represented in numerical form. This is not a metaphor or a simplification; it is literally what happens in the system.
The output consists of numbers as well. For instance, a number close to 1 might mean «yes, it's a cat», while a number close to 0 means «no, it's not a cat.» Or it could be a set of numbers, each corresponding to the probability of one of the possible answers. What exactly these results mean is determined by the person setting the task, not by the network itself.
The network doesn't know what a cat is. It doesn't know what «yes» and «no» are. It transforms an input set of numbers into an output set, guided by patterns that were captured during the training process.
Layers: Step-by-Step Transformation
The key idea of a neural network is not individual operations, but their sequence organized into layers.
Imagine an assembly line. At the first stage, raw materials undergo primary processing. At the second, the part becomes more complex. At the third, it takes its final shape. Each stage takes the result of the previous one and passes the processed data to the next.
Layers in a neural network function the same way.
The input layer receives the raw data – those very numbers into which pixels, words, or sounds are encoded. Then the data passes through several intermediate layers – commonly called hidden layers. Each of them takes information from the previous layer and performs its own transformation. The output layer forms the final result.
What exactly happens in each layer? Each element of a layer processes the data it receives and calculates a new number based on it. This number is passed further. As a result, at each stage, the initial data is «recalculated» into a new representation, which becomes the input for the next step.
Crucially, no single layer sees the «big picture.» Each one works only with what was delivered directly to it. The final conclusion does not arise at a single point; it becomes the cumulative result of the entire chain.
The Role of Weights in Neural Network Training
Weights: Where the Network's Learning is Stored
If layers are the structure, then weights are the content.
Every connection between elements of adjacent layers has a numerical coefficient. This is the weight. It determines how strongly one number influences the next. A large weight means the signal is amplified and has a noticeable impact. A small weight, or one close to zero, means the signal is almost ignored. A negative weight influences the result in the opposite direction.
It is in the weights that everything the network has learned is concentrated. When people say a model is «trained», it means its weights have been adjusted so that the network gives correct answers to tasks from the training set.
Before training, weights are usually set to random numbers. A network with such parameters produces nonsensical results. During training, weights are gradually corrected depending on how much the output data differed from what was expected. We discussed exactly how this process works in the articles «How AI Learns from Mistakes: The Feedback Mechanism» and «How a System Learns: Step by Step»; here, it is only important to understand that weights are not pre-set values, but the result of a long process of fine-tuning.
The number of weights in modern models is enormous – billions or even trillions of coefficients. This is exactly what makes them so flexible: with the right selection of weights, the same architecture can solve completely different tasks.
Benefits of Using Multiple Hidden Layers in Deep Learning
Why Depth Matters: Several Layers Are Better Than One
A natural question arises: why are several layers needed? Why not just use one?
A single transformation layer can only solve a specific class of tasks – those where the answer can be obtained through a simple linear operation on the input data. There aren't many such tasks. Most real-world situations are structured differently: data dependencies are non-linear, there are many variables, and they interact with each other in complex ways.
Each additional layer allows the network to build increasingly complex representations. A layer does not work with the raw data directly – it operates on what the previous level has «synthesized.» As a result, representations emerge in the deep layers of the network that are already far removed from the raw input data: they better reflect the structural patterns important for solving the task.
This is the key idea behind multi-layering: each level complicates and refines the representation, building on the results of the previous stage. A single layer cannot do this – it has nothing to «build» abstractions from. This is why deep networks (those with many layers) have proven capable of solving tasks that previously seemed fundamentally inaccessible to machines.
We will discuss in more detail what exactly changes when there are truly many such layers, and why this leads to a new quality of capabilities, in the article «Deep Learning: What Changes as Layers Increase».
Understanding the Internal Processes of a Neural Network
What Happens «Inside»
When first getting to know neural networks, there is often a feeling that something incomprehensible is happening inside. This impression is understandable, but not entirely correct.
Nothing mystical happens inside. Every element of every layer does the same thing: it takes a set of numbers, multiplies them by the corresponding weights, adds them up, and passes the result further. This is arithmetic. A huge volume of arithmetic operations performed in parallel and sequentially. No reasoning, intentions, or understanding of the essence of what is happening.
The difficulty lies not in the «mystery» of the process, but in the fact that tracking exactly why a specific combination of weights leads to a particular result is extremely difficult due to the enormous number of intermediate steps and coefficients. This is the problem of interpretability, and it is indeed a relevant issue. But it is fundamentally different from «magic»: the system does exactly what was built into it; it is just that the results of its work are hard to trace manually.
A neural network is not conscious of its actions. It doesn't know if it's being asked about a cat or a credit risk. It processes numbers according to the weights and outputs new numbers. The interpretation of this data always remains the responsibility of the human.
Summary: A Construction, Not a Mind
A neural network is an architecture of sequential numerical transformations. Input data passes through several layers, in each of which a new representation is calculated based on the previous one. The parameters of these calculations – the weights – are selected during the training process so that the result meets expectations.
Multi-layering allows the system to capture complex patterns inaccessible to simpler models. This is not the result of «understanding», but a consequence of the fact that each level of transformation adds a new level of abstraction to the data representation.
It is important to maintain this understanding: a neural network works with numbers and the patterns within them. It does not operate with meanings, form judgments, or know what the data refers to. Its capabilities are determined by its architecture and training data, not by an internal mind.