Inside a Neural Network — Step by Step

A beginner-friendly workshop · Presented by Ouaell Ben Amara · Open Research Tunisia

Neural networks power everything from ChatGPT to face recognition — but what actually happens inside one? In this workshop we open the black box and follow a real prediction through a real network, number by number. No AI, math, or programming background required: if you can multiply and add, you can follow everything.

What we will cover

1. What is a neural network?

A machine that turns numbers into numbers — a photo into "cat, 97%", house data into a price. The key idea that makes it different from classical programming: we don't write the rules, we show it examples with answers and it discovers the rules itself.

2. The artificial neuron

The smallest piece of the machine. Inputs, weights (how much each input matters), the bias, and the weighted sum — computed by hand with real numbers: z = 0.5·2 + (−1)·3 + 1 = −1. Just multiply and add, nothing more.

3. The activation function

Why multiplying and adding can only draw straight lines, and why real problems are curvy. The sigmoid (squash any number into 0…1, a soft yes/no) and ReLU (the modern favourite) — the functions that bend the line.

4. From one neuron to a network

Neurons organised in layers: input, hidden, output. Fully connected networks, and counting the learnable "knobs" — our demo network has 13; ChatGPT-class models have billions. Same idea, bigger machine.

5. The forward pass — with real numbers

The heart of the workshop. A concrete task — will this student pass the exam? — computed completely by hand through a 2-3-1 network, every multiplication visible, ending in a prediction: 59% chance of passing.

6. The loss — measuring the mistake

The student actually passed, so how wrong were we? The error, why we square it, and how one single number scores the entire network: L = (1 − 0.59)² ≈ 0.17. Training = pushing that number toward zero.

7. Gradient descent — learning by rolling downhill

The loss as a valley in the fog: we can't see the bottom, but we can feel the slope under our feet. The update rule w ← w − α·slope, and what happens when the learning rate is too small (we crawl) or too big (we jump over the valley).

8. Backpropagation — who gets the blame?

The mistake appears at the output, but every weight helped cause it. How the error flows backward through the network, layer by layer, and how the chain rule ("multiply the ripples") gives the slope of every weight in one sweep.

9. The training loop

Everything chained together: forward → loss → backward → update, repeated thousands of times. Watching the loss fall and the prediction climb toward the truth. The demystifying conclusion: no magic — multiply, add, squash, measure, nudge, millions of times.

Hands-on: a neural network from scratch in C++

We finish by reading a complete, working neural network implemented from scratch in ~500 lines of heavily commented C++ — no libraries, no frameworks:

• Demo 1 re-computes the exact network from the slides, printing every intermediate value we calculated by hand.
• Demo 2 trains a network to solve XOR — a problem no straight line can solve — and watches it learn, live.

Participants leave with the full source code, the animated presentation, and suggested exercises (break the learning rate, shrink the network, remove the activation…) to experiment on their own.

Practical details

• Audience: complete beginners in computer science / AI
• Prerequisites: none — basic arithmetic only
• Duration: ~60–90 minutes (talk + code walkthrough + Q&A)
• Materials provided: animated presentation, presenter's example code (C++ / CMake), exercise list