Web Simulation 

 

 

 

 

Perceptron I 

The perceptron is the simplest neural network unit: a weighted sum of inputs followed by an activation function. It is the building block from which every multi-layer network is constructed, and learning how a single perceptron classifies is the cleanest way to build intuition for what neural networks do at all scales.

This tutorial shows the perceptron computing a 2-input logic-gate output in real time. You can edit the two weights and the bias directly to see how the decision boundary moves, switch between activation functions to see how their curves shape the output, or hit Train to let the perceptron learn an AND/OR/NAND/NOR truth table on its own.

NOTE: For underlying theory, see this note.

Sections

Mathematical Foundation

A perceptron computes a weighted sum of its inputs plus a bias, then passes the result through an activation function:

y = f(Σi wi · xi + b) = f(w1x1 + w2x2 + b)

The decision boundary — the locus where the perceptron flips between output classes — is the line in input space where the weighted sum equals zero:

w1x1 + w2x2 + b = 0

So a 2-input perceptron can only draw straight-line decision boundaries. That's the central constraint that makes it solve some logic gates but not XOR.

Activation Functions

The activation f shapes the output. The simulator supports five common choices:

Function

Formula

Output range

Trainable by gradient?

Step

1 if z ≥ 0 else 0

{0, 1}

No (slope 0)

Sigmoid

1 / (1 + e−z)

(0, 1)

Yes

Tanh

(ez − e−z) / (ez + e−z)

(−1, +1)

Yes

ReLU

max(0, z)

[0, ∞)

Yes (slope 1 for z > 0)

Linear

z

(−∞, +∞)

Yes (slope 1)

Step is non-differentiable, so it can only be trained by the classical perceptron rule (not gradient descent). All others are trained by gradient descent.

Perceptron Learning Rule

For a step-activation perceptron, weights and bias are updated by:

wi ← wi + η · (target − output) · xi
b   ← b  + η · (target − output)

If the prediction is correct, the error is 0 and nothing changes. If it's wrong, weights move in proportion to the error and the input that produced it. Rosenblatt's convergence theorem guarantees this rule converges in finite steps for any linearly separable problem.

Which Gates Are Learnable

A 2-input perceptron can solve any 2-input Boolean function whose 1s and 0s can be separated by a single straight line:

Gate

Truth table

Linearly separable?

Solvable

AND

1 only at (1,1)

Yes

Yes

OR

1 except at (0,0)

Yes

Yes

NAND

0 only at (1,1)

Yes

Yes

NOR

0 except at (0,0)

Yes

Yes

XOR

1 at (0,1) and (1,0)

No

No (needs hidden layer)
Why XOR isn't here: the 1s of XOR sit at opposite corners of the unit square; any straight line separating them also splits one class in two. This is the foundational limitation that motivated multi-layer networks — see MLP I.

Simulation

The interactive simulator is below. Edit weights and bias directly with the sliders, toggle inputs to see the output, or pick a gate preset and hit Train to watch the perceptron find the right weights on its own.

 

Parameters

Followings are short descriptions on each parameters
  • Activation Function: Selects the transfer function f(Σ) used to convert the weighted sum into an output. For example it is set to Step, so the output becomes 0 or 1 based on whether Σ ≥ 0.
  • Gate Preset: Loads a predefined set of weights and bias to represent a logic gate behavior. For example it is set to AND Gate.
  • Bias (b): Adds a constant offset to the weighted sum. It shifts the decision boundary without changing the input values. For example b = 0.1.
  • Weight w1: Multiplier applied to input x1. It controls how strongly x1 affects the sum. For example w1 = 0.3.
  • Weight w2: Multiplier applied to input x2. It controls how strongly x2 affects the sum. For example w2 = 0.3.
  • Input x1: Binary input value for the first input node. The checkbox controls 0 (unchecked) or 1 (checked). For example x1 = 0.
  • Input x2: Binary input value for the second input node. The checkbox controls 0 (unchecked) or 1 (checked). For example x2 = 0.
  • Learning Rate: Step size used when updating weights and bias during training. Larger values change parameters faster. For examplet it is 0.10.
  • Train Update Speed (sec): Controls how often one training step runs. Smaller values make training update more frequently. For example it is 0.5 sec.

Buttons

Followings are short descriptions on each Button
  • Reset: Stops training (if running) and restores the simulation to a known starting state. If a gate preset is selected, it reloads that gate's default weights, bias, activation, and the initial input pattern. If no preset is selected (manual mode), it resets to the default manual values.
  • Randomize: Stops training (if running) and assigns random values to the weights and bias. It also randomizes the binary inputs (0 or 1). It keeps the currently selected activation function and it keeps the selected gate preset as-is.
  • Train: Starts the training loop using the selected gate's truth table as training data. It updates weights and bias repeatedly based on the learning rate and the update speed. It stops automatically when all truth-table cases are classified correctly, or it can be stopped manually by pressing the button again.
  • Test: Stops training (if running) and try all the possible input combinations. If all the input combinations lead to Decision 1(pass), it pops up 'Test PASS'