Introduction to Perceptron

Perceptrons: The Building Blocks of Artificial Intelligence

Have you ever wondered how computers learn to recognize patterns, make decisions, or even play games? It all started with a simple idea called the perceptron. What is a Perceptron? A perceptron is the most basic building block of artificial intelligence. Invented by Frank Rosenblatt in 1957, it's like a single brain cell (neuron) that makes simple yes/no decisions. Back in the 1950s, when computers were enormous machines filling entire rooms, Rosenblatt created the perceptron algorithm. As an article from Cornell University describes it: "An IBM 704 - a 5-ton computer the size of a room - was fed a series of punch cards. After 50 trials, the computer taught itself to distinguish cards marked on the left from cards marked on the right." This might seem simple today, but it was revolutionary at the time - a machine that could actually learn!

How Does a Perceptron Work?

Let's break down how a perceptron works in a way that's easy to understand:

The Perceptron's Job: Classification

A perceptron's job is to classify data into two categories - think of it as sorting items into two separate piles. For example, a perceptron might determine whether:

• An email is spam or not spam.
• A student will pass or fail a test.
• A concert is worth attending or not.

Let's take a look at a figure depicting the perceptron neuron:

• Note that $\sum$ means adding the inputs $[ 1,𝑥_1,...,𝑥𝑛 ]$ together.
• Formally, the unit's output activation is the weighted sum: $$\sum_{i=0}^{n} w_{i} x_i$$ where $w_{i}$ is the weight on the link from input $x_i$ to the neuron.
• In general, we can cast an activation function $g$ on the weighted sum as follows: $$g(\sum_{i=0}^{n} w_{i,j} x_i)$$ where $g$ can be an activation function, like the step function, sigmoid, tanh, ReLu, etc.

How It Makes Decisions

A perceptron makes decisions by:

1. Taking in several inputs (features)
2. Multiplying each input by a "weight" (how important that input is)
3. Adding all these weighted inputs together
4. Adding the bias term - $w_0$ or sometimes noted $b$
5. Outputting:

• 1 meaning "Yes" if the sum is greater 0
• 0 meaning "No" if it's less than than or equal to 0

A Simple Example: Should I Go to the Concert?

Imagine you're deciding whether to attend a concert. You might consider:

• How much you like the band (Band Score)
• How far away the concert is (Distance Score)

Each factor has a certain importance to you (weight). Let's say:

• Band Score has a weight of 0.9 (very important to you)
• Distance Score has a weight of 0.5 (moderately important)
• Your bias is -0.5 (you're slightly inclined to stay home unless convinced otherwise)

Now, let's say you're considering a concert where:

• Band Score = 0.8 (you like them quite a bit)
• Distance Score = 0.5 (moderately close to you)

The perceptron calculates: $$(0.9 \times 0.8) + (0.0 \times 0.5) - 1.0 = 0.72 + 0 - 1.0 = -0.28$$

Since 0.47 > 0, the perceptron outputs 1

Yes, go to the concert! 💃🎶

The Learning Process

The real magic of perceptrons is that they can learn! Here's how it works with our concert example:

1. Start with random guesses about how important band quality and distance are (random weights and bias)
2. Consider past concerts you've attended or skipped
3. For each past concert, predict whether you should have gone based on your current weights
4. Compare your prediction with what actually happened.
5. If your prediction was wrong, adjust how much you care about band quality and distance using this formula: $$w_{new} = w_{old}+\alpha*y*x$$
6. Repeat this process until your predictions match your actual preferences

Imagine you initially thought band quality wasn't very important (low weight), but as you analyze your passed data you realize that you care way more about the band than the distance. The perceptron would gradually increase the weight for band quality until its predictions align with your true preferences and it can correctly classify your past concert data.

The learning rate, $\alpha$, determines how quickly you adjust your preferences. If you're stubborn (low learning rate), you'll make small adjustments. If you're more flexible (high learning rate), you'll make bigger changes based on each concert experience.