Welcome :)

Workshop materials QR code Discord QR code

Wi-Fi

Name: FAC

Pass:

StochasticParrots27

Materials

Discord

Starting at 18:10

The Perceptron

The Mark I Perceptron

Cornell Aeronautical Laboratory, 1960

Mark I Perceptron machine with a large letter C in front of it, 1960

Character recognition.

An artificial neuron

A Logical Calculus of the Ideas Immanent in Nervous Activity, McCulloch and Pitts, 1943

A single neuron cell: weighted inputs enter, a sum feeds a threshold, output 0 or 1

A weighted sum of inputs and an activation: AND, OR, NOT.

Can you explain what this diagram is showing?

The activation function

The piecewise equation on the left, the step function graph on the right

Why is the activation function needed?

Learning from experience

The same neuron diagram The same neuron with the weights ringed, and a dashed arc labelled experience, with a question mark, sweeping from the output back to the weights

What would have to change for this neuron to learn?

The perceptron

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain, Frank Rosenblatt, 1958

A table of labelled examples feeds the neuron with a bias added, which produces labelled points in 2D separated by one line

Adjustable weights and bias, trained on labelled examples.

What can this machine actually learn?

Weights and bias

The boundary rotates in one panel and shifts without rotating in the other

Two parameters, two properties of the line.

Which part of the model controls which property of the line?

The prediction

Table of the four combinations of expected and prediction, error column shown as question marks The same table with the errors filled in: zero in green, minus one and plus one in red

Error = Expected − Prediction

What are the possible values of the error?

The update

An error causes the separating line to move toward a better position

What gets updated and in what direction?

The learning loop

The learning loop as six numbered steps with the update feeding back into the weighted sum, repeated per example; a full run through the data is an epoch

The XOR limit

XOR truth table and the four points plotted, no boundary drawn The same plot with three candidate straight lines, each failing

Where does the line go on this graph?

Layers

A two-layer network of step units, OR and NAND feeding AND, and the XOR points separated by the two hidden lines

Three units, two layers: OR and NAND, then AND.

Could the machine have learned these weights itself?

The perceptron today

A transformer block drawn as attention followed by an MLP, with the MLP highlighted

The multi-layer perceptron (MLP) inside a transformer block.

Whiteboard questions

  1. Draw a perceptron: inputs, weights, bias, weighted sum, activation function, and prediction.
  2. Draw a 2D classification problem: two feature axes, two classes, and a possible separating line.
  3. On the boundary, show what changing the weights does and what changing the bias does.
  4. Make an AND truth table, plot the four points, and draw a boundary.
  5. Make an XOR truth table, plot the four points, and try to draw a single boundary. Why does it fail?

Let's do it!

Workshop materials QR code

One more thing...

Machine Learning Apprenticeship

52 in-person workshops format
Fully funded funding
Peer-led discussion every session cohort
Projects tied to real work outcomes

Eligibility

Employed or own company employed
Resident for 3+ years residency
Apply QR code