CS 587, Introduction into Neural Networks

Assignment I, Due date: Wednesday, February 9, 2005

1. (5 pts) Delta rule training.

Given the Delta rule: with learning rate and starting from zero initial weights find a set of final weights for the AND-function. Use a hard threshold activation function centered at 0. The four patterns are:

x₁ x₂ bias target

1: 1 1 1 1

2: 1 0 1 -1

3: 0 1 1 -1

4: 0 0 1 -1

Show the equation of the separator in the two-dim subspace (x₁, x₂) after each epoch and also show a drawing of this separator in pattern space.

2. (45 pts) A simple one-layer network for character recognition

Implement a single layer neural network for character recognition. Use the character representation as discussed in chapter 3; you will find the training and test set for this assignment on my home page http://cs.uccs.edu/~mfa/.

a. Use a hard threshold activation function and the Hebb learning rule to train the network. Use both a bi-polar representation for the characters and targets. Explicitly show results for each test pattern.

b. Use a soft threshold function and implement the (extended) Delta learning rule for training. Again, use a bi-polar representation and vary the learning rate in this experiment.

Hand in: A hard copy of your code

A detailed report of your experiments and findings. Organize your results in the form of Tables. Compare hard and soft threshold learning. Investigate the influence of the magnitude of the learning parameter. Explain your observations.