,

ISBN:, Edition: ,

Price: -

Table of Contents

Preface 9

I FUNDAMENTALS 11

1 Introduction 13

2 Fundamentals 15
2.1 A framework for distributed representation 15
2.1.1 Processing units 15
2.1.2 Connections between units 16
2.1.3 Activation and output rules 16
2.2 Network topologies 17
2.3 Training of artificial neural networks 18
2.3.1 Paradigms of learning 18
2.3.2 Modifying patterns of connectivity 18
2.4 Notation and terminology 18
2.4.1 Notation 19
2.4.2 Terminology 19

II THEORY 21

3 Perceptron and Adaline 23
3.1 Networks with threshold activation functions 23
3.2 Perceptron learning rule and convergence theorem 24
3.2.1 Example of the Perceptron learning rule 25
3.2.2 Convergence theorem 25
3.2.3 The original Perceptron 26
3.3 The adaptive linear element (Adaline)27
3.4 Networks with linear activation functions: the delta rule 28
3.5 Exclusive-OR problem 29
3.6 Multi-layer perceptrons can do everything 30
3.7 Conclusions 31

4 Back-Propagation 33
4.1 Multi-layer feed-forward networks 33
4.2 The generalised delta rule 33
4.2.1 Understanding back-propagation 35
4.3 Working with back-propagation 36
4.4 An example 37
4.6 Deficiencies of back-propagation 39
4.7 Advanced algorithms 40
4.8 How good are multi-layer feed-forward networks? 42
4.8.1 The effect of the number of learning samples 43
4.8.2 The effect of the number of hidden units 44
4.9 Applications 45

5 Recurrent Networks 47
5.1 The generalised delta-rule in recurrent networks 47
5.1.1 The Jordan network 48
5.1.2 The Elman network 48
5.1.3 Back-propagation in fully recurrent networks 50
5.2 The Hopfield network 50
5.2.1 Description 50
5.2.2 Hopfield network as associative memory 52
5.2.3 Neurons with graded response 52
5.3 Boltzmann machines 54

6 Self-Organising Networks 57
6.1 Competitive learning 57
6.1.1 Clustering 57
6.1.2 Vector quantisation 61
6.2 Kohonen network 64
6.3 Principal component networks 66
6.3.1 Introduction 66
6.3.2 Normalised Hebbian rule 67
6.3.3 Principal component extractor 68
6.3.4 More eigenvectors 69
6.4 Adaptive resonance theory 69
6.4.1 Background: Adaptive resonance theory 69
6.4.2 ART1: The simplified neural network model 70
6.4.3 ART1: The original model 72

7 Reinforcement learning 75
7.1 The critic 75
7.2 The controller network 76
7.3 Barto's approach: the ASE-ACE combination 77
7.3.1 Associative search 77
7.3.2 Adaptive critic 78
7.3.3 The cart-pole system 79
7.4 Reinforcement learning versus optimal control 80

III APPLICATIONS 83

8 Robot Control 85
8.1 End-effector positioning 86
8.1.1 Camera{robot coordination is function approximation 87
8.2 Robot arm dynamics 91
8.3 Mobile robots 94
8.3.1 Model based navigation 94
8.3.2 Sensor based control 95

9 Vision 97
9.1 Introduction 97
9.2 Feed-forward types of networks 97
9.3 Self-organising networks for image compression 98
9.3.1 Back-propagation 99
9.3.2 Linear networks 99
9.3.3 Principal components as features 99
9.4 The cognitron and neocognitron 100
9.4.1 Description of the cells 100
9.4.2 Structure of the cognitron 101
9.4.3 Simulation results 102
9.5 Relaxation types of networks 103
9.5.1 Depth from stereo 103
9.5.2 Image restoration and image segmentation 105
9.5.3 Silicon retina 105

IV IMPLEMENTATIONS 107

10 General Purpose Hardware 111
10.1 The Connection Machine 112
10.1.1 Architecture 112
10.1.2 Applicability to neural networks 113
10.2 Systolic arrays 114

11 Dedicated Neuro-Hardware 115
11.1 General issues 115
11.1.1 Connectivity constraints 115
11.1.2 Analogue vs. digital 116
11.1.3 Optics 116
11.1.4 Learning vs. non-learning 117
11.2 Implementation examples 117
11.2.1 Carver Mead's silicon retina 117
11.2.2 LEP's LNeuro chip 119

References 123

Index 131