• People
• MatusUzak
• ANNvisualization
• FullViews

MLP Extrapolation Performance

Following images show responses of output neuron of a Multilayer Perceptron (MLP) network to data from visualization set. Visualization set includes all data from training set, testing set and data from outside this sets. That means the extrapolation is tested. The learning process of the 'spiral' and the 'square' task was studied. Different learning states can be observed dependent on weights initialization and number of neurons in hidden layers. Our research concentrates on states where teacher is able to decide whether to continue or to stop the training process based on his impression of the knowledge formation process.

A common practice in MLP training is to stop the undesirable training process, reinitialize weights and start training from scratch. Regarding the interactive training of artificial neural networks (INNT), there is one more possibility: some of the neuron responses can be modified in order to acquire better approximation. More information on INNT can be found in the papers section.

A part of recent research are the following images, created in the training process of MLP with a period of 5000 presentations of the training set. In compare to 'spiral' representation in previous images where networks of small topologies were studied (max 15 neurons in 1st and 10 in 2nd hidden layer) a more complex responses can be observed. However the networks are not able to build up the appropriate representation of 'spiral'. Sigmoidal activation function with steepness equal to 0.25 was used. This small steepness parameter enables the user to interact with the backpropagation learning algorithm during the whole training process, because it takes longer for a neuron to became saturated. Sorry for the white pixels in images, its a bug in the synchronization of threads.

After addition of a virtual sensor sending into the network a multiplication of the real two inputs, the learning performance rapidly increased.

Benchmark data

The process of benchmark data generation is following. A 2D picture of selected 2 class classification problem is mapped into a given region of feature space. That means each pixel is mapped to a 2D space and gets a class label (class0 - black color, class1 - white color). Selected percentage of generated data is randomly chosen into the training set. Visualization set includes all generated data and a given amount of data from outside their region.