Data mining Computerassignment 3

Data mining
‘Epochs & Accuracy’

~ Using the MultiLayerPerceptron function on a deliverd database ~

COMPUTER ASSIGNMENT 3

BARRY KOLLEE

10349863

Computerassignment 3

Exercise 1a): In the GUI screen you need to adjust the epochs (500 is too much). Start by
choosing 5. Select Start. Select Accept. The output screen gives you the accuracy of the
learned model.

I’ve loaded up the soybean.arff file into weka and did the following:
• Used the MultiLayerPerceptron classifier
• Set training set under test options
Set GUI on true

I performed the MultiLayerPerceptron with an Epoch of 5. The accuracy of an epoch of 5 (5 iterations) is
given in red.

Epoch of 5

Correctly Classified Instances 534 78.1845 %
Incorrectly Classified Instances 149 21.8155 %

Info about this evaluation

Epoch = 5
Error per Epoch = 0.0164204
Learning Rate = 0.3
Momentum = 0.2

Exercise 1b): Repeat the training with different settings (epochs 5, 10, 50 and 100) and see
what happens to the performance.

Epoch set to 10



Epoch = 10
Learning Rate = 0.3
Momentum = 0.2

Epoch set to 50



Epoch = 50
Learning Rate = 0.3
Momentum = 0.2

2


Epoch set to 100



Epoch = 100
Learning Rate = 0.3
Momentum = 0.2

Now we place all our findings of the different epochs into one table. Then it becomes clear which epoch
gives us the highest accuracy. The learning rate within the underneath table has been set to 0.3.

Number of Epochs/iterations Accuracy (number of correct classified instances)

5 78.1845 %
10 93.4114 %
50 98.6823 %
100 99.4143 %

It becomes clear that, when we increase the number of epochs (iterations), the accuracy climbs up also.
The increasing of the accuracy is mostly occurring when the epoch (number of iterations) is between 1
and 10. When we increase the number epochs to a value which is higher then 10, then there’s a
significant lower increase of the accuracy. The percentage of incorrect classified instances, when using
100 as the input value of Epoch, has been decreased to 0.5657 %. So that might be ‘negligbible’. In
conclusion we can state that using higher values for epoch wouldn’t be necessary with this learning rate.

If we also check the tables with ‘info about this evaluation’ we can also see that the error per epoch has
been decreased significantly when we increased the number of epoch(s).

3


Exercise 2: Pick one epoch value, and start playing with the "learning rate" parameter, e.g. try
0.1, 0.3, and 0.6. Again, look for each value what happens to the accuracy.

I’ve chosen for using an epoch value of 10 within the three outputs listed below. The accuracy (number
of correct and incorrect instances) is given in red.

Learning rate set to 0.1



Epoch = 10
Learning Rate = 0.1
Momentum = 0.2


Epoch set to 10



Epoch = 10
Learning Rate = 0.3
Momentum = 0.2




Epoch = 10
Learning Rate = 0.6
Momentum = 0.2

Now we place our findings in a table where we put the learning rate

Learning rate Accuracy (number of correct classified instances)
0.1 72.328 %
0.3 93.4114 %
0.6 93.4114 %

We see that increasing the learning rate from 0.1 to 0.3 has affected the accuracy in a good way.
However we see that if we would increase the learning rate it’s value even more that we won’t have a
higher accuracy. It might be interesting to take a look at a learning rate of 0.2 and check if the accuracy,
when using a learning rate of 0.2, has been affected if we compare it to a learning rate of 0.1.

4


The result of a learning set which has been set to 0.2 is listed below. The accuracy is given in red.



Epoch = 10
Learning Rate = 0.2
Momentum = 0.2

We see that the accuracy has climbed up significantly if we compare the learning rates of 0.1, 0.2 and
0.3.

learning rate 0.1 = 72.328 %
89.6047 / 72.3280 ≈ 1.24 increase factor

93.4114 / 89.6047 ≈ 1.04 increase factor

93.4114 / 89.6047 ≈ 1.29 increase factor

Another conclusion we can make is that the highest increasing factor is between a learning rate of 0.1
and 0.2. When increasing the learning rate any further is has a small effect on the accuracy of our model.

Another thing we can conclude is that the highest accuracy within this test has occurred when using a
1
learning rate of 0.3. We did observed a decrease of the error per epoch of 0.001 between the learning
rates 0.3 and 0.6, but that hasn’t affected the accuracy (number of correctly classified instances).

1
See ‘info about this evaluation’ of learing rate 0.3 & 0.6. Difference between these values.
2
Unfortunately Weka was not able to set the learning rate to a higher value then 1.0. There was a fixed maximum set

5


Exercise 3. As a final experiment, pick one value for both epoch and learning rate, and play
with the hiddenLayers (in the same window as where you set GUI to true). The default value
is 'a', try setting it differently (e.g. 5 or 10) and try multiple values at the same time (e.g. 10,10).
Again, look at performance of the model for various settings.

For the final experiment I’ve chosen to an epoch of 50 and a learning rate of 0.2.

These are the results when the hiddenLayers feature is set to it’s default value ‘a’. The accuracy is given
in red.



Epoch = 50
Learning Rate = 0.2
Momentum = 0.2
hiddenLayers = ‘a’

Now we set the value of hiddenLayers to 1. The output is listed below (and the accuracy in red).



Epoch = 50
Learning Rate = 0.2
Momentum = 0.2
hiddenLayers = ‘1’

Now we set the hiddenLayer option to 3. The accuracy is given in red again



Epoch = 50
Learning Rate = 0.2
Momentum = 0.2

Now we set the hiddenLayer option to 5. The accuracy is given in red again.



Epoch = 50
Learning Rate = 0.2
Momentum = 0.2

6


Now we set the hiddenLayer value to 10. The accuracy is given in red (again):



Epoch = 50
Learning Rate = 0.2
Momentum = 0.2

Now we set the hiddenLayers option to 20. The accuracy is given in red again.



Epoch = 50
Learning Rate = 0.2
Momentum = 0.2

Now we do a final test to see if there’s still a significant increase within the accuracy. We set the
hiddenLayers option to 30. The results are listed below with the accuracy in red.



Epoch = 50
Learning Rate = 0.2
Momentum = 0.2

We can now make a clear table where we note all accuracy’s weighted to the hiddenLayers it’s input
value. The underneath table shows us that the most significant increase of the accuracy is within 1 to 10
layers. After 20-30 hiddenLayers we have an almost perfect accuracy.

hiddenLayers value Accuracy (number of correct classified instances)

‘a’ (default) 98.5359 %
1 13.3236 %
3 59.7365 %
5 69.9854 %
10 90.0439 %
20 97.6574 %
30 98.3895 %

I’ve also visualized the outputs. It’s becoming clear that the more hiddenLayers we put in the more
connections can me made between the input attributes and the output attributes. We see that the red
dots represents the number of hiddenLayers which we have put in.

7


Epoch of 1 Epoch of 3 Epoch of 10 Epoch of 20 Epoch of 30

Now we test the same number of hiddenLayers but then with a learning rate of 0.3 and we leave the
epoch as it was (50 iterations). The accuracy of these output are listed below. If we compare these
numbers to the previous table we can state that they’re pretty similar and that it’s hard to say how they
differ.

hiddenLayers value Accuracy (number of correct classified instances)

‘a’ (default) 98.6823 %
1 24.0117 %
3 55.7833 %
5 68.6676 %
10 91.8009 %
20 98.0966 %
30 98.6823 %

Now we try different settings for the hiddenLayer feature. Instead of adding one integer we add two. This
will result in 2 layers of hiddenLayers. We leave the learning rate as 0.3 and use an epoch of 50 (50
iterations) as we have used in the previous examples also. We see that we get a ‘multi layer perceptron’.

Epoch set to 5,1 Epoch set to 1,5 Epoch set to 5,5

8


hiddenLayer value Accuracy (number of correct classified instances)

1,1 26.2079 %
5,1 32.0644 %
10,1 26.2079 %
1,5 30.7467 %
5,5 62.0791 %
10,5 64.1288 %
1,10 32.7965 %
5,10 65.593 %
10,10 71.5959 %

Now we do the same thing but we only adjust the learning rate to 0.6


1,1 26.2079 %
5,1 32.7965 %
10,1 39.6779 %
1,5 26.2079 %
5,5 74.0849 %
10,5 79.063 %
1,10 23.1332 %
5,10 77.5988 %
10,10 94.2899 %

Now we do the same thing but we only adjust the learning rate to 1.0. Unfortunately weka was unable to
set a higher learning rate then 1.0. So 1.0 is the heighest value which we could pick.


1,1 26.2079 %
5,1 38.9458 %
10,1 49.3411 %
1,5 28.1113 %
5,5 69.5461 %
10,5 80.9663 %
1,10 23.1332 %
5,10 76.2811 %
10,10 89.019 %

As a final test we want to know what the accuracy is we have a maximum value but below 1.0. So for
this final test we set the learning rate to a value of 0.9.


1,1 26.2079 %
5,1 27.0864 %
10,1 48.3163 %
1,5 28.5505 %
5,5 67.4963 %
10,5 82.4305 %
1,10 28.1113 %
5,10 73.3529 %
10,10 94.8755 %

If we observe the tables which are listed above it’s hard to say where our increase of accuracy is. What
we do see is that our maximum hiddenLayer gives us the highest accuracy. This counts for using both
2
the learning rates 0.6 and 0.9 (94,8755 %).

2
Unfortunately Weka was not able to set the learning rate to a higher value then 1.0. There was a fixed maximum set
of 1.0 withing the program

9

Data mining Computerassignment 3

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

Similaire à Data mining Computerassignment 3

Similaire à Data mining Computerassignment 3 (20)

Plus de BarryK88

Plus de BarryK88 (10)

Data mining Computerassignment 3