Understanding the error of this trained Neural Network

[Rolodex]

I just trained a Neural Network with:

> nn <- neuralnet(score ~ ., data = ds_trn_noc, hidden = 2, err.fct = "sse")

and this was the result:

> plot(nn, rep = "best")

> nn$result.matrix
##                                         [,1]
## error                           1.988157e+02
## reached.threshold               9.903357e-03
## steps                           4.576900e+04
## Intercept.to.1layhid1           1.111093e+00
## gender2.to.1layhid1            -6.918958e-01
## age.to.1layhid1                -2.555627e-01
## pointsearned.to.1layhid1        1.790195e+00
## sponsorPlayStation.to.1layhid1 -1.925275e-01
## servernode37.to.1layhid1       -8.135250e-01
## servernode38.to.1layhid1        1.419319e+00
## servernode39.to.1layhid1        3.581786e+01
## servernode40.to.1layhid1        4.036410e-01
## servernode41.to.1layhid1        6.097005e-01
## servernode42.to.1layhid1        3.528988e+01
## servernode43.to.1layhid1       -1.305064e+00
## servernode44.to.1layhid1       -1.108396e-01
## servernode45.to.1layhid1        4.300516e-01
## servernode46.to.1layhid1        1.189133e+00
## servernode47.to.1layhid1       -8.662055e-02
## servernode48.to.1layhid1       -2.005513e+00
## servernode49.to.1layhid1        1.963471e+00
## distancefromserver.to.1layhid1  1.819308e-02
## Intercept.to.1layhid2           3.140212e+01
## gender2.to.1layhid2            -3.991657e+00
## age.to.1layhid2                 6.927878e+01
## pointsearned.to.1layhid2       -2.554987e+02
## sponsorPlayStation.to.1layhid2  3.364711e+01
## servernode37.to.1layhid2       -2.002457e+00
## servernode38.to.1layhid2        9.290339e+01
## servernode39.to.1layhid2       -3.948298e+01
## servernode40.to.1layhid2        5.419248e+01
## servernode41.to.1layhid2        3.549839e+01
## servernode42.to.1layhid2       -2.105324e+01
## servernode43.to.1layhid2       -8.174286e-01
## servernode44.to.1layhid2       -1.054152e+02
## servernode45.to.1layhid2        4.318452e+01
## servernode46.to.1layhid2        1.075459e+02
## servernode47.to.1layhid2       -2.244469e+00
## servernode48.to.1layhid2        4.064927e-01
## servernode49.to.1layhid2        3.412306e+01
## distancefromserver.to.1layhid2  6.846479e+02
## Intercept.to.score             -3.693540e-01
## 1layhid1.to.score               3.675358e+00
## 1layhid2.to.score               1.120348e+00

I have some questions about this:

1. Do you know where can I find any documentation about the output parameters: { error, reached.threshold, steps }? I didn't find anything useful on internet.

2. How is the error above calculated?, on this case: 1.988157e+02 = 198.82

3. Is there any general rule about how to know when an error is fine enough to use it on Production?

Thanks!

[jcblum]

@Rolodex

• Do you know where can I find any documentation about the output parameters: { error, reached.threshold, steps } ? I didn't find anything useful on internet.
• How is the error above calculated?, on this case: 1.988157e+02 = 198.82

One place to start might be the R Journal article that accompanied the original release of the package:

RJournal_2010-1_Guenther+Fritsch.pdf

(note that some of the syntax has changed in the intervening years, e.g., you no longer use compute() to get predictions, but can just use the R standard predict()).

[Rolodex]

@jcblum, do you know if for the parameter: err.fct can I use something like: mae or rmse instead of sse?

[jcblum]

I'm not terribly familiar with this package, but based on the documentation, your only pre-defined options are "sse" and "ce". However, you can also supply a custom function. There's an example of doing this for act.fct in the docs, which you could use as a guide.

[Rolodex]

thank you @jcblum, that helped!