# Deep Learning with Javascript 3.2.2-3.2.2

Posted on April 10, 2020

Notes on chapter three of Deep Learning with Javascript

### 3.2.2 Precision Recall Accuracy and ROC Curves of Binary Classification

Confusion Matrix

##### Key ↘︎
The four result types
in classification
prediction prediction
positive negative
truth positive true positive false negative
truth negative false positive true negative
##### Example ↘︎
The four result types
in classification
prediction prediction
positive negative
truth positive 30 28
truth negative 40 02

Accuracy

We can understand accuracy from the table above as the in the following steps:

• `(30 + 02)/(30 + 02 + 40 + 28)`
• `(32)/(100)`
• `0.32`
• `32.00%`

``````/**
Accuracy:
=> ((#TP + #TN) / (#examples))
=> ((#TP + #TN) / (#TP + #TN + #FP + #FN))
**/
;(30 + 02) / (30 + 02 + 40 + 28)``````

Uneven distributions can make accuracy a misleading metric in vbinary classification.

Precision and Recall address this shortcoming in using accuracy to evaluate classification models.

Precision

The ratio of positive predictions made by the model that are actually positive.

• `(30 )/(30 + 40)`
• `30 / 70`
• `0.42857142857142855` or `42.86%`

While this is an imporovement we can still game even this metric by intentionally setting a conservative policy on emmiting positive predictions.

Recall

The ratio of actual positive examples that are classified as positive.

• `(30 )/(30 + 28)`
• `30 / 58`
• `0.5172413793103449` or `51.72%`

To game this metric one can declare all examples positive and score 100 recall.

Precision and recall is about tuning the model.

We tune the model to:

• find at least X% of positives, Precision at X% Recall
• adjust tricky places where there is uncertainty about the correct answer warrant this kind of analysis.

The threshold value applied to the sigmoid output doesnt necessarily have to be 0.5 by setting it lower it can allow the model to be more liberal in applying positive outputs (higher recall lower precision). Setting thresholds higher can lead to higher precision and lower recall by being more cautious in applying positive labels.

### 3.2.3 The ROC Curve - Tradeoffs in Binary Classification

• `FPR = #FP / (#FP + #TN)`
• `TPR = #TP / ( #TP + #FN) = recall`