Notes on chapter three of Deep Learning with Javascript

### 3.2.2 Precision Recall Accuracy and ROC Curves of Binary Classification

**Confusion Matrix**

##### Key ↘︎

The four result types in classification |
prediction | prediction | |
---|---|---|---|

positive | negative | ||

truth | positive | true positive | false negative |

truth | negative | false positive | true negative |

##### Example ↘︎

The four result types in classification |
prediction | prediction | |
---|---|---|---|

positive | negative | ||

truth | positive | 30 | 28 |

truth | negative | 40 | 02 |

**Accuracy**

*We can understand accuracy from the table above as the in the following steps:*

`(30 + 02)/(30 + 02 + 40 + 28)`

`(32)/(100)`

`0.32`

`32.00%`

```
/**
Accuracy:
=> ((#TP + #TN) / (#examples))
=> ((#TP + #TN) / (#TP + #TN + #FP + #FN))
**/
;(30 + 02) / (30 + 02 + 40 + 28)
```

Uneven distributions can make accuracy a misleading metric in vbinary classification.

*Precision* and *Recall* address this shortcoming in using accuracy to evaluate classification models.

**Precision**

*The ratio of positive predictions made by the model that are actually positive.*

`(30 )/(30 + 40)`

`30 / 70`

`0.42857142857142855`

or`42.86%`

While this is an imporovement we can still game even this metric by intentionally setting a conservative policy on emmiting positive predictions.

**Recall**

*The ratio of actual positive examples that are classified as positive.*

`(30 )/(30 + 28)`

`30 / 58`

`0.5172413793103449`

or`51.72%`

To game this metric one can declare all examples positive and score 100 recall.

Precision and recall is about tuning the model.

We tune the model to:

- find at least
*X%*of positives,*Precision at X% Recall* - adjust tricky places where there is uncertainty about the correct answer warrant this kind of analysis.

The threshold value applied to the sigmoid output doesnt necessarily have to be 0.5 by setting it lower it can allow the model to be more liberal in applying positive outputs (higher recall lower precision). Setting thresholds higher can lead to higher precision and lower recall by being more cautious in applying positive labels.

### 3.2.3 The ROC Curve - *Tradeoffs in Binary Classification*

* Receiver operating characteristic* or

*stems from early radar systems.*

**ROC**- The horizontal access of an roc curve is the false positive rate
`FPR = #FP / (#FP + #TN)`

- the ratio of actually negative examples to the number false positives

- The vertical axis is the true positive rate.
`TPR = #TP / ( #TP + #FN) = recall`

- this is the same definintion as recall