An optimization problem has to be solved by adjusting the threshold and seeking the optimum in order to balance the trade-off between the decrease in revenue and a decrease in cost.

An optimization problem has to be solved by adjusting the threshold and seeking the optimum in order to balance the trade-off between the decrease in revenue and a decrease in cost.

Then by using the layout of the confusion matrix plotted in Figure 6, the four regions are divided as True Positive (TN), False Positive (FP), False Negative (FN) and True Negative (TN) if“Settled” is defined as https://badcreditloanshelp.net/payday-loans-tx/sherman/ positive and “Past Due” is defined as negative,. Aligned with all the confusion matrices plotted in Figure 5, TP may be the good loans hit, and FP could be the defaults missed. Our company is interested in both of these areas. To normalize the values, two widely used mathematical terms are defined: true rate that is positiveTPR) and False Positive Rate (FPR). Their equations are shown below:

In this application, TPR may be the hit price of great loans, also it represents the capacity of earning funds from loan interest; FPR is the rate that is missing of, plus it represents the probability of losing profits.

Receiver Operational Characteristic (ROC) bend is considered the most widely used plot to visualize the performance of a category model after all thresholds. In Figure 7 left, the ROC Curve associated with the Random Forest model is plotted. This plot really shows the partnership between TPR and FPR, where one always goes into the exact same way as one other, from 0 to at least one. a classification that is good would also have the ROC curve over the red standard, sitting by the “random classifier”. The region Under Curve (AUC) can be a metric for assessing the category model besides precision. The AUC associated with Random Forest model is 0.82 away from 1, which will be decent.

Although the ROC Curve demonstrably shows the partnership between TPR and FPR, the limit is an implicit adjustable. The optimization task cannot purely be done by the ROC Curve. Consequently, another measurement is introduced to add the limit adjustable, as plotted in Figure 7 right. Considering that the orange TPR represents the capacity of getting cash and FPR represents the opportunity of losing, the instinct is to find the limit that expands the gap between curves whenever you can. The sweet spot is around 0.7 in this case.

You can find restrictions for this approach: the FPR and TPR are ratios. Also we still cannot infer the exact values of the profit that different thresholds lead to though they are good at visualizing the impact of the classification threshold on making the prediction. Having said that, the FPR, TPR vs Threshold approach makes the presumption that the loans are equal (loan quantity, interest due, etc.), however they are really perhaps not. Individuals who default on loans could have a greater loan quantity and interest that require become repaid, also it adds uncertainties into the modeling outcomes.

Luckily for us, detail by detail loan amount and interest due are offered by the dataset itself.

The thing staying is to locate a solution to link these with the threshold and model predictions. It isn’t hard to determine a manifestation for revenue. These two terms can be calculated using 5 known variables as shown below in Table 2 by assuming the revenue is solely from the interest collected from the settled loans and the cost is solely from the total loan amount that customers default