Assessing Predictive Accuracy Using Cross-Validation

Course video 7 of 9

This week you will perform some predictive analytics tasks, including classifying loans and predicting losses from defaulted loans. You will try a variety of tools and techniques this week, as the predictive accuracy of different tools can vary quite a bit. It is rarely the case that the default model produced by ASP is the best model possible. Therefore, it is important for you to tune the different models in order to improve the performance.This week’s assignments require you to build predictive models for both classification and regression tasks. <p> Before working on the assignments, you may review a few videos to remind yourself several important concepts, such as cross validation. These concepts are discussed in the videos Cross Validation and Confusion Matrix and Assessing Predictive Accuracy Using Cross-Validation. You may also find a refresher on XLMiner useful. The videos Building Logistic Regression Models using XLMiner and How to Build a Model using XLMiner discuss how to build logistic regression and linear regression models. Depending on your needs, you may also go back to the videos that discuss how to build trees and neural networks. </p>

The analytics process is a collection of interrelated activities that lead to better decisions and to a higher business performance. The capstone of this specialization is designed with the goal of allowing you to experience this process. The capstone project will take you from data to analysis and models, and ultimately to presentation of insights.
In this capstone project, you will analyze the data on financial loans to help with the investment decisions of an investment company. You will go through all typical steps of a data analytics project, including data understanding and cleanup, data analysis, and presentation of analytical results.
For the first week, the goal is to understand the data and prepare the data for analysis. As we discussed in this specialization, data preprocessing and cleanup is often the first step in data analytics projects. Needless to say, this step is crucial for the success of this project.
In the second week, you will perform some predictive analytics tasks, including classifying loans and predicting losses from defaulted loans. You will try a variety of tools and techniques this week, as the predictive accuracy of different tools can vary quite a bit. It is rarely the case that the default model produced by ASP is the best model possible. Therefore, it is important for you to tune the different models in order to improve the performance.
Beginning in the third week, we turn our attention to prescriptive analytics, where you will provide some concrete suggestions on how to allocate investment funds using analytics tools, including clustering and simulation based optimization. You will see that allocating funds wisely is crucial for the financial return of the investment portfolio.
In the last week, you are expected to present your analytics results to your clients. Since you will obtain many results in your project, it is important for you to judiciously choose what to include in your presentation. You are also expected to follow the principles we covered in the courses in preparing your presentation.