Once a candidate model is available, there our numerous questions one addresses.
How well does the model fit the observed data?
How well does the model apply to new data?
If one used a variable selection method, how stable is the model selected?
Introduction to some methods to examine the performance of a candidate model.
A review of goodness of fit.
Examining fit using covariate patterns.
The Hosmer-Lemeshow Goodness of fit statistic.
Calibration Plots, Part 1
Calibration Plots, Part 2
Decile Calibration Plots, Part 1
Decile Calibration Plots, Part 2
R-squared.
Examining External Validty.
Examining Model Stability
Review (or introduction), Bootstrap Sampling Distribution
PROC SURVEYSELECT to obtain bootstrap samples
A simple example, bootstrapping the standard error of the mean and obtaining 95% confidence intervals
Examining model stability when forward variable selection was used.
Homework
Examine the performance of your candidate model (from last week's assignment) on the lipid2018_b data set.
NOTE:
The submission for this homework should be a text document (word, rtf, plain text, etc.). It should in the form of a report and include the programs you used. In this report explain the methods used for examining model fit, external validity, and model stability and what you found.