Building Softheon’s Health Insurance Predictive Model

Softheon developed a classification model to predict subscribers likely to experience a lapse in coverage due to nonpayment based on data collected from the many health insurance carriers we work with. The SQL query pulled subscriber, payment, and grace period data from the first 3 months of 2021. Softheon trained the model with data for “termed” enrollees pulled from the entire 2021 plan year.

To produce a predictive model with a high accuracy, Softheon focused on feature engineering. The model featured several new variables from the raw data including, but not limited to:

• Premium Amount (policy rate- APTC amount)
• APTC Percent (APTC amount/policy rate)
• Premium Amount per Person (premium/Number of members)
• APTC Group (APTC amount over 0 or not)

In addition, Softheon transformed some of the quantitative variables into categorical variables:

• Coverage (coverage over four months)
• Number of Payments (over four payments)
• Premium per Person (over $400)
• Months in Grace Period (over one month)
• Grace Period Entries (over three entries)

Review Softheon’s ACA Predictive Analytics Report for further information on model uses and benefits to health plans.

Table of Contents

1. Examining the Coverage Lapse Model’s Performance
2. Identifying Final Predictors
3. Conducting Additional Testing and Recommendations
4. Softheon’s APTC Data Analysis and Visualization
5. Appendix: Key Terms Used to Describe the Model’s Data

Examining the Coverage Lapse Model’s Performance

The Coverage Lapse Predictive Model predicted terminations at a FI score of.48, tripling performance compared to simply predicting that everyone would terminate their insurance. By considering ~20% of a health plans enrollment data, Softheon can accurately identify 67% of the people who will lapse due to nonpayment.

Metrics of the Coverage Lapse Model include:

• F1 score: 48%
• Accuracy: 84.1%
• Recall: 66.5%
• Precision: 37.8%

Review Appendix: Key Terms Used to Describe the Model’s Data for additional information regarding these terms.

Identifying Final Predictors

Each individual feature in the model has a level of importance, and some are more important than others in making coverage lapse predictions. Review the provided graph for the relative importance of the top features in the final model:

Final Predictors in Relation to Significance

Feature Redundancy: Correlation Matrix

Production of a correlation matrix ensured that the final predictors were not collinear; the predictors were not significantly correlated with each other. The confirmation of unrelated factors prevented the model from double counting a predictor when determining at-risk enrollees. The correlation matrix is featured below:

Correlation Matrix

Based on the correlation matrix, the final predictors represented in the model included:

Model Selection and Tuning

The model selected for testing was a weighted logistic regression. Weighted logistical regression accounted for the imbalanced data-a very high percentage of subscribers did not term and Softheon wanted to predict the minority. First, the data was classified into a training set and a testing set. The training set consisted of 75% of the data; the percentage of those who termed was equally distributed between training and testing.

Scaling and Normalizing

Softheon scaled final predictors using a Standard Scaler function. Standard Scaler standardizes a variable by subtracting the mean and then scaling to unit variance. This results in a distribution with a standard deviation equal to 1 and a mean of 0. Therefore, about 68% of the values will lie be between -1 and 1.

The Logistic Regression Model

Softheon used a weighted logistic regression with ‘balanced weights’, but this also allows the flexibility to control weights. For instance, the base model performed well in terms of accuracy, but still missed some of the termed cases; Adjusting the weights allows us to decide the level of importance of catching a termination vs. preventing ‘false positives’.

These decisions allow for the maximization of the final model metrics, with an emphasis on F1 and Recall.

Conducting Additional Testing and Recommendations

Data from 2021 was pulled for another insurance carrier that the model had not been trained on and used it to predict those cases. The overall lapse rate for our data is 14%. The Coverage Lapse Model increases the F1 score to 58%, essentially tripling the effectiveness of the predictions. In addition, carriers will only need to look at approximately 22-25% of their member data (for interventions) to identify over 60% of coverage lapses.

Train a model for each insurance carrier with multiple years’ worth of data (or as much as available). Run the predictions on members in April of each year to find out which members to reach out to for preventing coverage lapse due to nonpayment.

Softheon’s APTC Data Analysis and Visualization

Before building the Coverage Lapse Predictive Model, Softheon examined the open-source data U.S. Centers for Medicare & Medicaid Services (CMS) during the 2021 Open Enrollment Period. This analysis precedes the Termination Predictive Model and established the need for such identification efforts.

Based on this data, Softheon examined how many of those consumers received APTC and how that impacted their average monthly premium.

Average Premiums Before and After APTC

At first, this chart looks intuitive – as the metal tiers increase so do premiums (as seen in the light blue). In addition, the gold metal plan has the lowest percentage of APTC received; however, closer consideration was needed when examining average premiums for silver and bronze enrollees.

Silver metal plans include the most consumers with APTC (over 95%). What’s more, after adjusting the premium amount, the silver metal premium is slightly lower on average than the bronze premium amount.

This data suggests that APTC is allowing some consumers to obtain a higher level of health care plan than they would normally be able to afford on their own. With a bronze plan, the insurance pays 60% of the healthcare costs, whereas with silver they pay 70% of the costs. This translates to higher premiums for silver plans, which could deter some people from affording this level – that is, without APTC.

Appendix: Key Terms Used to Describe the Model’s Data

Accuracy: Ratio of correctly predicted observation to the total observations

Accuracy = TP+TN/TP+FP+FN+TN

Recall: Ratio of correctly predicted positive observations to all observations in actual class

Recall = TP/TP+FN

Precision: Ratio of correctly predicted positive observations to the total predicted positive observations

Precision = TP/TP+FP

 F1 Score: Weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. Intuitively it is not as easy to understand as accuracy, but F1 is usually more useful than accuracy, especially if you have an uneven class distribution.

F1 Score = 2*(Recall * Precision) / (Recall + Precision)

Download Softheon’s ACA Predictive Analytics Report to learn about the most influential factors for identifying members at-risk for coverage lapse and suggested retention strategies.