Cloud Computing & Big Data

Author: Xinran Liu

Coauthor(s): Gang Luo, PHD

Status: Work In Progress

Can feature engineering with clinical input improve machine learning model accuracy and interpretability

Although machine learning (ML) holds great promise in predicting healthcare related outcomes such as patient costs, healthcare utilization, and clinical outcomes, current models suffer from poor accuracy, which limits their trust and usability. For example, models predicting hospital utilization by patients with asthma typically miss 75% of future inpatient or ED visits, models predicting future patient costs generally have R2 values less than 20%, and models predicting future patient outcomes often have AUROC values less than 0.8. One method to improve accuracy is using more advanced models such as deep learning. However, such models feel like black boxes, as it can be difficult to interpret why the model made a certain prediction. Such difficulty can limit trust in the model, as well as how useful the model can be (e.g. it would be important to know why a patient is at high risk for readmission, so that the patient can receive appropriate targeted interventions to decrease readmission risk).
Another potential method to improve healthcare ML model accuracy and interpretability that is not well explored is feature engineering using clinical input. A feature is a measurable quantity of an observable phenomenon that is used in a ML model. The features that are currently used in healthcare ML models are not very complex. For example, last, max, min, average, and count of number of values are features that are commonly used. HgbA1c is a blood test that can be used to track the severity of diabetes. It might be more important to look at the overall trend rather than the max, last, or average values. A patient with an upward sloping HgbA1c level might be sicker compared to a patient with a higher but stable HgbA1c level. Data scientists might not have the medical knowledge to create such features by themselves. We wish to study whether features engineered with clinical input can be used to improve ML model accuracy and interpretability.

To achieve our goal, we propose the following steps:
1) Using clinical input, create a novel list of ML features that could be used to predict future costs in diabetic patients
2) Using clinical input, create a novel list of ML features that could be used to predict future costs in asthma patients
3) Create a framework to conceptualize all possible features (akin to the differential diagnosis in medicine)
4) Test engineered features on dataset to see if there is an improvement in accuracy or interpretability in terms of predicting future costs.

Preliminary results
To date, we have a preliminary draft of 1). We also have a preliminary draft of 3). The framework for item 3) is currently designed as 4 dimensions:

1st dimension: feature type (continuous value, binary, multiclass).
2nd dimension: feature timeframe (e.g. 1 month, 3 month, 12 months, etc)
3A dimension: simple time representation (e.g. last, max, min, etc)
3B dimension: complex time representation (e.g. slope, trend, order, etc)
4th dimension: clinical concepts (e.g. severity of disease, appropriate management, social determinants of health, etc).