VET News

Machine Learning Approaches Project Completion Rates for VET

This paper summarises exploratory analysis undertaken to evaluate the effectiveness of using machine learning approaches to calculate projected completion rates for vocational education and training (VET) programs. It compares this with the current approach used at the National Centre for Vocational Education Research (NCVER) — Markov chains methodology.

NCVER publishes annual observed VET qualification completion rates for qualifications that commenced four years prior to the most recent data collection period, based on the assumption that sufficient time has passed for all students who intended to complete their qualification to have done so. Projected rates are published for the more recent years, as the actual completion rates cannot be known until enough time has passed for the qualifications to be completed and the outcomes reported to NCVER.

While the Markov chains methodology currently used by NCVER has demonstrated that it is reliable, with predictions aligning well with the actual rates of completion for historical estimates, it has not been reviewed for some time, and it does have some limitations. The evaluation of machine learning techniques for predicting VET program completion rates was undertaken to overcome some of these limitations and with a view to improving our current predictions.

This report includes:

• an overview of the methodologies: Markov chains and two machine learning algorithms that were applied to predict completion rates for VET programs (XGBoost and CatBoost)
• a comparison of the accuracy of the predictions generated by both methodologies
• an evaluation of the relative strengths and limitations of both methodologies.
• For the 2016 commencing cohort, the completion rate predictions using machine learning algorithms were generally more accurate than the rates achieved using Markov chains methodology. When evaluated against actual published completion rates:

Download the full paper here.

Date posted Apr 27, 2023