Leverages, Outliers and the Performance of Robust Regression Estimators

Abstract

This study compared the performance of some robust regression methods and the Ordinary Least Squares Estimator (OLSE). The estimators were compared using varied levels of leverages and vertical outliers in the predictors and the dependent variables. An anthropometric dataset on total body fat with height, Body Mass Index (BMI), Triceps Skin-fold(TS), and arm fat as percent composition of the body (parmfat), as the predictors. The effects of outliers and leverages on the estimators, were investigated at (5% leverages and 10% vertical outliers, 5% leverages with 15% vertical outliers). The criteria for the comparison: coefficients, Root Mean Square Error (RMSE), Relative Efficiencies (RE), coefficients of determination (R-squared) and power of the test. The findings from this study revealed that, OLSE was affected by both outliers and leverages whilst Huber Maximum likelihood Estimator (HME) was affected by leverages. The Least Trimmed Squares Estimator (LTSE) was slightly affected by high perturbations of outliers and leverages. The study also showed that Modified Maximum likelihood Estimator (MME) and S Estimator (SE) were robust to all levels of outliers and leverage perturbations. Therefore leverages and outliers in datasets do affect the post hoc power analysis of the methods which cannot resist them.