COMPARISON OF MACHINE LEARNING ALGORITHMS FOR DETECTING RARE DISEASES

Machine learning is a powerful tool for finding important trends in data. One area of application for machine learning is to detect a specific type of disease from changes in symptoms or lab results of a patient. In real life, data about a disease is unbalanced, because there will often be more data from healthy subjects, and there may be a limited amount of data for a rare disease. In this study, various machine learning algorithms were compared to find which one would be the most effective in classifying diseases across all types of prevalence of the disease, whether rare or common. Without any machine learning tuning, linear SVM, Naïve Bayes, and Neural Network models performed the best with an accuracy over 90% for each model. With further exploration of SVM classifiers by tuning the regularization parameter and classification function degree, the classification boundary function is best to be a polynomial with odd degree or just linear. The study found that the disease data was inherently a linear problem due to the degree 1 curve having the highest accuracy overall. An even polynomial did not have negative outputs, so it was therefore limited in its scope. Because our classification problem of disease progression required a hyperplane that divided data points between positive and negative changes, an SVM with an odd-degree polynomial might be necessary.