Is Machine Learning Magic?

Machine learning is not the panacea. When facing a problem with a huge amount of data, the first thing is to take some time to deeply understand the extent of the problem and what really is in the data. A model of Machine learning is a powerful tool to help you solve a problem but is rarely the answer. In many cases, a solution based only on pure Machine Learning will fail. In this article, we show some techniques to model a problem in a way that smartly combine a predictive Machine Learning model and a post-processing on the output of the model.

Introduction

 

The Machine Learning has developed very rapidly during the past years and its use has become widely popular. It is sometimes considered as a magic formula that solves all problems involving large volumes of data. The effectiveness of Machine Learning models is no longer to be demonstrated, but it is often forgotten that their performance is conditioned by a detailed understanding of the problem to be solved.

 

Some problems can be difficult to express in the form of a pure machine learning problem. In particular, it may be too costly / impossible to obtain labelled data, making supervised learning techniques obsolete. Unsupervised learning methods may also not be applicable to the problem at all. In this case, it may be advisable to rephrase the problem into an intermediate problem, often more general and for which Machine Learning techniques will be relevant and effective, then apply a post-processing step on the predictions of the model to give a solution to the problem.

Post-processing consists of applying some wisely chosen transformations to the predictions produced by a model. Post-processing is often overlooked, even though it is just as important as feature engineering or model development. Post-processing ensures that predictions are consistent and that the model’s prediction error remains reasonable. It also corrects biases inherent to a model. Intelligent post-processing can also automatically transform the predictions to make them more meaningful and usable.

Share This