While I’m certainly not an expert on machine learning, I have had to use it on several occasions to optimize some of my projects. I decided to write an article explaining what this technique can (and can’t) do based on my (limited) experience. I’ll also introduce a practical example of a machine learning problem solved using Ruby on Rails.
First, machine learning is a subfield of data analytics and its purpose is to give computers the ability to learn from experience and predict outcomes. In practice, you take data from the past and feed it into algorithms, which – if correctly engineered – will be able to detect counterintuitive patterns and/or predict future events.
Machine learning has been around for many years but has recently been made accessible thanks to the increase in available data, affordable processing power, and inexpensive storage.
Applications are found in different fields:
- The health care sector is using it to help diagnose diseases.
- Art historians are using it to detect influences between artists.
- Economists are using it to predict economic movements.
- Traders are using it to predict share value fluctuations.
- Email services use it to detect spam.
- Banks and governments are using it for fraud detection.
- Financial services are using it to identify investment opportunities.
- Companies such as Amazon and Netflix are using it for their recommendation engines.
- Google, Tesla, Uber, etc, are using it in their self-driving cars.
- Postal services use it for handwriting recognition.
- Oil and gas industries use it to find new energy sources, streamline oil distribution, predict refinery sensor failure, etc.
- Transportation institutions are using it to make routes more efficient, predict potential problems and increase profitability.
- Marketers are using it for recommending items you might like.
- Companies are using it to analyse facial expressions and emotions in order to improve the customer experience.
Big data vs artificial intelligence vs machine learning vs deep learning
Big data is the data sets involved in the science of extracting meaningful insight from a large set of data. It is particularly useful in depicting patterns that are not visible otherwise. Big data applies techniques from different areas, including machine learning and deep learning.
The term “artificial intelligence” has been around since the first computers were available – in the 1950s. This was probably the first attempt to describe machines that possessed the same characteristics as human intelligence.
Machine learning came around later and can be described as algorithms that are able to parse data, learn from them and predict some kind of outcome. So, instead of having hard-coded routines with specific instructions to emulate the human brain, we now have machines that are able to learn.
Deep learning can be seen as a branch of machine learning in which a set of algorithms attempts to model high-level abstraction of data. It is inspired by the neuronal network and the interconnections between the neurons in a human brain. It has been used for image recognition. Deep learning requires a lot of initial data and processing power – which explains why it has evolved only recently with the increase of computer power.
Popular machine learning methods
Machine learning can be divided into:
- Supervised learning
In supervised learning, algorithms are trained with a set of data that have been pre-labeled in different classes. For example, an article can be about “Football” or “Volleyball”. After the system has been trained, classification algorithms are used to classify unclassified data. Some well-known classifiers are:
- Latent Semantic Indexing
- Neural networks
- Logistic regression
- Decision trees
- Unsupervised learning
This method uses data that have no historical labels, which means that the system is not instructed with the “right” answer. The algorithms will provide suggestions about what the best classes can be.
- Semi-supervised learning
This is basically a combination of both supervised learning – where we feed an algorithm with the “right” answers – and unsupervised learning – where we let the algorithm propose classes.
- Reinforcement learning
In this configuration, the algorithm learns through trial and error which action yields the best outcome. It is often used in robotics and gaming.