Machine Learning - What the Beginner Should Understand

In this article, we will try in an accessible form to answer the question: “Machine learning - what is it?”.

Content:
1. Cantonese Puzzle Ball
2. Choosing a definition
3. What is machine learning worth
4. Ways of learning
5. Machine learning in business

Machine learning has been at the top of the most popular and discussed technologies for several years now. In addition, it very quickly changes the lives of billions of people. But only a few specialists closely associated with machine learning clearly understand its essence.

Before explaining the essence of machine learning, let's find out its place and role in a group of closely related IT technologies.

Machine learning interacts very closely with artificial intelligence, neural networks and deep learning. But what is a subset of what?

Let me give you an example comparison. In Chinese art, there is such a spectacular artifact as the “Cantonese ball”. It is a carved ivory sphere, inside of which are several more spheres. Moreover, each freely rotates with respect to its neighbors.

Let's arrange the discussed technologies in the following order: artificial intelligence – machine learning – neural networks – deep learning.

The outer sphere of such an imaginary Cantonese ball will be artificial intelligence. Each subsequent technology in our list is, as it were, nested in the previous one. So, machine learning is a subset of artificial intelligence. Deep learning is a part of machine learning and neural networks are the basis for deep learning algorithms. Let me remind you that the simplest neural network consists of three layers – a data input layer, an output layer and a data processing layer (hidden layer) placed between them. If the neural network has more than one hidden layer, then it is called deep. And the process of training such a neural network is called deep learning.

Let's talk specifically about machine learning. Several definitions of this phenomenon are possible. First, machine learning is a field of information technology that uses a special kind of computer algorithms to copy the processes that occur in the human brain during learning. Of course, we are not talking about physical and chemical processes, but about a mathematical model implemented in a software and hardware complex.

If we talk about the reproduction of human learning, then this is nothing more than a figurative expression. In fact, we cannot yet compete with nature in this process. There is too little knowledge about how the brain works, the hardware base is too weak - even with all the obvious successes in the development and production of chips.

Let me give you another definition. Machine learning is the extraction of meaningful data by studying large amounts of information in detail using algorithms that predict unknown data based on known ones.

In other words, the computer sifts through the data provided to it, looks for patterns in them, and answers the question formulated by the person.

In the above interpretation, there are no links either to “artificial intelligence”, or to computer training, or to the work of the brain. The marketing and advertising halo disappears. Machine learning is becoming what it really is – just one of the areas of IT development, especially in demand in recent years. Why did it happen?

It turned out that initially programmed systems are not able to cope with the processing and analysis of colossal arrays of heterogeneous data. To curb the "information tsunami" new approaches are required. One of which is machine learning.

As in the view of the ancient sages, the flat Earth stood on three elephants, so machine learning is based on three components:

Let's talk briefly about each of them.

Data is what we will “feed” the computer to identify patterns and relationships. There is not much data. The more data - the better the result (forecast, classification, etc.). For example, the task is to track fraudulent SMS messages. Even a few tens of thousands of message samples are negligible to get started. Enough to train a beginner.

How to collect the required amount of data is a separate task that deserves a separate publication. I will only note that social networks are an invaluable source of data on a variety of issues - from taste preferences to attitudes towards world events. Hundreds of millions of posts, likes and comments - just a paradise for a machine learning specialist.

Well-chosen data arrays (datasets) for training have become a huge value, which their owners are very reluctant to disclose.

Let's move on to the features . You can call them differently - properties, parameters, characteristics. In relation to our example of creating a filter for SMS fraud, a feature can be the frequency of occurrence of a word in a message, the age of the recipient, or his marital status.

Let's remember the template phrases from such messages: “your card is blocked”, “you have been transferred XXX dollars”, etc. Age is quite a working feature. First, the elderly are more trusting. Secondly, let's remember the classic of the fraudulent genre: “Mom, I'm in the police. We urgently need money."

Picking up a set of features by which the computer will look for patterns is a very important and complex art. If you set more features "just in case" - the system will work unreasonably slowly. Throw out some important features - there is a risk of getting the wrong answer, missing some important patterns. Therefore, the selection of parameters takes a very large part of the time during machine learning.

Finally, the algorithm A lot depends on it. For example, the speed of work, the accuracy of the result. Often, different algorithms can be applied to the same problem. Here everything depends on the experience and knowledge of the person. But it is worth remembering that nothing good will come of low-quality datasets. With decent data selection, even a weaker, imperfect algorithm will give a more reliable result.

There are four ways of machine learning:

As in the previous chapter, we give a brief description of each method.

Classical machine learning is a proven, reliable tool. Helps out in a good half of situations where it makes sense to use machine learning. More than anything else, classical learning requires the intervention of Homo sapiens. It branches into two directions - learning with and without a teacher.

Method “with a teacher”: according to a predetermined set of features, some external mind explains to the computer that this SMS message is fraudulent, but this one is neutral.

Method “without a teacher”: the computer is given an array of data without any indication of features, and the machine itself must identify patterns. In our example with fraudulent SMS, the system should identify suspicious ones without human intervention.

Consider reinforcement learning. The Russian-language established name does not convey the essence of this method very clearly. Reinforcement learning refers to the fact that the system is “approved” for doing the right thing. and penalized for mistakes. Reinforcement learning is used in a situation where a computer needs not to analyze data, but to act in a constantly changing environment. A good example of such a situation is games and video games in particular. Or “adjusting the brains” of a robot vacuum cleaner. A more responsible and difficult task is the management of the resources of an industrial enterprise.

Sharing (ensemble) is to apply several algorithms at the same time and train them to find and correct each other's errors. Examples: train several algorithms, then pass their solutions as input to the last one. He will choose the best. Another option is sequential training of several algorithms. Each subsequent one works with moments where his predecessor made a mistake.

Neural networks and deep learning are a powerful and versatile method that can replace the others described above. I described the method of training the neural network in more detail in one of the publications. Here I will only mention that training a neural network consists in adjusting the strength of interaction between the elements that make up the neural network (neurons).

Machine learning has become an integral part of most businesses in recent years. This is a fact that does not require proof. More interesting is what approaches in machine learning are preferred in a particular area.

Thus, classical training has long and reliably been used in the fight against spam, fraud in the banking sector, text and objects recognition. Systems built on classical learning predict stock prices on stock exchanges, predict sales rates and fluctuations in demand. Unsupervised learning is used in marketing and retail to analyze purchased goods, arrange product items on shelves. It’s easier to say that retail can’t work without machine learning than to list specific examples.

Reinforcement learning was discussed above. So let's just repeat what we said. Beyond self-driving vehicles and video games, this approach has found application in enterprise resource management.

Collaborative work (ensemble) is able to work in all the areas mentioned, but with greater accuracy. In addition to them, she performed well in search engines, image recognition.

Neural networks and deep learning are a universal tool. It's just too powerful and expensive for general use. However, in certain areas they become simply irreplaceable. For example, in machine translation or speech recognition. Below is a list of illustrative examples of the use of machine learning.

Machine learning is not a fad in the information technology world. It's here forever. Owners and managers of businesses of all kinds should realize the inevitability of using machine learning. And here the most important thing is to realize the very possibilities of technology and choose the best tool for specific tasks.

Apix-Drive is a universal tool that will quickly streamline any workflow, freeing you from routine and possible financial losses. Try ApiX-Drive in action and see how useful it is for you personally. In the meantime, you are setting up connections between systems, think about where you are investing your free time, because now you will have much more of it.