Automatic spam filtering, facial recognition, translation, and the display of recommended videos are all applications of machine learning. Machine learning is a technology in which a computer learns patterns from a large amount of data by itself, instead of having humans program rules one by one. It is currently a central method of AI (artificial intelligence). In this lesson, you will learn the three types of machine learning and important points to remember when using them.
Difference from Traditional Programming
In a traditional program, humans write all the rules: “If this happens, do that.” However, it is nearly impossible to write every rule for determining whether a cat appears in a photograph. In machine learning, a large number of cat photographs are provided so that the computer can automatically discover the features of “cat-like” images from the data. Instead of writing rules, we make the computer learn rules from data and answers—this is the change in approach.
Three Types of Learning
① Supervised learning: A method that learns from data with correct labels. It is given many correct answers, such as “this email is spam” and “this image is a cat,” so that it can predict answers for unknown data. Problems in which the answer is a category (spam/regular, cat/dog) are called classification, while problems in which the answer is a numerical value (tomorrow’s sales, the price of a house) are called regression. The regression line from the previous lesson can be regarded as the simplest form of supervised learning.
② Unsupervised learning: A method that finds hidden structures in data with no correct labels. A typical example is clustering, which automatically groups similar items together. For example, customers of an online store can be divided into several groups according to purchasing patterns, and different advertisements can be shown to each group.
③ Reinforcement learning: A method that learns the “action that maximizes rewards” through trial and error, using rewards instead of correct-answer data. It is used for Go AI, robot walking control, game AI, and other applications. The system is given only evaluations such as “+1 point for success and −1 point for failure” and discovers good actions by itself.
・Supervised learning ＝ Studying with a workbook that includes answers. By seeing many pairs of “problems and correct answers,” you become able to solve unfamiliar problems.
・Unsupervised learning ＝ Reading a large number of materials without answers and grouping them independently by deciding, “This material is similar to that material.”
・Reinforcement learning ＝ Club practice. The coach does not teach the correct movement but only says, “That was good” or “That was bad.” You improve through trial and error.
Training Data, Test Data, and Overfitting
In machine learning, it is essential to divide collected data into training data (used for learning) and test data (used to check performance). This is because scoring a model on the data used for learning does not reveal its true ability—it is like getting a perfect score on a workbook by memorizing the answers.
The phenomenon in which a model fits the training data too closely, achieving high scores on training data but reduced performance on unknown data, is called overfitting. It occurs because the model memorizes even minor coincidences and noise in the training data as “rules.” Countermeasures include increasing the amount of training data, simplifying the model, and monitoring performance with test data.
Points to Note When Using Machine Learning
Machine learning learns the tendencies of the data it is given as they are. If the data is biased, the system also learns that bias. For example, if past hiring data is biased, an AI that learns from it will repeat the same biased judgments. Another issue is that it can be difficult for humans to explain the basis of its decisions, making it a black box. Therefore, humans must verify AI output instead of accepting it blindly.
- Machine learning＝technology that automatically learns patterns from data. Rules are not written manually.
- Supervised learning uses labeled data (classification and regression). Unsupervised learning uses unlabeled data (such as clustering). Reinforcement learning uses trial and error based on rewards.
- Divide data into training data and test data. Do not evaluate a model using data that was used for learning.
- Overfitting＝the condition in which performance on unknown data declines because the model fits the training data too closely.
- Bias in data becomes bias in AI. Verifying output is a human responsibility.
Practice Problems
- For each task, state whether supervised learning, unsupervised learning, or reinforcement learning is appropriate. ① Classifying handwritten digit images as 0〜9 ② Dividing questionnaire respondents into groups with similar response patterns ③ Learning strong Othello moves by repeatedly playing matches
- Explain the difference between classification and regression in supervised learning, giving examples.
- Explain overfitting and state why data is divided into training data and test data to detect it.
Answers・Explanations
- Answer: ① Supervised learning ② Unsupervised learning ③ Reinforcement learning
Explanation: ① is a classification problem learned from labeled data such as “this image is 7.” ② is grouping without correct answers＝clustering. ③ uses trial and error based on wins and losses (rewards). - Answer: Classification is a problem in which the answer is a category (for example, determining whether an email is spam or regular mail). Regression is a problem in which the answer is a numerical value (for example, predicting the number of visitors tomorrow from past data).
Explanation: Remember that classification predicts “which group,” while regression predicts “how many” or “what value.” - Answer: Overfitting is the phenomenon in which a model memorizes even details and noise in the training data, achieving a high accuracy rate on the training data but reduced performance on unknown data. If evaluation is performed with the data used for learning, even memorization may receive a high score. Therefore, it is necessary to measure true ability—the ability to handle unknown data—using test data that was not used for learning.
Explanation: “Good performance on training data but poor performance on test data” is a sign of overfitting.