Machine Learning and Deep learning

This document provides an overview of Machine Learning (ML) and Deep Learning (DL), including their definitions, differences, and applications. It also discusses the importance of features and targets in ML, as well as the challenges associated with image data. The document concludes with a brief introduction to the history of AI and its evolution into the current state of technology.

1. Introduction to Machine Learning

Machine Learning (ML) is a branch of Artificial Intelligence (AI) focused on developing algorithms that learn from data over time, rather than being explicitly programmed. By analysing data, ML algorithms identify patterns and improve their performance as more data becomes available. However, after a certain point, the performance gains from additional data diminish, leading to a plateau. Additionally, ML algorithms can be categorized into different types based on their learning approach, such as supervised, unsupervised, and reinforcement learning.

1.1. Learning from Data

ML programs learn by repeatedly seeing data, rather than following a set of rules programmed by humans. For example, in deciding whether emails are spam or not, a dataset with labelled emails (spam or not spam) is used. These emails are preprocessed and fed into an ML algorithm, which learns the patterns of spam versus not spam. The more emails it processes, the better it becomes at making accurate predictions.

1.2. Features and Target

Understanding features and targets is crucial in ML. Using the iris ¹ dataset as an example, the goal is to predict the species of a flower (target) based on features like sepal length, sepal width, petal length, and petal width. This dataset helps introduce basic ML concepts, although more complex datasets will be used as the course progresses.

1.3. Iris Dataset Overview

The Iris dataset is a classic dataset in machine learning, often used to demonstrate basic concepts. Below is a table summarizing the dataset’s features and target:

This dataset contains 150 samples, with 50 samples for each species: Setosa, Versicolor, and Virginica. It is widely used for classification tasks in machine learning.

In above example, what represent a feature and goal?

• Feature: Sepal length, sepal width, petal length, and petal width are the features used to predict the species of the flower.
• Goal: The goal is to predict the species of the flower based on the given features.

1.4. General Types of Learning

There are two main types of ML: supervised and unsupervised learning.

1. Supervised Learning: Uses labelled data to predict outcomes, such as classifying emails as spam or not spam. The goal is to predict the label accurately. Thus, it would have a feature column dataset in a data set along with a target(goal) column.
   • Example: Predicting whether an email is spam or not based on features like the sender, subject line, and content.
   • In supervised learning, the model learns from a training dataset with known labels and then applies that knowledge to new, unseen data.
2. Unsupervised Learning: Involves finding underlying structures in data without labels. For example, customer segmentation in marketing campaigns uses unsupervised learning to find similar groupings within the dataset. Thus, it would not have a preassigned feature column or target column.

• Example: Clustering news articles based on topics without predefined categories.

  • In unsupervised learning, the model identifies patterns and relationships in the data without any prior knowledge of the labels.

    Learning Type	Dataset	Goal (target)	Example
    Supervised Learning	Uses labelled data	Make prediction	Feature detection
    Unsupervised Learning	Doesn’t have labelled data	Find structure in the data	News article clustering

    {_style=primary}

1.5. Example of Supervised Learning: Fraud Detection

Fraud detection is a common ML problem where a large dataset with labelled transactions (fraud or not fraud) is used. In case of Credit card fraud detection, features like transaction time, amount, location, and category of purchase help predict whether new transactions are fraudulent or not. The model learns from the labelled data and applies that knowledge to new transactions, flagging potential fraud.

1.6. Example of Unsupervised Learning: Customer Segmentation

In customer segmentation, unsupervised learning finds similar groups within an e-commerce dataset to target them accordingly. There is no right or wrong answer`, and different models need to be tested to see which results make the most sense.

1.7. Challenges with Image Data

Defining features in images is significantly more complex compared to structured data. For instance, an image with dimensions of 256x256 pixels contains over 65,000 features. Treating each pixel as an independent feature disregards the spatial relationships between pixels, which are crucial for understanding the image. This is where deep learning excels, as it can automatically learn these spatial relationships and extract meaningful features.

1.7.1. Example: Identifying Dogs and Cats

Consider a dataset of images containing dogs and cats. The task is to classify each image as either a dog or a cat. A deep learning model learns to identify features such as fur texture, ear shape, and eye color. These features are then used to classify new, unseen images. However, before the model can make accurate predictions, it must be trained on a large dataset of labeled images, where each image is associated with its correct label (dog or cat).

1.7.2. How Does the Model Learn Features

Deep learning models use a process called backpropagation to learn features. During training, the model makes predictions and calculates the error by comparing its predictions to the actual labels in the dataset. It then adjusts its internal parameters to minimize this error. By iteratively updating these parameters, the model learns to recognize patterns and features that distinguish between different classes, such as dogs and cats.

This ability to automatically learn features from raw data makes deep learning particularly effective for image classification tasks, where manually defining features is challenging or impractical.

2. Deep Learning

Deep Learning is a subset of ML that uses complex models called deep neural networks. These models can determine the best representation of data. In classic ML, humans have to define features, but deep learning allows models to learn these features autonomously, which is especially useful for tasks like image classification.

2.1. Differences Between ML and Deep Learning

Classic ML requires defining features before feeding data into the model, while deep learning combines this process. Neural networks receive image pixels as input and learn to extract meaningful features by combining them in complex ways. Although these intermediate features may not always be interpretable, they are useful for tasks like image classification.

2.2. Classic Machine Learning vs. Deep Learning

2.2.1. Classic Machine Learning

In Classic Machine Learning (ML), recognizing a human face involves the following steps:

1. Feature Identification: Identify features like eyes, nose, and mouth by defining rules for recognizing these features.
2. Model Training: Feed these features into a pre-selected model (ML classifier algorithm). Train the model using these features and the corresponding labels (e.g., “Person A”, “Person B”).
3. Prediction: Once trained, the model predicts the class of a new image based on the defined features and rules.

2.2.2. Deep Learning

In Deep Learning, the same problem is approached differently:

1. Raw Data Input: Feed the raw image data (pixels) into a deep neural network without defining features.
2. Automatic Feature Extraction: The neural network automatically learns to extract relevant features from the images during training.
3. Model Training: The model learns to recognize faces by adjusting its internal parameters based on the training data.
4. Prediction: Predict the class of a new image based on the learned features and patterns.

This distinction highlights how Deep Learning eliminates the need for manual feature engineering, making it more effective for complex tasks like image recognition.

    flowchart TD
	  subgraph Classic_ML[Classic Machine Learning]
	    A1[Input Data] -->|Feature Extraction| B1[Manual Features]
	    B1 -->|Train Model| C1[ML Model]
	    C1 -->|Make Predictions| D1[Output]
	  end
	
	  subgraph Deep_Learning[Deep Learning]
	    A2[Input Data] -->|Raw Data| B2[Deep Neural Network]
	    B2 -->|Automatic Feature Extraction & Training| C2[DL Model]
	    C2 -->|Make Predictions| D2[Output]
	  end
	
	  style Classic_ML fill:#f9f,stroke:#333,stroke-width:2px
	  style Deep_Learning fill:#bbf,stroke:#333,stroke-width:2px

3. Conclusion

In conclusion, deep learning represents the cutting edge of ML and is the focus of most ML research today. It outperforms other algorithms on large datasets. However, for smaller datasets or data that changes frequently, traditional ML algorithms might perform better. AI and ML are rapidly evolving fields with significant real-world applications. Understanding the history, definitions, and impact of these technologies is crucial for anyone interested in the future of technology and its implications for various industries. Now AI is referred as a new electricity, and it is expected to have a similar impact on the world as electricity did a century ago. The future of AI and ML holds great promise, with ongoing advancements in technology and research leading to new applications and possibilities.

4. FAQ

5. References

• LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.
• Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
• Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction. Springer Science & Business Media.
• Bishop, C. M. (2006). Pattern recognition and machine learning. Springer.
• Murphy, K. P. (2012). Machine learning: A probabilistic perspective. MIT Press.
• Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117.
• Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798-1828.
• Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255-260.
• Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT Press.
• Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2), 179-188.