Machine learning serves as a ubiquitous and transformative tool across various scientific and engineering disciplines. It drives significant modern advancements in fields like robotics, logistics, computational biology, particle physics, and astronomy by optimizing complex systems and accelerating research discovery.

Scientific Applications of Machine Learning

A deep learning model is a powerful type of machine learning model differentiated by its architecture, which consists of many successive transformations of data chained together from top to bottom. This depth of continuous data transformations allows these models to tackle complex tasks by automatically learning multi-level representations. By doing so, deep learning effectively replaces not only the labor-intensive process of manual feature engineering, but also the shallow models that are typically used at the end of traditional machine learning pipelines.

Deep Learning Model

To build a formal mathematical system for learning, machine learning algorithms rely on objective functions, which are formal mathematical measures that quantify how well or poorly a model is performing its task. These functions provide the fundamental evaluation metric that optimization algorithms seek to improve.

Objective Function

Artificial General Intelligence (AGI) refers to a hypothetical, sentient artificial intelligence capable of reasoning about itself, improving its own architecture, and solving general tasks without human intervention. The textbook emphasizes that current AI systems are far from AGI; they are engineered, goal-oriented combinations of rules, heuristics, and statistical models that only give the illusion of general intelligence.

Artificial General Intelligence (AGI)

When statistical machine learning models are applied without care, they can inadvertently learn and perpetuate biases, such as racial, gender, or age bias. If these automated systems are used to drive consequential decisions, such as assessing creditworthiness or granting bail, they raise significant concerns about procedural fairness and equity.

Algorithmic Bias in Machine Learning

The application of artificial intelligence has profound societal implications, particularly concerning employment. As machine learning automates routine and menial tasks, such as harvesting operations using farm robots, it reduces costs but also displaces jobs that provide significant employment. This automation phase is considered a much more pressing, realistic concern than hypothetical scenarios of malevolent superintelligence destroying humanity.

Societal Impact of Artificial Intelligence

Machine learning is pervasive in modern life, operating widely behind the scenes to automate complex tasks. This includes financial applications like scoring creditworthiness and detecting fraudulent transactions—which form the backbone of major e-commerce payment systems—as well as internet applications where it feeds search, recommendation, personalization, and ranking algorithms.

Pervasive Machine Learning Applications

In machine learning, parametric models are algorithms that rely on a fixed, finite number of parameters to define their behavior. When these models are trained, updated, and saved, it is these specific parameters that are modified and written to disk. Because they possess a set number of parameters regardless of the dataset size, they typically contrast with nonparametric methods, whose complexity grows with the amount of available training data.

Parametric Models

A machine learning model is the computational machinery designed to ingest data of a specific type, apply mathematical transformations, and output predictions. In modern applications, the focus is on statistical models whose internal transformations can be estimated and refined using a dataset.

University of Michigan - Ann Arbor

Claude

- Computational models
- Textual models
- Conceptual models

Categories of Models

A dataset is collections of data, which can be in the form of a table, network, text, image, video, sound, ...

Dataset

A machine learning example, interchangeably referred to as a data point, sample, or instance, represents an individual record within a dataset. To be useful for learning algorithms, each example is characterized by a specific numerical representation, forming the basis for a model's predictions.

Machine Learning Example

Badham, Jennifer. Agent-Based Modelling for the Self Learner. MIT Press., 2015. 

Agent-Based Modelling for the Self Learner

Dive into Deep Learning

A description using only written language.

Textual model

Models that organize concepts through entities and relationships but do not compute anything.

Conceptual model

Machine Learning Model

- Number of data points (instances)
- Attributes (variables)
- Types of attributes (variables)
- Number of attributes (variables)
- Statistical measures

Characteristics of a dataset

Here are a list of free resources to find datasets. Most will be available in CSV format for easy integration into R or Python. 

Sample Datasets

It important to examine your dataset before starting to work with it for a machine learning task to:
- Understand how much missing data there is in the dataset.
- Get an idea of whether the features need further cleaning.
- Understand what techniques to use for analyzing the data.

The first step to analyze a dataset:

- 1) Equilibrium
- 2) Cyclic (periodic)
- 3) Random 
- 4) Complex

Wolfram's four classes of empirical data

Data can be distributed in several ways. Some of these include:
- Normal (Gaussian) distributions
- Uniform distributions
- Exponential distributions

Data Distributions

Supervised learning is a type of machine learning problem where the objective is to predict a designated unknown label based on known inputs. This approach requires a training dataset consisting of examples for which the ground truth labels are already known, allowing the model to learn the mapping from inputs to classifications.

Supervised Learning

Before data can be fed into any machine learning model, its overall quality must be evaluated and improved. Real-world datasets are often plagued by flaws such as extreme outliers, faulty sensor measurements, and human recording errors. Addressing these discrepancies is an essential part of the data preparation process to ensure the model learns from reliable information.

Data Quality

In real-world machine learning scenarios, datasets are often not provided in a single, simple CSV file. Instead, they may be stored in a relational database, where information is distributed across multiple tables. For example, an e-commerce application might store customer addresses in one table and their purchase data in another, requiring practitioners to extract and merge these separate files before modeling.

Relational Database

Beyond standard categorical and numeric variables, deep learning practitioners frequently work with myriad unstructured data formats. These complex data types include text strings for natural language processing, images and point clouds for computer vision, and audio data. Handling these diverse formats requires specialized approaches and advanced tools compared to traditional tabular data.

Deep Learning Data Types

A data processing bottleneck occurs when the time taken to load, transform, and prepare complex data—such as high-resolution images or large text corpora—exceeds the time required for actual model training. To prevent data preprocessing from severely slowing down the overall machine learning pipeline, practitioners must implement efficient algorithms and utilize advanced tools.

Data Processing Bottleneck

The success of a machine learning model heavily depends on having the right data, as encapsulated by the principle 'garbage in, garbage out.' If a dataset contains mistakes, features that do not predict the target, or reflects societal prejudices by under-representing certain groups, the learning process will fail or produce biased, harmful results.

Machine Learning Dataset Quality

Comma-separated values (CSV) files are a ubiquitous format for storing tabular, spreadsheet-like data. In a CSV file, each line represents a distinct data record, and the individual fields within that record are separated by commas.

CSV File

A data batch, commonly referred to as a minibatch, is a discrete, smaller subset of a larger dataset that is processed simultaneously during a single iteration of machine learning model training or evaluation. Training typically requires multiple passes over a dataset, grabbing one minibatch at a time. Each minibatch consists of a tuple of features and corresponding labels. Processing data in minibatches rather than the entire dataset balances computational efficiency with hardware memory constraints.

Data Batch

When loading datasets for machine learning, the order in which data is read depends on the current mode. For training, data is typically read in a random order (shuffled) to properly update the model. For validation, data is read in a pre-defined, sequential order, which is important for debugging purposes and consistent evaluation.

Training vs. Validation Data Reading Order

Optical character recognition (OCR) systems are a prominent, long-standing application of machine learning. Deployed since the 1990s for tasks such as automated mail sorting and reading bank checks, these systems drove early advancements in the field and led to the creation of the famous MNIST dataset of handwritten digits.

Optical Character Recognition (OCR)

In a machine learning dataset, a feature (also referred to as an input or covariate) is an individual, measurable attribute of a given data point. A model relies on these numerical representations to make its predictions.

Learn Before

Related

Learn After