Statistical learning involves using supervised tools, unsupervised tools, time series learning, and semi-supervised tools and methods to analyze data to understand the data, extract knowledge, and infer insights.

Statistical learning

There are two types of research methods: 

- Quantitative research
- Qualitative research

Both quantitative and qualitative types of research are required and often complement each other.  Science became more dependent on quantitative data and conduct quantitative tests and use qualitatively described relations.


Research methods in Data Science: Qualitative & Quantitative

The marriage of Statistics, Computer Science, and Domin knowledge built Data Science.

University of Michigan - Ann Arbor

Science is a systematic effort to build and organize knowledge as testable statements and predictions. Data science, as an interdisciplinary field, involves data and, by extension, statistics, or the systematic study of the organization, properties, and analysis of data and its role in inference, including our confidence in the inference. It uses scientific methods, processes, and algorithms to extract knowledge and infer insights from structured and unstructured data.

Data Science

Statistics is the process of gathering, analyzing, and interpreting information in order to help people make decisions when faced with uncertainty.

Statistics

In the dominant machine learning approach, rather than designing explicit step-by-step rules to map inputs to outputs, practitioners define a flexible program whose behavior is governed by a set of adjustable parameters. By presenting this program with a large dataset of examples and labels, a learning algorithm determines the optimal parameter values that maximize the model's performance on the desired task.

Machine Learning Approach

Dhar, V. (2013). Data science and prediction. Communications of the ACM, 56(12), 64-73.

Data science and prediction

The list of helpful broad references about data science is categorized as follows:
1. Data science books
2. Data science online courses
3. Data science research papers
4. Images
5. Data science videos

Data science references

Data science is interdisciplinary

A dataset is collections of data, which can be in the form of a table, network, text, image, video, sound, ...

Dataset

Study of financial risk and uncertainty. Combines elements of Business, Data Science, Statistics, and Computer Science to help quantify financial risk. Actuaries are mostly hired into the Financial or Insurance industries to model various economic uncertainties.

Actuarial Mathematics

Data science in sports involves the application of statistical analysis, predictive modeling, and machine learning to sports-related data. It enables sports organizations to evaluate athletic performance, formulate game strategies, predict and prevent injuries, optimize scouting, and improve business operations such as fan engagement and ticket sales.

Data Science in Sports

The list of helpful broad references about Statistics is categorized as follows:
- Statistics books
- Statistical Software 
- Statistics online courses
- Statistics research papers

Statistics References

A few of the common statistical paradoxes are:"

- Monty Hall's paradox 
- Berkson's paradox
- Simpson's paradox

Common statistical paradoxes

Probability is the branch of math concerned with the likelihood of an event occuring, or the likelihood that an event is true. 

Probability Theory

Bayesian statistics is an approach to inferential statistics where a researcher specifies the prior probability that the null hypothesis and alternative hypotheses are true before conducting a study. Once the study is conducted, the probabilities are updated based on the newly collected data. This approach offers a potential alternative to traditional null hypothesis testing by providing a logical framework for quantifying uncertainty using probability distributions.

Bayesian Statistics

Components of Statistical Analysis

Descriptive statistics are a set of quantitative techniques used to summarize, organize, and visually display the data collected from a research sample. Important examples include measures of central tendency (such as the mean, median, and mode), which describe average scores, and measures of dispersion (such as the range, standard deviation, and variance), which indicate how spread apart the scores are.

Descriptive Statistics

- Printed circuit boards
- Communication hardware, interfaces and storage
- Integrated circuits
- Very large scale integration design
- Power and energy
- Electronic design automation
- Hardware validation
- Hardware test
- Robustness
- Emerging technologies

Hardware in ACM Computing Classification

- Architectures
- Embedded and cyber-physical systems
- Real-time systems
- Dependable and fault-tolerant systems and networks

Computer systems organization in ACM Computing Classification

- Network architectures
- Network protocols
- Network components
- Network algorithms
- Network performance evaluation
- Network properties
- Network services
- Network types

Networks in ACM Computing Classification

- Software organization and properties
- Software notations and tools
- Software creation and management

Software and its engineering in ACM Computing Classification

- Models of computation
- Formal languages and automata theory
- Computational complexity and cryptography
- Logic
- Design and analysis of algorithms
- Randomness, geometry and discrete structures
- Theory and algorithms for application domains
- Semantics and reasoning

Theory of computation in ACM Computing Classification

- Discrete mathematics
- Probability and statistics
- Mathematical software
- Information theory
- Mathematical analysis
- Continuous mathematics

Mathematics of computing in ACM Computing Classification

- Data management systems
- Information storage systems
- Information systems applications
- World Wide Web
- Information retrieval

Information systems in ACM Computing Classification

- Cryptography
- Formal methods and theory of security
- Security services
- Intrusion/anomaly detection and malware mitigation
- Security in hardware
- Systems security
- Network security
- Database and storage security
- Software and application security
- Human and societal aspects of security and privacy

Security and privacy in ACM Computing Classification

- Human computer interaction (HCI)
- Interaction design
- Collaborative and social computing
- Ubiquitous and mobile computing
- Data Visualization
- Accessibility

Human-centered computing in ACM Computing Classification

- Electronic commerce
- Enterprise computing
- Physical sciences and engineering
- Life and medical sciences
- Law, social and behavioral sciences
- Computer forensics
- Arts and humanities
- Computers in other domains
- Operations research
- Learning Science
- Document management and text processing

Applied computing in ACM Computing Classification

- Professional topics
- Computing / technology policy
- User characteristics

Social and professional topics in ACM Computing Classification

The list of helpful broad references about ML is categorized as follows:
1- ML books
2- ML online courses
3- ML research papers

Machine Learning references

There are several categories for ML algorithms, including:
 - Supervised Learning
 - Semi-supervised Learning
 - Unsupervised Learning
 - Reinforcement Learning
 - Feature Learning (Representation Learning)

The types of machine learning algorithms differ in their approach, the type of data they input and output, and the type of task or problem that they are intended to solve.


Machine Learning Categories

Python is widely used for machine learning various reasons.
- It is relatively easy to learn.
- The data handling capacity is better than most other languages.
- The community is large, as well as support for other platforms.
- Its’s Open Source.
- It supports developers with frameworks and libraries such as TensorFlow


Machine Learning with Python


here is a cycle for doing machine learning evaluation and selection. The first is representation. You extract and select object features. The second is train. You train the model by fitting the estimator to the data. Afterwards, you evaluate the model to look at its accuracy, recall and all the relevant metrics. Finally you refine the model and feature and go back to step one.

Represent/Train/Evaluate/Refine Cycle

1.Identify diseases and diagnosis
2.Drug discovery and manufacturing
3.Medical imaging diagnosis
4.Personalized medicine
5.Machine learning-based behavioral modification
6.Smart health records
7.Clinical trial and research
8.Crowdsourced data collection
9.Better Radiotherapy
10.Outbreak prediction

Machine learning and applications in healthcare

For example, the linear regression algorithm combines a dataset consisting of X and y, the cost function


The cost function typically includes at least one term that causes the learningprocess to perform statistical estimation. The most common cost function is the negative log-likelihood, so that minimizing the cost function causes maximum likelihood estimation. 

In simple terms, the cost function is used to evaluate whether the parameter value W is reasonable.

Cost Function

Graph Representation Learning is the application of machine learning techniques on graph-structured data. A few examples of growing topics in Graph Representation Learning include techniques for deep graph embeddings, Graph Neural Networks (GNNs), and applications to knowledge graphs.

Graph Representation Learning

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

Active learning (AL) attempts to maximize a model’s performance gain while annotating the fewest samples possible.

Active Learning

In machine learning, parameters can be intuitively understood as adjustable 'knobs' that dictate the behavior of a flexible program. By manipulating these parameters, practitioners can change how the program maps inputs to outputs. Once these parameter values are determined and fixed, the resulting program is referred to as a model.

Machine Learning Model Parameter

A learning algorithm is a 'meta-program' that utilizes a dataset to determine the optimal parameter values for a machine learning model. Instead of explicitly coding the model's behavior, the learning algorithm automatically discovers the best configuration of parameters to improve the model's performance on a specific task.

Learning Algorithm

A good machine learning practitioner must know how to select an algorithm for a specific application and how to monitor and respond to experimental feedback to improve the model. In the textbook *Deep Learning*, Ian Goodfellow, Yoshua Bengio, and Aaron Courville recommend a four-step practical design process:

1. **Determine Goals:** Choose appropriate error metrics and set target values that are driven directly by the problem the application is intended to solve.
2. **Establish a Pipeline:** Set up a working, end-to-end pipeline as soon as possible, including tools to estimate the selected performance metrics.
3. **Instrument the System:** Monitor and analyze the pipeline to identify performance bottlenecks. Diagnose whether poor performance is due to overfitting, underfitting, or issues in the software or data.
4. **Iterate and Refine:** Repeatedly make incremental adjustments—such as collecting more data, tuning hyperparameters, or selecting different algorithms—guided by the specific findings from your instrumentation.

Practical Design Process for Machine Learning Systems

Almost all deep learning algorithms can be described as specific instances of a simple recipe. They are constructed by combining four core components: a dataset specification, a cost function, an optimization process, and a parametric model.

Learn Before

Related

Learn After