Evgeniy's ML Topic List (M0-M8)
Hi all, here is a topic list that I would like to stick to fro the models section of the guide, once they are stable I will add book and article resources by section and that focus more on software tools and productisation. For your notes below is the modelling topic list as it is now.
Best,
Evgeniy
M. Models
M0. Statistical thinking and intuition
Probability, random variables, distributions.
Structure and causality. Reduced form, atheoretic models.
Inference (econometrics) vs generalisation (machine learning).
Breiman's Two Cultures (2001) and rebuttals.
Treatment and (heterogeneous) treatment effects.
Observation, experiment and experiment design.
Measurement errors and missing data.
Model lifecyle, data and model drift.
M1. Statistical inference
Data generating process, sample vs population.
Sampling techniques and inference. Statistical model.
Frequentist, Bayesian and other views of a statistical model.
Modelling tradeoffs and “no free lunch”.
Measuring model quality and performance.
M2. Econometrics
Cross-section, time series, panel and spatial data.
Linear regression and ordinary least squares (OLS).
Violation of OLS assumptions (Peter Kennedy textbook and 10 commandments).
Difference-in-differences, instrumental variables, regression discontinuity.
Time series. Seasonal adjustment, smoothing, filtering.
Systems of equations.
Estimation (OLS and varieties, GMM, maximum likelihood, Bayesian estimates).
M3. Bayesian modeling and causal inference.
Bayesian modeling and probabilistic programming.
Causal inference and do-notation.
M4. Classic machine learning.
Statistical learning theory (loss, theoretical and empirical risk, PAC-Bayes), criteria for ML model.
Tasks: classification, regression, clustering, dimensionality reduction, decision trees, support vector machines and discriminant analysis.
Forecast combination, choosing forecasts and AutoML.
M5. Neural nets (NN) and deep learning.
Simple perceptron and NN construction (gradient descent, backpropagation, regularization).
Traditional NN architectures (feed-forward, convolutional, recurrent, generative аdversarial, transformers).
Artificial general intelligence (AGI) and tests for intelligence.
M6. NLP, CV and RL subfields.
Text and speech – classic NLP. Jurafsky and Martin.
Transformer architecture, prompt engineering, RAGs and fine-tuning.
Computer vision (CV).
Reinforcement learning (RL).
M7. Other modelling techniques.
Graphs and networks.
System dynamics (SD). Forrester.
Agent-based modeling (ABM).
Game theory and auction design.
Optimisation (LP).
M8. Additional topics (harder or less exciting)
Probability as part of measure theory.
Combinatorics.
Mathematical statistics (point estimation, confidence bands, hypothesis testing).
Central limit theorems, asymptotics and convergence.
Non-parametric statistics.
Statistical decision theory.
Entropy, cross-entropy and information theory.
Differential equations and random processes.