Reading (and read) list

Somewhat organized

Blogs, Tutorials, Courses and such

• Counterfactual Machine Learning
• Knowledge Discovery in Databases
• ML for Combinatorial Optimization
• Stochastic Bilevel Optimization in ML
• There and Back Again: A Tale of Slopes and Expectations (Slideslive)
• Sketching and Streaming Algorithms (Slideslive)
• Machine Learning for Astrophysics and Astrophysics Problems for Machine Learning
• Deep Implicit Layers - Neural ODEs, Deep Equilibirum Models, and Beyond
• Explaining Machine Learning Predictions
• Equivariant Networks (slideslive)
• Recent Advances in High-Dimensional Robust Statistics
• Theoretical Foundations of Reinforcement Learning
• Optimal Transport for Machine Learning tutorial
• The Matrix Cookbook
• Blog: Attention? Attention!
• Blog: Meta-learning – learning to learn fast
• Blog: The Transformer Family

Papers (mostly)

Algorithms (unsorted)

• REINFORCE algorithm paper
• Search Algorithms for \(m\)-Best Solutions for Graphical Models
• Select-and-Evaluate: A Learning Framework for Large-Scale Knowledge Graph Search

AutoML

• hibayesian/awesome-automl-papers: A curated list of automated machine learning papers, articles, tutorials, slides and projects
• Analysis of complexity of automatic pipeline generation problem
• Fitness Landscape Analysis of AutoML search spaces
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch

AutoML / Toolkits

• FLAML
• FLAML: A Fast and Lightweight AutoML Library
• RECIPE: A Grammar-Based Framework for Automatically Evolving Classification Pipelines
• Layered TPOT
• MLPlan Unlimited
• Oracle AutoML
• Optuna: Next-gen HPO framework

AutoML / CASH

• Selecting Near-Optimal Learners via Incremental Data Allocation
• Non-stochastic Best Arm Identification and Hyperparameter Optimization
• Weighted Sampling for Combined Model Selection and Hyperparameter Tuning
• AutoML Strategy Based on Grammatical Evolution: A Case Study about Knowledge Discovery from Text
• OBOE: Collaborative Filtering for AutoML Model Selection
• Exploiting Reuse in Pipeline-Aware Hyperparameter Tuning

AutoML / BBO

• Embedded Bandits for Large-Scale Black-Box Optimization
• Scalable Global Optimization via Local Bayesian Optimization
• Maximizing acquisition functions for Bayesian optimization
• Improving the Expected Improvement Algorithm
• Assessing Hyperparameter Importance
• Multi-Fidelity BBOpt with Hierarchical Partitions
• Noisy BBOpt Multi-Fidelity Queries: TreeSearch
• Hyperband demo by kgjamieson
• Hyper-parameter Tuning under a Budget Constraint
• PFNs4BO: In-Context Learning for Bayesian Optimization

AutoML / Combinatorial BBO

• AutoML Pipeline Selection: Efficiently Navigating the Combinatorial Space
• Optimizing Discrete Spaces via Expensive Evaluations: A Learning to Search Framework
• Fourier Representation for Black-box Optimization over Categorical Variables
• Combinatorial Black-Box Optimization with Expert Advice
• BO for Combinatorial Structures
• combinatorial-bayesian-optimization-using-the-graph-cartesian-product
• Bayesian Optimization of Combinatorial Structures
• Black-box Combinatorial Optimization using Models with Integer-valued Minima
• Mercer Features for Efficient Combinatorial Bayesian Optimization
• Bayesian Optimization over Permutation Spaces
• Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces
• Neural Set Function Extensions

AutoML / Meta-learning

• Meta-Learning for Symbolic Hyperparameter Defaults

AutoML / Feature Engineering

AutoML / RL

AutoML / NAS

• Geometry-Aware Gradient Algorithms for Neural Architecture Search

AutoML / Runtime Estimation

Bandits

Boosting

Compositional Generalization

Benchmarks

• COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
• A Study of Compositional Generalization in Neural Models

Formulations & Algorithms

• From Compositional to Systematic Semantics
• Compositionality I: Definitions & Variants
• Compositionality II: Arguments & Problems
• Routing Networks & the Challenges of Modular and Compositional Computation
• How Do In-Context Examples Affect Compositional Generalization?
• Compositional Semantic Parsing With Large Language Models
• Anti-unification and Generalization: A Survey
• Faith and Fate: Limits of Transformers on Compositionality
• Categorial Compositionality: A Category Theory Explanation for the Systematicity of Human Cognition
• Compositional Generalization from First Principles

Experimental Design

• Numerical methods for experimental design of large-scale linear ill-posed inverse problems
• Numerical methods for the design of large-scale nonlinear discrete ill-posed inverse problems
• Dynamic graph and polynomial chaos based models for contact tracing data analysis and optimal testing prescription

Interpretability

• Discovering State Variables Hidden in Experimental Data

Neural Networks for Table

• Graph Neural Network Contextual Embedding for Deep Learning on Tabular Data
• Learning Enhanced Representations for Tabular Data via Neighborhood Propagation
• Table2Graph: Transforming Tabular Data to Unifed Weighted Graph
• UniTabE: Pretraining a Unified Tabular Encoder for Heterogeneous Tabular Data

Learning Theory

• Exponentiated Gradient versus Gradient Descent for Linear Predictors

Large Language Models & Transformers

• Efficient Transformers: A Survey
• Formal Algorithms for Transformers
• Do Transformers Parse while Predicting the Masked Word?
• Mass-editing Memory in a Transformer

Symbolic Regression

• Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients