The Curious Engineer

All blogs / AI / ML

inFERENCe

inference.vc

A blog about machine learning research, deep learning, causal inference, variational learning, by Ferenc Huszár.

AI / MLMath

’15’20’25
Posts
112 from 9 Jan 2012 to 25 Feb 2026
Posting
now and then (2 in the last year)
Kind
personal blog
Feed
https://www.inference.vc/rss/

2026 2

  1. The Future of Software
  2. Deep Learning is Powerful Because It Makes Hard Things Easy - Reflections 10 Years On

2025 1

  1. Discrete Diffusion: Continuous-Time Markov Chains

2023 4

  1. We may finally crack Maths. But should we?
  2. Mortal Komputation: On Hinton's argument for superhuman AI.
  3. Autoregressive Models, OOD Prompts and the Interpolation Regime
  4. We May be Surprised Again: Why I take LLMs seriously.

2022 2

  1. Implicit Bayesian Inference in Large Language Models
  2. Eastern European Guide to Writing Reference Letters

2021 4

  1. Causal inference 4: Causal Diagrams, Markov Factorization, Structural Equation Models
  2. On Information Theoretic Bounds for SGD
  3. Notes on the Origin of Implicit Regularization in SGD
  4. An information maximization view on the $\beta$-VAE objective

2020 2

  1. Some Intuition on the Neural Tangent Kernel
  2. Notes on Causally Correct Partial Models

2019 12

  1. Meta-Learning Millions of Hyper-parameters using the Implicit Function Theorem
  2. The secular Bayesian: Using belief distributions without really believing
  3. Exponentially Growing Learning Rate? Implications of Scale Invariance induced by Batch Normalization
  4. On Marginal Likelihood and Cross-Validation
  5. Notes on iMAML: Meta-Learning with Implicit Gradients
  6. Invariant Risk Minimization: An Information Theoretic View
  7. ICML Highlight: Contrastive Divergence for Combining Variational Inference and MCMC
  8. Notes on the Limitations of the Empirical Fisher Approximation
  9. Perceptual Straightening of Natural Videos
  10. DeepSets: Modeling Permutation Invariance
  11. Causal Inference 3: Counterfactuals
  12. Causal Inference 2: Illustrating Interventions via a Toy Example

2018 9

  1. Online Bayesian Deep Learning in Production at Tencent
  2. 👻Halloween Special: Critical reviews of the worst NIPS 2018 papers.
  3. The Blessings of Multiple Causes: Causal Inference when you Can't Measure Confounders
  4. ML beyond Curve Fitting: An Intro to Causal Inference and do-Calculus
  5. The Lottery Ticket Hypothesis - Paper Recommendation
  6. Goals and Principles of Representation Learning
  7. Pruning Neural Networks: Two Recent Papers
  8. My notes on (Liang et al., 2017): Generalization and the Fisher-Rao norm
  9. The Generalization Mystery: Sharp vs Flat Minima

2017 23

  1. Alchemy, Rigour and Engineering
  2. Grosse's challenge: duality and exponential families
  3. Thanksgiving Special 🦃: GANs are Being Fixed in More than One Way
  4. A Cookbook for Machine Learning: Vol 1
  5. Gaussian Distributions are Soap Bubbles
  6. mixup: Data-Dependent Data Augmentation
  7. AlphaGo Zero: Minimal Policy Improvement, Expectation Propagation and other Connections
  8. GANs are Broken in More than One Way: The Numerics of GANs
  9. From Instance Noise to Gradient Regularisation
  10. Everything that Works Works Because it's Bayesian: Why Deep Nets Generalize?
  11. Exemplar CNNs and Information Maximization
  12. Is Maximum Likelihood Useful for Representation Learning?
  13. Unsupervised Learning by Predicting Noise: an Information Maximization View
  14. Evolution Strategies, Variational Optimisation and Natural ES
  15. Evolution Strategies: Almost Embarrassingly Parallel Optimization
  16. Choice of Recognition Models in VAEs: a regularisation view.
  17. Comment on "Overcoming catastrophic forgetting in NNs": Are multiple penalties needed?
  18. The Spherical Kernel Divergence
  19. Variational Inference using Implicit Models
  20. Variational Inference using Implicit Models Part III: Joint-Contrastive Inference, ALI and BiGAN
  21. Variational Inference using Implicit Models Part IV: Denoisers instead of Discriminators
  22. Variational Inference using Implicit Models Part II: Amortised Inference
  23. Variational Inference using Implicit Models, Part I: Bayesian Logistic Regression

2016 17

  1. 🎄 2016 Holiday Special: Deriving the Subpixel CNN from First Principles
  2. New Perspectives on Adversarial Training (NIPS 2016 Adversarial Training Workshop)
  3. Solving Intelligence vs Hastening the Arrival of the Next AI Winter
  4. Instance Noise: A trick for stabilising GAN training
  5. Are Energy-Based GANs any more energy-based than normal GANs?
  6. How powerful are Graph Convolutions? (review of Kipf & Welling, 2016)
  7. The Variational Rényi Lower Bound
  8. InfoGAN: using the variational bound on mutual information (twice)
  9. Time-Contrastive Learning for Latent Variable Models
  10. Understanding Minibatch Discrimination in GANs
  11. Dilated Convolutions and Kronecker Factored Convolutions
  12. Notes on "Unsupervised Represerntation Learning by Solving Jigsaw Puzzles"
  13. Adversarial Preference Loss
  14. An Alternative Update Rule for Generative Adversarial Networks
  15. Deep Learning is Easy - Learn Something Harder
  16. Adversarial Autoencoders (vs Moment Matching Autoencoders?)
  17. The Next Episode: I'm joining Magic Pony Technology

2015 20

  1. Accuracy vs Explainability of Machine Learning Models [NIPS workshop poster review]
  2. When is Machine Learning Worth it?
  3. How to Train your Generative Models? And why does Adversarial Training work so well?
  4. Idea of the Week: Maximum Spacing Estimation, Gaussianisation and Autoregressive Processes
  5. A Word of Caution on Scheduled Sampling for Training RNNs
  6. Unsupervised Feature Learning in Video: Learning to Linearize
  7. The Two Kinds of Uncertainty an AI Agent Has to Represent
  8. Comment on Generative Image Models via Laplacian Pyramids
  9. Exchangeable Models via Recurrent Neural Networks?
  10. From Autoencoders to Autoregressive Models (Masked Autoencoders ICML Paper)
  11. Unsupervised Learning by Inverting Diffusion Processes
  12. The Turing Test is About UX, not Intelligence
  13. Another Favourite Machine Learning Paper: Adversarial Networks vs Kernel Scoring Rules
  14. Conversational AI: The User Interface of the Future?
  15. The Holy Grail of Deep Learning: Modelling Invariances
  16. Why We Won't Build Sentient AI in our Lifetime
  17. The Three Ways AI Will Create Value
  18. Stereovision Autoencoder
  19. A new Favourite Machine Learning Paper: Autoencoders VS. Probabilistic Models
  20. Blockchain 1: What it is and what it is not

2014 8

  1. SaaS Metrics: Negative Churn
  2. The Story of Dark Blue Labs, and Google
  3. What makes European startup hubs unique?
  4. Never Incaffeinate Alone
  5. The Next Episode
  6. Finding Clusters in the London Startup Scene
  7. Javasctipt interactive d3.js test
  8. The Inherent Weakness of Likes and Star Ratings

2013 2

  1. The Science of Encouraging Honesty
  2. Hoxton Ventures - New Early Stage VC fund Targeting $1b European Exits

2012 6

  1. Exploration versus Exploitation: Deciding When to Get Married
  2. Sock Gnomes versus Laws of Large Numbers
  3. Micropayments and Crowdfunding in Academia
  4. Observe, don't just see, Dr Watson
  5. Influence and causality: castration doesn't make you live longer
  6. Influentomics: the science of social influence