Jacob Steinhardt (jsteinhardt@berkeley)

My goal is to make the conceptual advances necessary for machine learning systems to be reliable and aligned with human values. This includes the following directions:
  • Robustness: How can we build models robust to distributional shift, to adversaries, to model mis-specification, and to approximations imposed by computational constraints? What is the right way to evaluate such models?
  • Reward specification and reward hacking: Human values are too complex to be specified by hand. How can we infer complex value functions from data? How should an agent make decisions when its value function is approximate due to noise in the data or inadequacies in the model? How can we prevent reward hacking--degenerate policies that exploit differences between the inferred and true reward?
  • Scalable alignment: Modern ML systems are often too large, and deployed too broadly, for any single person to reason about in detail, posing challenges to both design and monitoring. How can we design ML systems that conform to interpretable abstractions? How do we enable meaningful human oversight at training and deployment time despite the large scale? How will these large-scale systems affect societal equilibria?
These challenges require rethinking both the theoretical and empirical paradigms of ML. Theories of statistical generalization do not account for the extreme types of generalization considered above, and decision theory does not account for cases where the reward function is only approximate. Meanwhile, measuring empirical test accuracy on a fixed distribution is insufficient to analyze phenomena such as robustness to distributional shift.

I seek students who are technically strong, broad-minded, and want to improve the world through their research. I particularly value creative, curious thinkers who are excited to revisit the conceptual foundations of the field.

Outside of research, I am a coach for the USA Computing Olympiad and an instructor at the Summer Program in Applied Rationality and Cognition. I also consult part-time for the Open Philanthropy Project. I like indoor bouldering and ultimate frisbee.

Teaching

STAT240 (Robust and Nonparametric Statistics) (Spring 2021)
STAT260 (Robust Statistics) (Fall 2019)

Past/Present

I joined the Statistics faculty at UC Berkeley in Fall of 2019, where I am also a member of the Berkeley Artificial Intelligence Lab and of the EECS department (by courtesy). I recently finished a PhD in machine learning at Stanford University working with Percy Liang. In-between I spent some time working at the Open Philanthropy Project and at OpenAI.

Blog

I maintain a (somewhat slow-updating) expository blog. I also used to keep an online daily research log early in graduate school.

Essays

AI Alignment Research Overview (October 2019) [link]
Research as a Stochastic Decision Process (December 2018) [link]
Long-Term and Short-Term Challenges to Ensuring the Safety of AI Systems (June 2015) [link]
The Power of Noise (June 2014) [link]
A Fervent Defense of Frequentist Statistics (February 2014) [link]
Beyond Bayesians and Frequentists (October 2012) [link]

Students/Post-docs

Publications

Publications

2021

2020

2019

2018

2017

2016

2015

2014

2012

2011

2010

2009

2007