Adaptive Inference Reading Group (Spring 2016)

Thank you to those of you who filled out the Doodle poll. Our regular meeting time will be Wednesday, 1-3pm, in Evans 1011.

Logistics

Meeting time: 1-3pm, Evans 1011.

Mailing list: adaptiveinference@lists.berkeley.edu (you can subscribe here)

Google doc for collaborative scheduling

Reading group tips for presenters and listeners (courtesy Lester Mackey, Percy Liang, and their reading groups)

Theme

This reading group will discuss and compare recent advances in adaptive inference: answering questions that were suggested by the data. In many modern applications large, complex data sets are collected without specific analyses in mind. Rather, the goal is expressly to explore the data in search of novel insights, discovering relationships and structure that we did not expect to find.

By contrast, most classical statistical procedures are only valid in a fixed research protocol where the analyst chooses what questions to ask, and how to ask them, before collecting the data. Violations of this fixed paradigm have long been widespread, and have contributed to a widely publicized crisis of credibility for statistical practice in the sciences. There is a pressing need for inference procedures that retain their validity even after an adaptive search for “interesting” questions.

Several recent lines of work offer useful tools and promising new research directions based on conditional inference, differential privacy, and other ideas. At each meeting a group member will present one paper, followed by a discussion of the topic and how it relates to the other topics we have covered.

Relevant Links

NIPS 2015 workshop on adaptive data analysis (see Moritz Hardt's summary here)