Michael Mahoney  Presentations
Talks and Presentations
Several recent presentations:
Scientific machine learning: methods to bridge scientific spatial and temporal modeling with machine learning
(Jun 2022)
(pdf)
Practice, Theory, and Theorems for Random Matrix Theory in Modern Machine Learning
(Jun 2022)
(pdf)
Building foundations for scientific machine learning at scale
(Mar 2022)
(pdf)
Continuous Network Models for Sequential Predictions
(Jan 2022)
(pdf)
Toward combining principled scientific models and principled machine learning models
(Nov 2021)
(pdf)
Column Subset Selection: TCS and NLA
("at" the CMC Seminar Series, Nov 2021)
(pdf)
Putting Randomized Matrix Algorithms in LAPACK, and Connections with Secondorder Stochastic Optimization
("at" the NeurIPS 2021 Optimization Workshop)
(pdf)
Least Squares in RandNLA
("at" the CMC Seminar Series, Sept 2021)
(pdf)
Incorporating second order ideas into first class machine learning methods
("at" the MOPTA Meeting, Aug 2021)
(pdf)
Overcoming Inversion Bias in Distributed Newton's Method
(May 2021)
(pdf)
Practical Theory and Neural Network Models
("at" TOPML Workshop and elsewhere, April 2021)
(pdf)
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
("at" MIT and elsewhere, September 2020)
(pdf)
Dynamical systems and machine learning: combining in a principled way datadriven models and domaindriven models
(September 2020)
(pdf)
ContinuousinDepth Neural Networks
(August 2020)
(pdf)
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
(older, August 2020)
(pdf)
Determinantal Point Processes and Randomized Numerical Linear Algebra
(April 2020)
(pdf)
Several tutorial presentations:
Statistical Mechanics Methods for Discovering Knowledge from Modern Production Quality Neural Networks
(at ACMSIGKDD 2019)
(pdf)
Sampling for Linear Algebra, Statistics, and Optimization
(Aug 2018, at Simons' Institute 2018 Big Data Bootcamp)
(pdf)
RandNLA: Randomization in Numerical Linear Algebra
(5.5 hr version at UW Madison Summer School, July 2018)
(pdf)
Randomization in Numerical Linear Algebra: Theory and Practice
(2.0 hr version at SIAM ALA Meeting, October 2015)
(pdf)
Past, Present and Future of Randomized Numerical Linear Algebra:
(3.0 hr version at Simons' Institute 2013 Big Data Bootcamp)
(Part I:
pdf,
ppt
and Part II:
pdf,
ppt)
Theory (and some practice) of Randomized Algorithms for Matrices and Data
(tutorial from FOCS 2012 Workshop)
(pdf,
ppt)
Geometric Tools for Identifying Structure in Large Social and Information Networks
(1.5 hr version at SAMSI Opening Workshop 2010, etc.)
(pdf,
ppt)
Geometric Tools for Identifying Structure in Large Social and Information Networks
(2 hr version at ICASSP 2011, etc.)
(pdf,
ppt)
Geometric Tools for Identifying Structure in Large Social and Information Networks
(3 hr version at ICML 2010 and KDD 2010, etc.)
(pdf,
ppt)
(The pdf file in four pieces:
here,
here,
here,
and
here.)
Randomized Algorithms for Matrices and Massive Data Sets
(at SIAMSDM06 2006 and VLDB 2006)
(ppt)
Randomized Algorithms for Matrices and Massive Data Sets
(at ACMSIGKDD 2005)
(ppt)
Older presentations:
Using dynamical systems ideas to combine in a principled way datadriven models and domaindriven models
(older, April 2020)
(pdf)
Exact expressions for double descent and implicit regularization via surrogate random design
(December 2019)
(pdf)
Making the Deep Learning Revolution Practical Through Second Order Methods
(at MIT, October 2019)
(pdf (big=40MB))
NewtonMR: Newton's Method Without Smoothness or Convexity
(October 2019)
(pdf)
Minimax and Bayesian experimental design: Bridging the gap between statistical and worstcase approaches to least squares regression
(at TTIC, Sept 2019)
(pdf)
Why Deep Learning Works: Traditional and HeavyTailed Implicit SelfRegularization in Deep Neural Networks
(at the ICML 2019 Workshop on Theoretical Physics for Deep Learning, June 2019)
(pdf)
Why Deep Learning Works: Implicit SelfRegularization in Deep Neural Networks
(at the SF Bay Area DMSIG Meeting, February 2019)
(pdf)
Why Deep Learning Works: Implicit SelfRegularization in Deep Neural Networks
(Sept 2018, at Simons' Institute 2018 Big Data RandNLA meeting)
(pdf)
Large Scale Training of Neural Networks
(Sept/Nov 2018)
(pdf)
Alchemist: An Apache Spark <=> MPI Interface
(June 2018)
(pdf)
Biomedicine & the Foundations of Data?
(May 2018)
(pdf)
Scientific Machine Learning with Alchemist (An Apache Spark <=> MPI Interface) and Beyond
(Apr 2018)
(pdf)
Numericallyintensive Machine Learning at Scale
(Fall 2017)
(pdf)
Secondorder Machine Learning
(Fall 2017)
(pdf)
UC Berkeley's FODA Institute: Foundations of Data Analysis
(NSF TRIPODS Kickoff, Oct 2017)
(pdf)
Secondorder Machine Learning
(Prefall 2017)
(pdf)
Local graph analytics: beyond characterizing community structure
(Spring 2017)
(pdf)
Terabytescale Computational Statistics
(talk Fall 2016)
(pdf)
Scientific Matrix Factorizations in Spark at Scale: Crossplatform performance, scaling, and comparisons with C+MPI
(talk at 2016 Dato Data Science Summit and elsewhere)
(pdf)
Optimization Algorithms for Analyzing Large Datasets
(talk at 2016 PCMI Summer School)
(pdf)
Foundations of Data Science
(talk at NSF preTRIPODS workshop/meeting, Apr 2016)
(pdf)
SubSampled Newton Methods
(talk at ITA 2016 and elsewhere)
(pdf)
Challenges in Multiresolution Methods for Graphbased Learning
(talk, 3of3, from NIPS15 Workshops)
(pdf)
Using Local Spectral Methods in Theory and in Practice
(talk, 2of3, from NIPS15 Workshops)
(pdf)
Column Subset Selection on Terabytesized Scientific Data
(talk, 1of3, from NIPS15 Workshops)
(pdf)
Linear and Sublinear Linear Algebra Algorithms: Preconditioning Stochastic Gradient Algorithms with Randomized Linear Algebra
(DIMACS, Aug 2015)
(pdf)
Overview of RandNLA: Randomized Numerical Linear Algebra
(pdf)
Treelike structure in social graphs
(pdf (big=28MB),
ppt (big=48MB))
Eigenvector localization, implicit regularization, and algorithmic antidifferentiation for largescale graphs and network data
(pdf,
ppt)
Locallybiased and semisupervised eigenvectors
(talk from MMDS 2014)
(pdf,
ppt)
Implicit regularization in sublinear approximation algorithms
(pdf,
ppt)
BIG Biomedicine and the Foundations of BIG Data Analysis (at Big Data in Biomedicine at Stanford's Medical School, 5/23/14)
(pdf,
ppt)
Revisiting the Nystrom Method for Improved LargeScale Machine Learning
(pdf,
ppt)
Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments
(version from Simons Big Data Workshop II)
(pdf)
Inputsparsity Time Algorithms for Embeddings and Regression Problems
(talk from Simons Big Data Workshop I)
(pdf)
Randomized Regression in Parallel and Distributed Environments
(talk from GraphLab 2013)
(pdf)
Extracting insight from large networks: implications of smallscale and largescale structure
(pdf,
ppt)
Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments
(version from MMDS 2012)
(pdf)
Sensors, networks, and massive data
(pdf,
ppt)
Randomized Algorithms for Matrices and Data
(pdf,
ppt)
Approximate computation and implicit regularization in largescale data analysis
(PODS vsn)
(pdf,
ppt)
Approximate computation and implicit regularization in largescale data analysis
(Stats vrsn1)
(pdf,
ppt)
Approximate computation and implicit regularization in largescale data analysis
(Short vrsn)
(pdf,
ppt)
Looking for clusters in your data ... in theory and in practice
(pdf,
ppt)
Fast Approximation of Matrix Coherence and Statistical Leverage
(pdf,
ppt)
Implementing regularization implicitly via approximate eigenvector computation
(pdf,
ppt)
Linear Algebra and Machine Learning of Large Informatics Graphs
(pdf,
ppt)
Geometric Network Analysis Tools
(talk from MMDS 2010)
(pdf,
ppt)
Algorithmic and Statistical Perspectives on LargeScale Data Analysis
(pdf,
ppt)
Community structure in large social and information networks
(newer)
(pdf,
ppt)
Statistical leverage and improved matrix algorithms
(newer and long)
(pdf,
ppt)
Approximation Algorithms as Experimental Probes of Informatics Graphs
(pdf,
ppt)
Community structure in large social and information networks
(talk from MMDS 2008)
(pdf,
ppt)
Community structure in large social and information networks
(older)
(pdf,
ppt)
Statistical leverage and improved matrix algorithms
(older and short)
(pdf,
ppt)
Sampling algorithms and coresets for Lp regression and applications
(pdf,
ppt)
CUR Matrix Decompositions for Improved Data Analysis (talk from MMDS 2006)
(pdf,
ppt)
A RelativeError CUR Decomposition for Matrices and Its Data Applications
(pdf,
ppt)
Sampling Algorithms for L2 Regression and Applications
(talk from SODA 2006)
(pdf,
ppt)
Approximating a Gram Matrix for Improved KernelBased Learning
(talk from COLT 2005)
(ps,
pdf)
Fast Monte Carlo Algorithms for Matrix Operations and Massive Data Set Analysis
(newer)
(pdf,
ppt)
Fast Monte Carlo Algorithms for Matrix Operations and Massive Data Set Analysis
(older)
(pdf)
CUR Matrix Decomposition with Applications to Algorithm Design and Massive Data Set Analysis
(pdf)
Fast Monte Carlo Algorithms for Massive Data Sets and Approximating MaxCut
(ps,
pdf)
The TIP5P Water talk:
Videos of Talks and Presentations
"Practice, Theory, and Theorems for Random Matrix Theory in Modern Machine Learning"

("at" RMT+NLAUW, June 2022).
"Continuous Network Models for Sequential Predictions"

("at" JHU MIDS, January 2021).
"Toward combining principled scientific models and principled machine learning models"

("at" LLNL, November 2021).
"Practical Theory and Neural Network Models"

("at" IARAI, October 2021).
"Practical Theory and Neural Network Models"

("at" TOPML, April 2021).
"ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning"

("at" MIT (and elsewhere), September 2020).
"Dynamical systems and machine learning: combining in a principled way datadriven models and domaindriven models"

("at" NJIT (and elsewhere), September 2020).
"ContinuousinDepth Neural Networks"

("at" Fields Institute, August 2020).
"Determinantal Point Processes and Numerical Randomized Linear Algebra"

("at" Tel Aviv, April 2020).
"Statistical Mechanics Methods for Discovering Knowledge from Modern Production Quality Neural Networks"

(at ACMSIGKDD, August 2019).
"Why Deep Learning Works: Traditional and HeavyTailed Implicit SelfRegularization in Deep Neural Networks"

(at the ICML 2019 Workshop on Theoretical Physics for Deep Learning, June 2019).
"Why Deep Learning Works: HeavyTailed Random Matrix Theory as an Example of Physics Informed Machine Learning"

(June 2019, at UW's Physics Informed ML Meeting;
and also at the Oct 2019 IPAM Interpretable Learning in Physical Sciences Workshop).
"Why Deep Learning Works: Implicit SelfRegularization in Deep Neural Networks"

(at the SF Bay Area DMSIG Meeting, February 2019).
"Why Deep Learning Works: Implicit SelfRegularization in Deep Neural Networks"

(Sept 2018, at Simons' Institute 2018 Big Data RandNLA meeting;
and also at the Sept 2019 DIMACS RandNLA Workshop).
"Sampling for Linear Algebra, Statistics, and Optimization I"
and
"Sampling for Linear Algebra, Statistics, and Optimization II"

(Aug 2018, at Simons' Institute 2018 Big Data Bootcamp).
"Biomedicine & the Foundations of Data?"

(at the BD2K Guide to the Fundamentals of Data Science seminar series).
"Matrix factorizations at scale: a comparison of scientific data analytics on Spark and MPI"

(at the 2016 Dato Data Science Summit).
"Challenges in Multiresolution Methods for Graphbased Learning"

(at the NIPS Workshops, December 2015).
"Using Local Spectral Methods in Theory and in Practice"

(at the NIPS Workshops, December 2015).
"Linear and Sublinear Linear Algebra Algorithms: Preconditioning Stochastic Gradient Algorithms with Randomized Linear Algebra"

(at DIMACS, August 2015).
"Eigenvector localization and implicit regularization for largescale graphs and networked data"

(at CIMAT, March 2015).
"Locallybiased and semisupervised eigenvectors"

(at MMDS 2014, June 2014).
"Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments"

(at the SF Bay Area DMSIG Meeting, June 2, 2014).
"BIG Biomedicine and the Foundations of BIG Data Analysis"

(at the Big Data in BioMedicine Conference at Stanford University, May 2014).
"Randomized Matrix Algorithms and Largescale Scientific Data Analysis"

(at the 2014 ASE Conference, May 31, 2014).
"Eigenvector localization, implicit regularization, and algorithmic antidifferentiation for largescale graphs and networked data"
(or try also here for another version)

(at ICERM and IMA, May and April 2014, respectively).
"Randomized matrix algorithms and largescale scientific data analysis"

(at the University of Michigan, April 2014).
"Recent Results in Randomized Numerical Linear Algebra"
(or click here)

(at the NIPS 2013 Workshops, December 2013).
"Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments"

(at the Simons Big Data Workshop II, October 2013).
"Inputsparsity Time Algorithms for Embeddings and Regression Problems"

(at the Simons Big Data Workshop I, September 2013).
"Past, Present and Future of Randomized Numerical Linear Algebra:
Part I (PD) and
Part II" (MM)

(at the Simons Big Data Bootcamp, September 2013).
"Randomized Regression in Parallel and Distributed Environments"

(at the GraphLab Workshop, July 2013).
"Sensors, networks, and massive data"
or click
here

(at Kavli FoS, November 2012).
"Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments"

(at MMDS 2012, July 2012).
"Extracting insight from large networks: implications of smallscale and largescale structure"

(at the University of Marlyand, April 2012).
"Fast Approximation of Matrix Coherence and Statistical Leverage"

(at the NIPS Workshops, December 2011).
"Approximate computation and implicit regularization in largescale data analysis"
or click
here

(at the Workshop on Beyond WorstCase Analysis, September 2011).
"Linear Algebra and Machine Learning of Large Informatics Graphs"

(at the NIPS Workshops, December 2010).
"Geometric Tools for Identifying Structure in Large Social and Information Networks"

(90 minute version, tutorial at SAMSI Opening workshop, August 2010).
"Geometric Tools for Graph Mining of Large Social and Information Networks"

(3 hour version, tutorial at KDD 2010, July 2010).
"Community Structure in Large Social and Information Networks"

(at the Newton Institute in Cambridge, June 2010).
"Algorithmic and Statistical Perspectives on LargeScale Data Analysis"

(at the SF Bay Area DMSIG Meeting, February 2010).
"Community Structure in Large Social and Information Networks,"
in avi
or mpg (note: slow to download),

(at the 2009 IIT Kanpur Processing Massive Data Sets Workshop, December 2009).
"Statistical Leverage and Improved Matrix Algorithms"

(at the 2009 ICML Workshops, June 2009).
"Sampling Algorithms and Coresets for Lp Regression and Applications,"
in mpg (note: slow to download),

(at the 2006 IIT Kanpur Data Streams Workshop, December 2006).
"CUR Matrix Decompositions for Improved Data Analysis"

(at Johns Hopkins University, March 2006).
"Fast Monte Carlo Algorithms for Matrix Operations and Massive Data Set Analysis"

(at the 2005 IPAM Summer School, July 2005).
