- The beta negative binomial process, terrorist organizations, and cancer subtypes.
2012 May 17. AMPLab Spring 2012 Retreat, Santa Cruz, California, USA.
[abstract]
[slides pdf]
We develop a Bayesian nonparametric approach to a general family of latent class problems in which individuals can belong simultaneously to multiple classes and where each class can be exhibited multiple times by an individual. Taking a Bayesian approach, we start by defining a generative model for random data of this form; to that end, we introduce the beta negative binomial process (BNBP). We derive algorithms for inference using the BNBP generative model and Bayes Rule. We present experiments in which we use these algorithms to analyze documents describing international terrorist attacks. We outline ongoing work in applying our model to infer both sparse characterizations of different cancer subtypes and sparse representations of the cancer subtypes present in a single tumor from copy number variation (CNV) data.
- Nonparametric Bayesian priors for unsupervised machine learning.
2012 April 24. Berkeley-Stanford Graduate Student Colloquium, UC Berkeley.
[abstract]
[slides pdf]
In clustering, we partition data points into mutually exclusive, mutually exhaustive classes.
A generalization of clustering is to group data points into some finite collection of "features", each containing
an arbitrary subset of the data points. A further generalization is to group data points into some
finite collection of features with multiplicities, i.e. each feature can now belong some non-negative integer
number of times to each group. We briefly explore applications in each of these frameworks as well as practical
statistical models that allow us to learn the groupings when the number of groups is not known in advance.
- Some thoughts on a Bayesian approach to the multiple assignment problem.
2011 April 12. Berkeley-Stanford Graduate Student Colloquium, Stanford University.
[abstract]
[slides pdf]
In a cluster analysis, we're interested in learning both the assignment of data points to latent clusters, i.e. groups, and the number of groups represented in the data set. I will consider a generalization of clustering wherein each data point can be assigned to any non-negative, finite number of groups and give a brief introduction to a Bayesian nonparametric model for learning the latent group assignments of the data points and the number of groups.
- Fast and flexible selection with a single switch.
2009 December 10. Mini-Symposia on Assistive Machine Learning for People with Disabilities, NIPS 2009, Vancouver, BC, Canada.
[announcement]
[extended abstract, pdf]
[video]
- Treed Gaussian process models for classification.
2009 April 17. Statistics and Probability Session, Young Researchers in Mathematics 2009, Centre for Mathematical Sciences, University of Cambridge, UK.
[abstract]
Recognizing the successes of treed Gaussian process (TGP) models as an interpretable and thrifty model for nonstationary regression, we seek to extend the model to classification. Both treed models and Gaussian processes (GPs) have, separately, enjoyed great success in application to classification problems. An example of the former is Bayesian CART. In the latter, real-valued GP output may be utilized for classification via latent variables under M-1 regression GP (priors) for M classes which provide classification rules via a softmax function. This leads to two ways of combining trees with GPs for classification. We can partition the data set once and associate M-1 GPs to each region of the partition, or we can form M-1 separate full TGPs. We take the latter route in the interests of faster mixing and use a Bayesian model averaging scheme to traverse the full space of classification TGPs (CTGPs) via joint proposals for the tree topology and the GP parameters at the leaves. We explore schemes for efficiently sampling the latent variables, which is important to obtain good mixing in the expanded parameter space. Our proposed CTGP methodology is illustrated on a collection of synthetic and real data sets. We assess performance relative to existing methods and thereby show how CTGP is highly flexible, offers tractable inference, produces rules that are easy to interpret, and performs well out of sample.
- Treed Gaussian processes for classification.
2009 March 14. DIS2: Discrimination II Session, 11th International Federation of Classification Societies Conference 2009, Dresden University of Technology, Germany.
[abstract]
A Gaussian process (GP) is a popular nonparametric model for regression that specifies a prior over functions. For ease of computation, typical priors often confine the functions to stationarity. While stationarity is a reasonable assumption for many data sets, still many more exhibit only local stationarity. In the latter case, a stationary model is inadequate, but a fully nonstationary model is undesirable due to the intractability of inference. A treed Gaussian process (TGP), in contrast, can take advantage of local trends more efficiently. It defines a treed partitioning process on the predictor space and fits distinct, but hierarchically related, stationary GPs in the regions depicted at the leaves. The treed form of the partition makes the model particularly interpretable while, at the same time, yielding smaller matrices for inversion than would be required under a standard GP, thereby actually facilitating faster inference.
Recognizing the successes of TGP for regression, we seek to extend the model to classification. Both treed models and GPs, separately, have enjoyed great success in application to classification problems. An example of the first case is Bayesian CART. In the second case, real-valued GP output may be utilized for classification via latent variables; for some number of classes, the real-valued (latent) outputs of a commensurate number of GPs can yield classification rules via a softmax function. This leads to two ways of combining trees with GPs for classification. We can partition the data set once and associate multiple GPs to each region of the partition, or we can form separate, full TGPs for each class. Taking the latter route in the interests of faster mixing, we use a Bayesian model averaging scheme that traverses the full space of classification TGPs using joint proposals the for tree-topology and the GP parameters at the leaves. We explore schemes for efficiently sampling the latent variables, which is important to obtain good mixing in the expanded parameter space.
Our proposed classification-TGP methodology is illustrated on a collection of synthetic and real data sets. We assess performance relative to existing methods and thereby demonstrate that our approach is highly flexible, offers tractable inference, produces rules that are easy to interpret, and performs well out of sample.