Learning algorithms for structured output prediction
Many real life applications correspond to problems that
cannot be described as the classification of an input into
one of a few categories or classes. These include tasks
like image labeling, machine translation, information
retrieval, speech recognition, and many others. Indeed,
the target to be predicted for such problems is typically
very high dimensional and structured.
I'm particularly interested in developing general
structured output learning algorithms for highly
non-linear models, as opposed to linear ones such as
structured SVMs and CRFs.
In
Conditional
Restricted Boltzmann Machines for Structured Output
Prediction,
Volodymyr
Mnih,
Geoffrey
Hinton and I investigated two such approaches for training
RBMs on structured output prediciton problems.
I'm also interested in algorithms that better take into
account the loss under which the model is
evaluated. Such loss functions can have as complex a structure as the
output
itself. In
Loss-sensitive
Training of Probabilistic Conditional Random Fields,
Maksim
Volkovs,
Richard
Zemel and I proposed and evaluated several different
learning algorithms for doing that, focusing on the document ranking
problems.
References
- Loss-sensitive Training of Probabilistic Conditional Random Fields [arxiv]
Maksim Volkovs, Hugo Larochelle and Richard Zemel,
arXiv, 2011
- Conditional Restricted Boltzmann Machines for Structured Output Prediction [pdf]
Volodymyr Mnih, Hugo Larochelle and Geoffrey Hinton,
Uncertainty in Artificial Intelligence, 2011
- Autotagging music with conditional restricted Boltzmann machines [arxiv]
Michael Mandel, Razvan Pascanu, Hugo Larochelle and Yoshua Bengio,
arXiv, 2011