Learning algorithms for structured output prediction

Many real life applications correspond to problems that cannot be described as the classification of an input into one of a few categories or classes. These include tasks like image labeling, machine translation, information retrieval, speech recognition, and many others. Indeed, the target to be predicted for such problems is typically very high dimensional and structured.

I'm particularly interested in developing general structured output learning algorithms for highly non-linear models, as opposed to linear ones such as structured SVMs and CRFs. In Conditional Restricted Boltzmann Machines for Structured Output Prediction, Volodymyr Mnih, Geoffrey Hinton and I investigated two such approaches for training RBMs on structured output prediciton problems.

I'm also interested in algorithms that better take into account the loss under which the model is evaluated. Such loss functions can have as complex a structure as the output itself. In Loss-sensitive Training of Probabilistic Conditional Random Fields, Maksim Volkovs, Richard Zemel and I proposed and evaluated several different learning algorithms for doing that, focusing on the document ranking problems.

References


  • Loss-sensitive Training of Probabilistic Conditional Random Fields [arxiv]
    Maksim Volkovs, Hugo Larochelle and Richard Zemel,
    arXiv, 2011

  • Conditional Restricted Boltzmann Machines for Structured Output Prediction [pdf]
    Volodymyr Mnih, Hugo Larochelle and Geoffrey Hinton,
    Uncertainty in Artificial Intelligence, 2011

  • Autotagging music with conditional restricted Boltzmann machines [arxiv]
    Michael Mandel, Razvan Pascanu, Hugo Larochelle and Yoshua Bengio,
    arXiv, 2011