TechTalks from event: ICML 2011

Tutorial: Learning Kernels

  • Tutorial: Learning Kernels Authors: Corinna Cortes, Mehryar Mohri, Afshin Rostamizadeh
    Kernel methods are widely used in statistical learning. Positive definite symmetric (PDS) kernels implicitly specify an inner product in a Hilbert space where large-margin techniques are used for learning and estimation. They can be combined with algorithms such as support vector machines (SVMs) or other kernel-based algorithms to form powerful learning techniques. But the choice of the kernel, which is critical to the success of these algorithms, is typically left to the user. To limit the risk of a poor choice of kernel, in the last decade or so, a number of publications have investigated the idea of learning the kernel from data. Rather than requesting the user to commit to a specific kernel, which may not be optimal, in particular if the user's prior knowledge about the task is poor, learning kernel methods require the user only to supply a family of kernels. The task of selecting (or learning) a kernel out of that family is then reserved to the learning algorithm which, as for standard kernel-based methods, must also use the data to choose a hypothesis in the reproducing kernel Hilbert space (RKHS) associated to the kernel selected. This tutorial describes the main theoretical, algorithmic, and empirical results related to learning kernels obtained in the last decade, including recent progress in all of these aspects in the last few years. Our tutorial will also introduce the audience to software libraries and packages incorporating the implementation of several of the most effective learning kernel algorithms and indicate how to use these algorithms in applications to effectively improve performance. Learning kernel is a fundamental topic for kernel methods and machine learning in general. The question of selecting the appropriate kernel has been raised since the beginning of kernel methods, in particular for SVMs. Significant improvements in this area will both reduce the requirements from the users when applying machine learning techniques and help achieve better performance. Additionally, the methods used for learning kernels, including the formulation and solution to the optimization techniques, the algorithms, and the theoretical insights can be useful in other areas of machine learning, such as learning problems with data-dependent hypotheses, feature selection or feature reweighting, distance learning, transfer learning and many others. Finally, there are many interesting research questions in this area that have not been explored sufficiently yet. This tutorial will provide a convenient introduction to both standard and advanced material in this area, which will help interested researchers to investigate these questions.
  • Tutorial on Learning Kernels - Part 2 Authors: Corinna Cortes, Mehryar Mohri, Afshin Rostamizadeh

Tutorial : Collective Intelligence and Machine Learning

  • Collective Intelligence and Machine Learning: Part 1 Authors: Haym Hirsh, Rutgers University
    "Collective intelligence" refers to ways that information and communications technologies are bringing people and computing together to achieve outcomes that were previously beyond our individual capabilities or expectations. Google's search algorithms, Wikipedia's millions of articles, Amazon's recommendations, and open source software's multiple successes are prominent examples of ways in which technology and people are being brought together to exhibit behaviors that, collectively, are more intelligent than is possible by people or machines alone. Collective intelligence makes contact with machine learning in three ways. First, machine learning scholars and practitioners are using collective intelligence as an element in conducting their work, such as using crowdsourcing resources like Amazon Mechanical Turk to create corpora in computational linguistics or computer vision or to evaluate results in user interfaces or information retrieval. Second, existing techniques and new innovations in machine learning have become a key enabler of many examples of collective intelligence, such as mining consumer behaviors and product review sentiments to facilitate product recommendation. Finally, collective intelligence offers a provocative phenomenon to consider by those seeking to expand our ability to build computational systems that can be said to learn. This tutorial will survey the state of the art in collective intelligence from a machine learning perspective. First, it will discuss examples in which people explicitly serve as participants in collectively intelligent systems, such as editing Wikipedia articles, participating in the Netflix Challenge, identifying astronomical objects in GalaxyZoo, providing reviews and ratings on Amazon or TripAdvisor, or using Amazon Mechanical Turk to label images with tags. Second, it will present examples in which collectively intelligent outcomes arise through the computationally distilled wisdom of the behaviors and creations of individuals otherwise acting for their own, often unrelated purposes, as exhibited by Google's page ranking algorithm and Amazon's recommendation system. The tutorial will conclude with a discussion of prospects for the future.
  • Collective Intelligence and Machine Learning: Part 2 Authors: Haym Hirsh, Rutgers University

Tutorial: Machine Learning in Ecological Science and Environmental Policy