Representation is a central issue in machine learning. Indeed, with good representations even simple algorithms can work, but even the most sophisticated machine learning algorithms will find it difficult to make up for a poor representation. Ideally, we would like the data representation itself to be the subject of learning, to avoid these difficulties. I will discuss some the challenges in learning useful representations and will give an overview of some of my work in discovering representations for image and text data, using methods from connectioninst deep learning to Bayesian nonparametrics.