In recent years, there has been a growing interest in algorithms that learn and use continuous representations for words, phrases, or documents in many natural language processing applications. Among many others, influential proposals that illustrate this trend include latent Dirichlet allocation, neural network based language models and spectral methods. These approaches are motivated by improving the generalization power of the discrete standard models, by dealing with the data sparsity issue and by efficiently handling a wide context. Despite the success of single word vector space models, they are limited since they do not capture compositionality. This prevents them from gaining a deeper understanding of the semantics of longer phrases, sentences and documents. Regarding this issue, some pertinent questions arise: should word/phrase/sentence representations be of the same sort? Could different linguistic levels require different modelling approaches ? Is compositionality determined by syntax, and if so, how do we learn/define it? Should word representations be fixed and obtained distributionally, or should the encoding be variable? Should word representations be task-specific, or should they be general?