This thesis studies musical anticipation, both as a cognitive process and design principle for applications in music information retrieval and computer music. For this study, we reverse the problem of modeling anticipation addressed mostly in music cognition literature for the study of musical behavior, to anticipatory modeling, a cognitive design principle for modeling artificial systems. We propose anticipatory models and applications concerning three main preoccupations of expectation: What to expect? How to expect? and When to expect? For the first question, we introduce a mathematical framework for music information geometry combining information theory, differential geometry, and statistical learning, with the aim of representing information content and gaining access to music structures. The second question is addressed as a machine learning planning problem in an environment, where interactive learning methods are employed on parallel agents to learn anticipatory profiles of actions to be used for decision making. To address the third question, we provide a novel anticipatory design for the problem of synchronizing a live performer to a pre-written music score, leading to Antescofo, a preliminary tool for writing of time and interaction in computer music. Despite the variety of topics present in this thesis, the anticipatory design concept is common in all propositions with the following premises: that an anticipatory design can reduce the structural and computational complexity of modeling, and helps address complex problems in computational aesthetics and most importantly computer music.