Mage 2.0: New features and its application in the development of a talking guitar

This paper describes the recent progress in our approach to generate performative and controllable speech. The goal of the performative HMM-based speech and singing syn- thesis library, called Mage, is to have the ability to generate natural sounding speech with arbitrary speaker's voice characteristics, speaking styles and expressions and at the same time to have accurate reactive user control over all the available production levels. Mage allows to arbitrarily change between voices, control speaking style or vocal identity, manipulate voice characteristics or alter the targeted context on-the-fly and also maintain the naturalness and intelligibility of the output. To achieve these controls, it was essential to redesign and improve the initial library. This paper focuses on the improvements of the architectural design, the additional user controls and provides an overview of a prototype, where a guitar is used to reactively control the generation of a synthetic voice in various levels.