Visualizing Structures of Speech Expressiveness

By thisisuser on October 23, 2008

Download PDF

Graugaard08_VisualizingStructuresOfSpeechExpressiveness

“Speech is both beautiful and informative. In this work, a conceptual study going through the myth of the tower of Babel and hypothesis on the apparition of articulated language is undertaken in order to create an artistic work investigating the nature of speech. Our interpretation is that a system able to recognize archetypal phonemes through vowels and consonants could be used with several languages to extract the expressive part of speech independently from the meaning of words. A conversion of speech energy into visual particles that form complex visual structures provides us with a mean to represent this expressiveness of speech into a visual mode. This system is presented in an artwork whose scenario is inspired from various artistic and poetic works. The performance was presented at the Re:New festival in May 2008.”

Funded by Aalborg University and Danish Center for Design Research.

List of Writings

Sonification of Emotion State In Family-Run Businesses
Paper (2008)
Visualizing Structures of Speech Expressiveness
Paper (2008)
Gesture and Emotion in Interactive Music: Artistic and Technologial Challenges
PhD dissertation (2006)
Applying a Performer’s Physical Gestures to Sound Synthesis in Real-Time
Paper (2006)
Implicit Relevance Feedback in Interactive Music: Issues, Challenges, and Case Studies
Paper (2006)
Sound Synthesis Affected by Physical Gestures in Real-Time
Poster (2006)
Mood Mapping Technologies within Hybrid Audio Design
Poster (2006)
Providing Rhythm Patterns in Sound Synthesis
Paper (2006)
Mood Mapping Technologies within Hybrid Design
Presentation (2006)
Unifying Performer and Accompaniment
Paper (2005)
The SoundGlove II: Using sEMG Data for Intuitive Audio and Video Affecting in Real Time
Paper (2005)
Open and Closed Form in Interactive Music
Paper (2005)

My main areas of research is advanced music composition, real-time music technology and music cognition. I am particularly interested in the relationship between core emotions and score notation and performance features, as well as novel methods for generative real-time performance that are founded in non-expert music cognition.

Some years ago I was involved in the Nordic SUM project – Systematic Understanding of Music – and a large part of the project concerned the implementation into computer code of concepts from probabilistic melody generation for real-time performance.

This work has since found its way into several commercial releases and is today a constant presence in my live laptop performances, either in duo or small group formats with a variety of instrumentalists or as a solo performer.