Visualizing Structures of Speech Expressiveness

Download PDF


“Speech is both beautiful and informative. In this work, a conceptual study going through the myth of the tower of Babel and hypothesis on the apparition of articulated language is undertaken in order to create an artistic work investigating the nature of speech. Our interpretation is that a system able to recognize archetypal phonemes through vowels and consonants could be used with several languages to extract the expressive part of speech independently from the meaning of words. A conversion of speech energy into visual particles that form complex visual structures provides us with a mean to represent this expressiveness of speech into a visual mode. This system is presented in an artwork whose scenario is inspired from various artistic and poetic works. The performance was presented at the Re:New festival in May 2008.”

Funded by Aalborg University and Danish Center for Design Research.

My main areas of research is advanced music composition, real-time music technology and music cognition. I am particularly interested in the relationship between core emotions and score notation and performance features, as well as novel methods for generative real-time performance that are founded in non-expert music cognition.

Some years ago I was involved in the Nordic SUM project – Systematic Understanding of Music – and a large part of the project concerned the implementation into computer code of concepts from probabilistic melody generation for real-time performance.

This work has since found its way into several commercial releases and is today a constant presence in my live laptop performances, either in duo or small group formats with a variety of instrumentalists or as a solo performer.