Speaking with Hands: Creating Animated Conversational Characters from Recordings of Human Performance Matthew Stone Rutgers University Friday, September 10 2:30PM Room 102 Weaver Hall 251 Mercer Street NYU, NYC, NYS ABSTRACT -------- People's utterances in conversation are composed of short, clearly-delimited phrases; in each phrase, gesture and speech go together meaningfully and synchronize at a common point of maximum emphasis. In our work on RUTH, the Rutgers University Talking Head, we rely on this structure to create general-purpose annotation and synthesis to animate an expressive talking face. In our ongoing work, we also exploit this structure in methods to create animated conversational characters using databases of recorded speech and captured motion. We start with tools that help create scripts for performers, help annotate and segment performance data, and construct specific messages for characters to use within application contexts. Our animations then reproduce the structure of natural utterances. They recombine motion samples with new speech samples to recreate coherent phrases, and blend segments of speech and motion together phrase-by-phrase into extended utterances. By framing problems for utterance generation and synthesis so that they can draw closely on a talented performance, our techniques support the rapid construction of animated characters with rich and appropriate expression.