Gesture

How are we to understand digital objects? How are we to relate ‘cyberspace’ to physical space? This chapter attempts to provide a set of theoretical tools to understand ‘spaces’ of online interaction and what happens within them without resorting to filamentous constructions of ‘disembodied’ online interaction or to the underlying idealistic Cartesian dualism that pervades many of the theoretical positions that ostensibly refute it. To do so, I make connections between cognitive processes of human subjectivity, the embodied, gestural enactments of physical social spaces, and the social interactions that take place in online environments. The crux of the argument is that like identity, meaning and subjectivity are social phenomena: individual cognition requires social interaction. Similarly, social interaction, mediated or immediate, defines our spaces of subjectivity. This connecting of online ‘spaces’ to embodied cognition may provide a way to understand digital objects and the online interactions they enable through a reconsideration of the concept of space.

“‘The Underlying Gesture’. Towards the Notion of Gesture in Jean d’Udine and Sergei Eisenstein”. In: Rossella Catanese, Francesca Scotto Lavina, Valentina Valente (eds.), From Sensation to Synaesthesia in Film and New Media. Cambridge Scholars Publishing 2019, 102-115.

In his keynote address, ‘Hearing the Nations in Chopin’, Jim Samson reflected on the three key ingredients of musical nationalism: agendas, musical materials, and appropriations. These three areas do not only bear relevance for Chopin reception and nineteenth-century European nationalism, but also in the context of the conference as a whole effectively frame the broader themes of nationalism in music which emerged from this three-day international gathering. My paper is more closely allied to the categories of ‘musical materials’ and their ‘appropriation’ in relation to the expression of Norwegian national cultural identity in the performance of Edvard Grieg’s later piano music. Focusing predominantly on the Slåtter, op. 72, I examine the interplay between the two aforementioned categories in recent (re)constructions of cultural identity through an empirical performance-analytical study of two pieces. This paper develops and extends my research on the performance practice of this repertoire. Building on both existing and new empirical findings, I investigate experiential and phenomenological aspects of rhythmic gesture in the musical performance of this repertoire, and the effect these have for the cultural performance of an alleged authentic Norwegian folk identity. I conclude by reflecting on the performativity of rhythmic gesture as a material of national-cultural identity.

This dissertation presents a new and expanded context for interactive music based on Moore’s model for computer music (Moore 1990) and contextualises its findings using Lesaffre’s taxonomy for musical feature extraction and analysis (Lesaffre et al. 2003). In doing so, the dissertation examines music as an expressive art-form where musically significant data is present not only in the audio signal but also in human gestures and in physiological data. The dissertation shows the model’s foundation in human perception of music as a performed art, and points to the relevance and feasibility of including expression and emotion as a high-level signal processing means for bridging man and machine. The resulting model is multi-level (physical, sensorial, perceptual, formal, expressive) and multi-modal (sound, human gesture, physiological) which makes it applicable to purely musical contexts, as well as intermodal contexts where music is combined with visual and/or physiological data.

The model implies evaluating an interactive music system as a musical instrument design. Several properties are examined during the course of the dissertation and models based on acoustic music instruments have been avoided due to the expanded feature set of interactive music system. A narrowing down of the properties is attempted in the dissertation’s conclusion together with a preliminary model circumscription. In particular it is pointed out that high-level features of real-time analysis, data storage and processing, and synthesis makes the system a musical instrument, and that the capability of real-time data storage and processing distinguishes the digital system as an unprecedented instrument, qualitatively different from all previous acoustic music instrument. It is considered that a digital system’s particular form of sound synthesis only qualifies it as being of a category parallel to the acoustic instruments categories.

The model is the result of the author’s experiences with practical work with interactive systems developed 2001-06 for a body of commissioned works. The systems and their underlying procedures were conceived and developed addressing needs inherent to the artistic ambitions of each work, and have all been thoroughly tested in many performances. The papers forming part of the dissertation describe the artistic and technological problems and their solutions. The solutions are readily expandable to similar problems in other contexts, and they all relate to general issues of their particular applicative area.

While human-human or human-object interactions involve very rich, complex and nuanced gestures, gestures as they are captured for human-computer interaction remain relatively simplistic. Our approach is to consider the study of variation of motion input as a way of understanding expression and expressivity in human-computer interaction and in order to propose computational solutions for capturing and using these expressive variations. The paper reports an at- tempt at drawing the lines of design guidelines for modelling systems adapting to motion variations. We propose to il- lustrate them through two case studies: the first model is used to estimate temporal and geometrical motion variations while the second is used to track variations of motion dynamics. These case studies are illustrated in two applications.

This book explores how digital technology is altering the relationships between people and how the very nature of interface itself needs to be reconsidered to reflect this – how
we can make sense of each other, handle ambiguities, negotiate differences, empathise and collectively make skilled judgments in our modern society. The author presents new directions for research at the relational-transactional intersection of contrasting disciplines of arts, science and technology,and in so doing, presents philosophical and artistic questions for future research on human connectivity in our digital age.

▶ Introduces the relational interface, which questions the historical idea of the interface as a conduit of the transactional and addresses an aesthetic and ethical balance
▶ Formulates tacit engagement as an emerging concept for the study of human relations
▶ Proposes a fundamental framework of mediation for expertise and decision making in complex human systems
▶ Critiques the concept of data and what makes for the success of knowledge transfer

Vera Kotelevskaya’s Review of Susanne Strätling’s Book "Die Hand am Werk: Poetik der Poiesis in der russischen Avantgarde". München: Wilhelm Fink Verlag 2017, 531 S., ISBN: 978-3-7705-6092-9

Ключевые слова: Велимир Хлебников; Всеволод Мейерхольд; Сергей Третьяков; русский авангард; русский формализм; русский конструктивизм; биомеханика; показ/рассказ; гаптическая поэтика; поэтология; карпалистика; жест; рука; интермедиальность

Keywords: Russian avant-garde; Russian formalism; haptic poetic; poetology; gesture; hand; Daniil Kharms; Alexey Gastev; Velimir Khlebnikov; Petr Miturich; Vsevolod Meyerhold; Sergey Tret’iyakov

Schlagwörter: Russische Avantgarde; Russischer Formalismus; haptische Poetik; Poetologie; Geste; Hand; Daniil Charms; Alexey Gastev; Welimir Chlebnikow; Petr Mituritsch; Wsewolod Meyerhold; Sergey Tret’ijakov

Cet article présente une étude de cas de l’expression de la négation jusqu’à 3 ans chez quatre enfants évoluant dans des environnements langagiers différents : majoritairement monolingue (anglais, français, LSF) ou bilingue (LSF-français), majoritairement unimodal (modalité visuo-gestuelle de la LSF) ou bimodal (visuo-gestuelle et audio-vocale). Nous décrivons l’émergence et le développement d’une expression gestuelle multimodale, selon que la langue cible intègre (LSF) ou non (anglais, français) cette gestuelle dans sa grammaire, et explorons ainsi l’interface geste-signe. Nous nous attardons sur deux formes gestuelles présentes dans les productions de quatre enfants pour en déterminer les invariants kinésiologiques sous forme de « patrons gestuels contrastifs » et nous analysons la façon dont ces patrons dessinent des sous-ensembles sémantiques dans l’expression gestuelle de la négation et son acquisition.

This edited collection is concerned with talk and activity inside cars, with examining the interior of a car as socially rich and meaningful. The papers here are concerned with both social interaction as the intertwining of multiple modalities of language, the body, and artefacts of the interior material world of the car, as well as with how the car’s movement in time and space through particular external physical surroundings contributes to, or is accomplished by, social interaction. Methodologically, the papers are primarily informed by principles and insights of conversation analysis, ethnomethodology, and multimodal interaction analysis.

In this collection the car is considered as a ‘place’ or ‘space’ for meaningful and mediated activities. The papers examine how the physical and spatial configuration of the car, and its possibilities for mobility, can constrain or afford particular interactional practices, social activities and understandings, and impact upon language and processes of interaction. Interaction in cars creates particular demands, opportunities and orientations for its participants, as the car moves through the semiotically rich external environment. Generally, the papers consider driving as not merely a requisite competence for accomplishing travel from point A to point B, but as occurring itself as a situated activity that is integrated with ordinary conversation.

Most research on driving and automobility follows one of two broad directions: driving safety research; and the social and cultural meanings of automobility and driving. Driving safety research, on the one hand, is mostly dominated by studies conducted within a psychological and cognitive scientific framework. In social sciences and human geography, on the other hand, such dominant approaches to driving and drivers have increasingly been critiqued for undermining the cultural and sociological meanings of automobility and driving. Consequently, with the emergence of ‘mobility studies’, scholars have begun to study the social, cultural, and ideological meanings and discourses of the car, car cultures, driving cultures, driving practices, automobility, road systems, and traffic systems, as parts of modern life.


The papers in this special issue can therefore contribute to interdisciplinary dialogue between social scientific and psychological driving research. Further empirical knowledge is needed of how people organize their talk and embodied activities for social activity in cars, and relative to the contingencies of the driving situation and the physical and spatial layout of the car. By drawing on a specific empirical research methodology that relies on recorded data collected from real-life situations inside the car, the papers of this collection can add to the important interdisciplinary discussion that surrounds driving, safety and automobility, and provide food for thought by addressing central issues, raising questions, and perhaps even by providing some answers.

The chapters are:

Pentti Haddington, Maurice Nevile, and Tiina Keisanen
'Meaning in motion: Sharing the car, sharing the drive'

Eric Laurier, Barry Brown, and Hayden Lorimer
'What it means to change lanes: Actions, emotions, and wayfinding in the family car

Pentti Haddington
'Movement in action: Initiating social navigation in cars'

Maurice Nevile
'Interaction as distraction in driving: A body of evidence'

Tiina Keisanen
' “Uh-oh, we were going there”: Environmentally occasioned
noticings of trouble in in-car interaction'

Lorenza Mondada
'Talking and driving: Multi-activity in the car'

Marjorie Harness Goodwin and Charles Goodwin
'Car talk: Integrating texts, bodies, and changing landscapes'

Elizabeth Keating and Gene Mirus
'The eyes have it: Technologies of automobility in sign language'

Chaim Noy
'Inhabiting the family-car: Children-passengers and parents-drivers on the school run

E. Grishina. Multimodal Russian Corpus (MURCO): Studying Emotions // 7th Conference on Language Resources and Evaluation LREC’2010, Valetta, Malta (workshop 23/05/10 Corpora for Research on Emotions and Affects) http://www.lrec-conf.org/proceedings/lrec2010/index.html

"The paper introduces the Multimodal Russian Corpus (MURCO), which has been created in the framework of the Russian National Corpus (RNC). The MURCO provides the users with the great amount of phonetic, orthoepic, intonational information related to Russian. Moreover, the deeply annotated part of the MURCO contains the data concerning Russian gesticulation, speech act system, types of vocal gestures and interjections in Russian, and so on. The access to The Corpus is on free access. The paper describes the main types of annotation and the interface structure of the MURCO.

"

This paper compares the opening sequences of Finnish, French and Hungarian YouTube videos. We concentrate on addressing, greeting and related gestures, important elements when YouTubers interact with their imagined viewers, using data consisting of 138 videos in the three languages. This study falls within the field of the pragmatics of social media and interpersonal pragmatics, and data were analysed using multimodal discourse analysis. Shared practices included the frequent use of greetings, a preference for general nominal address forms as well as for iconic and deictic gestures. Cross-cultural differences revealed that Finnish and Hungarian were closer to each other than to French. Shared practices may be connected to the genre of YouTube videos, whereas differences appear related to cross-cultural practices generally.

Becoming commercially available in the mid 1960s, video quickly became integral to the intense experimentalism of New York City's music and art scenes. The medium was able to record image and sound at the same time, which allowed composers to visualize their music and artists to sound their images. But as well as creating unprecedented forms of audiovisuality, video work also producedinteractive spaces that questioned conventional habits of music and art consumption. This book explores the first decade of creative video work, focusing on the ways in which video technology was used to dissolve the boundaries between art and music.

In the last few years, affect has entered the field of politics through both the media and political actors themselves. This article is based on the hypothesis that the images of affect that illustrate contemporary political narratives in the media reflect different visual motifs which are linked to an iconographic tradition that began in the art world. Through the iconographic analysis of images published in the three newspapers in Spain with the largest general readership (El País, El Mundo, and La Vanguardia) from 2011 to 2017, this article identifies three visual motifs of affect: the handshake, the hug, and the kiss. The study of these motifs highlights the survival of certain forms of emotion that are easily recognisable and engage with a longstanding iconographic tradition. At the same time, the article inquiries into the meanings derived from these iconographies and their implications on the representation of politics and of political leaders. The proliferation of motifs of ...

An increasing number of today's consumer devices such as mobile phones or tablet computers are equipped with various sensors. The extraction of useful information such as gestures from sensor-generated data based on mainstream imperative languages is a notoriously difficult task. Over the last few years, a number of domain-specific programming languages have been proposed to ease the development of gesture detection. Most of these languages have adopted a declarative approach allowing programmers to describe their gestures rather than having to manually maintain a history of event data and intermediate gesture results. While these declarative languages represent a clear advancement in gesture detection, a number of issues are still unresolved. In this paper we present relevant criteria for gesture detection and provide an initial classification of existing solutions based on these criteria in order to foster a discussion and identify opportunities for future gesture programming languages.

El área de los estudios sobre gestualidad tiene importantes implicaciones para la perspectiva descriptiva de la lengua, entre ellas, que gesto y habla (o gesto y señas) forman una unidad esencial para la expresión lingüística en todos los niveles y, por lo tanto, habría que reconsiderar la adecuación descriptiva de los recursos conceptuales que asumen la división entre estos dos componentes de la comunicación. En el presente artículo, se propone la revisión de la contribución metodológica de los estudios gestuales, para presentar un breve análisis de la llamada 'marcación de tópico' en la LSM que muestra las ventajas de aplicar esta perspectiva y nos permite reevaluar el análisis de los recursos comunicativos de esta lengua. / The fast growing area of gesture studies has shown important implications for the perspective used in the description of language, among them, the fact that gesture and speech (or gesture and sign) form an essential unit for linguistic expressions on every level. Because of this, we must look closely at the descriptive adequacy of the conceptual notions that assume the split between this fundamental components of communication. In this paper, an overview of the methodological contribution of gesture studies is presented to set things up for a brief but concise analysis of so called 'topic-marking' in LSM that shows the advantages of assuming this perspective and allow us to reassess the analysis of the communicative resources of this language.

My book, Digital Prohibition: Piracy and Authorship in New Media Art, was published by Continuum International Publishers (named the top academic press in Britain this year) in 2012. It starts from the premise that the new changes in copyright law are functioning as a new Prohibition, that is to say as something that it so unpopular, so untenable and so unenforceable that it is only a matter of time before public backlash destroys the current corporate monopoly on creativity. This is the first book to discuss the global politics of creative work and emergent models of authorship in a digital age. The book examines the creation of new media forms by artists and groups who use technology to challenge established models of creation. It starts from the premise that creativity is no longer a useful concept in an age of data glut and perfect copies; instead we must now think of creative practice as a kind of creative critique that repurposes existing materials in order to explore the nature of media and how they affect us. It does this through three different aesthetic approaches: interruption (stoppage and repetition), disturbance (critique and event), and capture/leakage (performance and documentation). The book is wide-ranging in its definition of authorship, exploring methods as diverse as sampling, mashups, hacktivism, machinima, social media, tactical media, productive mistranslation, and digital anthropophagy. It includes extensive research in Asia (Japan, Indonesia, Thailand, India, and especially China and Pakistan) to see how and what creative practices flourish when they are not fettered by copyright regulations.

"My inquiry will focus on an attempt to locate an ethos of music education. Because I acknowledge that the ethos of a society must logically precede a study of ethics (Ryan, 1972, p. 291, Miller, 1974, p. 309), and given the dramatic changes over the past 30 years in the ways that we create, record, store, access, and perform music, much of my exploration will be positioned in an examination of music and new digital media in the 21st century and how it is used by countless adolescents. Thus, in the development of my line of rhetoric I will (1) begin with basic Aristotelian (1984, 1999) notions of art making, artistic activity, and ethos, (2) discuss the relationship between téchne and technology, and (3) téchne and logo, (4) briefly describe the phenomenon of music making in the digitally enriched 21st century, (5) attempt to discover an ethos of music and music education in the 21st century, and (6) based on the possible discovery of an ethos, determine the ethical outcomes regarding curriculum and pedagogy in music education.

I argue that that the profession has not been responsible in the ways that we have constructed an ethos—any notion of ethos—over the past 80 years, in music education textbooks, music curricula, research, and position documents, and in the ways that we (re)present reasons for why music belongs in the schools. Our profession seems to be grasping at straws in our advocacy for ‘justification’ and ‘academic positioning,’ while we may merely need to identify the individuals and communities of people whose lives have been changed by their experiences with music and enlist them to help us (re)define and refine our role not only in education but in society. At best, we may have an ethos that is antiquated and not related to 21st century youth, and most catastrophically, no sensibility regarding the music identity of the majority of youth in digitally enabled nations. It seems that we possess no center, no unified voice, no sense of music of the 21st century, and no sense of “knowing our market” (see Rideout et al, 2010). We need to (re)elaborate what is perceived as ‘music activity,’ in and out of schools and in structured and unstructured learning situations (Gouzouasis & Bakan, 2011). Moreover, we are not speaking to the very people who can understand the need for music education in the schools—the youth of today, who will become the creators and consumers of music and policy makers of tomorrow. Thus, this paper questions how we can create an ethos of music education for our profession without including the individuals who are being ‘educated’ by our notions of music education—our youth themselves.
"

Safe interaction with non-expert users is increasingly important in the development of robotic assistants. Ethical “codes” can serve as a guide as to how this interaction should take place with lay users in non-structured environments. Such codes suggest that robots should behave in a way that is intuitive to users. Previous research has demonstrated that the implicit channel is useful for intuitive human-robot interaction. Our work described in this position paper investigates how a robot should behave when it is uncertain of its human partner’s intentions. In this context, uncertainties arising in human-robot shared-tasks should be made transparent to a human user. We posit that hesitant hand motion used by people and animals is a natural modality for a robot to communicate uncertainty. To test our hypothesis we propose to characterize and implement human hesitation gestures onto a robot, and investigate its ability to communicate uncertainty.

The paper presents a framework for classifying gestures in terms of different parameters, and shows that the parameter of iconicity cuts across that of cognitive construction, which distinguishes codified gestures — those represented in memory as stable signal–meaning pairs — from creative ones — those invented on the spot on the basis of a few generative rules. While creative gestures are necessarily iconic, because they can be understood only thanks to their iconicity, codified gestures can be iconic too. A model for the generation of iconic gestures is presented, according to which, to create a new gesture, people select the features of the meaning to imitate in terms of their distinctiveness, ease of representation and ontological type. Finally, some principles for measuring the iconicity of gestures of Hearers and Signs of the Deaf are illustrated, thus outlining the continuity of iconic devices between creative and codified gestures.