Please join us Monday, 4/18 (4:30pm to 5:30pm), for a talk by CIS student Yiting Wang in advance of her upcoming dissertation proposal: “Multimodal analysis: The performativity of short-form videos.”
Abstract:
With social media becoming visual and with more ubiquitous short-form videos, making sense of such visual communication is imperative. In other words, a robust research method to study short-form videos, and a holistic system to explain short-form videos are needed. Previous peer-reviewed and published pilot studies show that short-form videos are performative, and can be explained by three theatrical devices: situation, suspense, and mimesis. Building on this, this dissertation probes further, to understand the interactions between technology and people.
I ask, what affordances do short-form videos as a medium offer? How do users use these affordances in user-generated short-videos to create a performance? Finally, among different communicative forms, do users’ use of affordances differ? To answer these questions, I will first walk through short-form video platforms (TikTok, YouTube Shorts, Likee, and Snapchat) to familiarize with the mediums, and then compile an affordance catalog. Following this, I will use the core method of multimodal analysis to investigate the modes used to manipulate understanding: speech, caption, props, color choices, and costuming, to name a few, within videos collected from the aforementioned platforms.
Following a five-step process, videos are broken into a sequence of images, then transcribed into written cues. After annotating and analyzing each individual mode, I analyze across modes, to eventually discover and connect with social theories. Exercising and refining multimodal analysis, this dissertation aims to find other devices on top of the theatrical ones. In addition, using three criteria of affordances in the literature, I will further test the multi-layered fashion of affordances, that is commonly observed but rarely proven, in order to complete the affordance equation by adding the last outcome: audiencing, or audience engagement. Specifically, I explore how affordances of short-form videos are multi-layered, and that the outcomes of affordances at one level become affordances at a new level, with audiencing as the ultimate outcome. Bridging various disciplines, this dissertation aims to contribute to the field of human computer interaction, media and communication studies, and performance studies.
Bio:
Yiting Wang is a 4th year CIS PhD student. Her research interests cover visual communication, specifically short videos, and performative communications on social media. She is in an early stage of exploring human computer interaction. Prior to her doctoral program, she received her communication degrees from the National Academy of Chinese Theatre Arts in Beijing.