Congratulations to Yiting Wang for passing her dissertation defense on Friday, November 15, 2024!
Title of Dissertation: Multimodal Analysis: The Performativity of Short-Form Videos and Their Affordances
Abstract: With short-form videos being ubiquitous, making sense of their performativity as well as the potential relationship (affordances) between people and short videos are imperative. Studies of social media from a performative perspective are increasing. However, two aspects are yet to be updated: a robust research method to study the emerging visual phenomenon, and a holistic system to explain short-form videos. Multimodal analysis, a method that pays attention to visual sense-making, comes in handy for researching visual and media literacy.
Previous peer-reviewed and published pilot studies showed that short-form videos are theatrical. Building on this, this dissertation probes further, with a broader frame of reference that is performative, to understand the interactions between technology and people. The objective of this qualitative study was to study how the design of short video apps impacts people’s creation, and the performative activities afforded by the platform. The goal is to enrich media literacy through multimodal communication.
This research asks three questions: What affordances do short-form videos as a medium offer? How do users use these affordances in user-generated short-videos to create a performance? Among different communicative forms, do users’ use of affordances differ? Using walkthroughs, multimodal analysis, and several procedures from grounded theory, I explored how affordances of short-form videos are multi-layered, and the relationships between the layers.
The results revealed 5 performative devices, 3 steps of video analysis, and 2 techniques to apply when determining modes. Results demonstrated three layers of affordances reflected by modes such as facial expressions (low-level); spatial relationships (intermediate level); and five performative devices (high-level). Results also suggest that behaviors of performativity are universal across three platforms, and that many principles applicable in still images work the same in short videos.
The findings contribute to both the method level and the theory level of digital sense-making and media literacy. The dissertation adds onto ways of looking at short videos effectively and informs creators and users ways their creations can be looked at. It connects with the current discourse of disinformation and misinformation, the harm “harmless” humor can make, and media multitasking issues. As interdisciplinary research, this study bridges disciplines and contributes to the fields of media and communication studies, performance studies, and human computer interaction.
Committee Members:
Dr. Daniel Suthers (CIS) – Chairperson
Dr. Wayne Buente (CIS)
Dr. Rich Gazan (CIS)
Dr. Scott Robertson (CIS)
Dr. Markus Wessendorf (Department of Theatre & Dance) – University Representative