Connect

Zoom's gesture controls redefine virtual communication, providing a hands-free experience that effortlessly enhances your online meetings.

Enhance your virtual meetings with our gesture control feature, which allows seamless interaction with Zoom. Effortlessly mute, navigate, and manage presentations without traditional input devices, transforming your meetings into a dynamic and intuitive experience.

Project Overview 

Gestural interfaces enable users to interact with digital devices directly without the need for physical hardware, enabling them to easily use their fingers, hands, arms, and even their whole bodies to control UI elements. Compared with tapping keys on a keyboard, gestural interfaces can include a much wider range of actions and sometimes feel so natural.
The primary goal is to create a gestural interface for virtual meetings that enhances communication and user experience. The project aims to improve the efficiency of virtual meetings by reducing the reliance on traditional input devices and making the interaction more intuitive.

Skills

Design Research
Prototyping
UI Design
User Flow
Usability Testing
Wireframing
Experience Design
Green Screen Filming

Tools

Project Timeline

Adobe Premier Pro
Adobe After Effects
Figma 
Notion
Adobe XD
August - October 2023  
(8 weeks) 
Individual (User Experience Designer)

Academic Project

The Problem

The introduction of gesture controls on Zoom, as a solution to these challenges, offers users the freedom to effortlessly switch perspectives, capture important moments, and enhance collaboration with the natural ease of intuitive gestures, which would make the virtual meeting experience more intuitive and responsive.
In the dynamic landscape of virtual meetings on the Zoom app, the conventional reliance on buttons and keyboard commands presents a formidable challenge.
As conversations seamlessly transition between participants and discussions, the need for intuitive gestures becomes paramount. 
Navigating between views while actively engaged in conversation, capturing screenshots on the fly, and annotating on-screen during collaborative sessions demand a level of fluidity that traditional controls struggle to deliver.

Who all are facing this problem? 

Collaborating with a team on a single device can be challenging, limiting their interactions with a single digital surface.
Proximity to devices is frequently required for real-time interface interaction, causing strain.
Users need to multitask while working, which hinders their ability to interact with the screen consistently during meetings.

Introducing Version 5.16.10

Zoom Connect 

Zoom Connect represents the latest advancement in the Zoom platform, incorporating innovative gesture-controlled features that respond to in-air gestures, providing a touchless virtual meeting experience. 
This enhancement aims to reduce meeting-related stress, allowing you to maintain focus on the conversation without the need for physical interaction with devices.
In this project, I designed a gestural interface for Zoom on an iPad. The idea is to give the user an app's functionality while leaving their hands free to engage with other objects and spaces around them and also, producing a gestural language that is learnable, via an app that teaches the user how to use gestures.

1. Update the Zoom App 

Upon logging in, users are directed to update the Zoom version with a specialized gestural control interface, which follows along with an onboarding process that helps them learn and adapt to the new gestural commands to use in virtual meetings.
It seeks to overcome some of the limitations of traditional videoconferencing tools by introducing a novel way to convey non-verbal cues and interact in a more lifelike manner.
🧑🏽‍💻 The users must update their existing Zoom to Version 5.16.10 to enable the new gestural commands. After the update, users have an option to switch the feature on and off at their convenience.

2. Onboarding 

Guiding users through the onboarding process, assisting them in practicing gestures, and aiding their comprehension of the interface's echo and semantic feedback.

The user who updated their Zoom version to 5.16.10, takes a mock meeting to learn the twelve gestures before proceeding to take virtual meetings with an enabled gesture-controlled setting.
For the first-time user experience, Zoom Connect, the updated version will introduce the users to some basic gestures and help them practice them in a mock meeting scenario, easing the transition to a gesture-controlled experience in their virtual meetings. 
Zoom Connect introduces twelve new gestures to control the interface of Zoom during an ongoing meeting, which are all thoughtfully crafted to enhance the user's interaction with Zoom.
👩🏻‍💻 Zoom Connect recommends users complete the onboarding process before utilizing the features for a seamless transition. However, users have the flexibility to opt out by clicking the 
End button, ensuring that it does not interrupt their meetings.

3. Virtual Meeting

This meeting example demonstrates how users can use gestural commands to interact with Zoom, and transform their navigation through their virtual meeting by hand gestures.  
While creating this intuitive solution for everyone facing Zoom fatigue, I recognized the significance of substituting features with gestures to cultivate a comprehensive experience incorporating personal expressions.
My objective was to relieve users of the need to reach out to the screen during an ongoing conversation, aiming to streamline the process of seamlessly navigating between screens, such as gallery view, speaker view, screen sharing, and breakout rooms. This is especially beneficial in situations where it becomes challenging to keep track of participants.
 
User’s first encounter with the updated Zoom Connect app, after the demo call using gestures to communicate with the digital interface while attending a meeting.

Gesture Library Assistance

Situated in the upper-right corner, an accessible "Gesture Library" enables users to review and reinforce their understanding of gesture commands at any time during the meeting. 
This feature ensures that users can easily refresh their memory and confidently navigate through specific gestures while engaging in the meeting interface.

First Zoom Meeting with new version

The Evolution of Design

The goal of the research on personal intentional and unintentional gestures, as well as observing a TedTalk by Daniel Levitin on How to Stay Calm When You Know You'll Be Stressed, was to gain a deeper understanding of non-verbal communication.
Non-verbal communication plays a crucial role in conveying emotions, intentions, and thoughts, often complementing or even contradicting the spoken word.

Researching Gestures 

Observing unintentional gestures reveals their task-dependent variability, frequent repetition, and a preference for complementary hand and facial expressions, and increased utilization of wrist-palm gestures while talking.
Being aware of these gestures can improve our ability to understand and connect with others, both in personal and professional contexts.
By studying these gestures, I aimed to uncover insights into how I use body language to express thoughts and to better appreciate the nuances of human interaction.
Observing someone incorporate gestures while speaking also taught me that certain neutral gestures are universally comprehensible.
Observing individuals in cafes, colleges, and airports, I noticed how people are confined to their seats during meetings, restricting their ability to multitask and contributing to health issues. This issue seemed to persist regardless of age or profession.

Defining the problem

Collaborating with a team on a single device can be challenging, limiting their interactions with a single digital surface.
Proximity to devices is frequently required for real-time interface interaction causing strain.
Users need to multitask while working, which hinders their ability to interact with the screen consistently during calls.
In developing my approach to representing gestures for this concept, I initially aimed for a balance between innovation and familiarity. I wanted to create a system that integrated both well-established and universally understood gestures, along with some innovative ones that could enhance the user experience in virtual meetings.
My main goal was to make the interactions intuitive, less tiring, accessible, and engaging for a wide range of users since Zoom is a universal platform used worldwide.

Representing Gestures 

I conducted user testing for my gestures designed for the Zoom app, involving diverse age groups and various professions. I aimed to understand how people from different professions and ages perceive these gestures. This approach allowed me to gather insights into the effectiveness and user-friendliness of the gestures across a broad spectrum of users.

Refining Gesture Language

Turn your hand and slide your thumb up on the index finger. 
Snap your finger
As I progressed, I realized that ensuring the universal understanding of these gestures was paramount. 
To achieve this, I conducted user testing with a diverse group of participants from various cultural backgrounds. This testing revealed that while some of our innovative gestures were intriguing, they were not universally comprehensible and could lead to confusion or misinterpretation.
Some gestures were considered complex, prompting me to revise the gesture language to make it universally understandable, simpler, and serve as a mental model for everyone.
In the design of the gestural interface for interactive systems, feedback played a crucial role. It was imperative for users to receive visual cues from the system, ensuring they could ascertain whether the gesture was registered and responsive to their actions.

Echo and Semantic Feeback

The gesture of thumbs up is a part of the feedback an individual gets when a registered hand is raised.

Echo and semantic feedback are crucial in gesture-controlled Zoom to provide users with immediate and meaningful responses, enhancing their understanding and confidence in the system's interpretation of gestures.
Semantic feedback: Indicates successful actions with processed visuals of emoji and tactile feedback, driving the user’s confidence in the system's responsiveness.
Echo feedback: Feedback at the top right corner of the name window provides instant reflection of the user’s in-time movement without affecting the interface.

User Flow Diagramming 

Leveraging user flow diagrams, I enhanced the clarity of the product's navigation pathways and identified potential steps, allowing me to address potential user challenges and optimize for a smooth, user-centric experience. 
This approach not only facilitated a comprehensive understanding of the overall user journey but also enabled me to meticulously track and refine a specific flow for enhanced user engagement.

User Interface Patterns 

Integrating numerous gestures into the onboarding process presented a challenge in achieving a smooth integration without disrupting Zoom’s existing design language, as my initial UI solutions appeared to clash with the established pattern of the platform.
Feedback through user testing initial versions of UI gave me insights that helped me choose the layout, pattern, style of my final version of interface, seamlessly integrating with Zoom’s existing UI pattern

Wireframes 

Creating iterative wireframes allowed me to architect the user interface, prioritizing elements like layout, functionality, and user flow to streamline user testing. My focus on enhancing functionality entailed setting aside final design components such as colors and images, establishing a foundation for subsequent aesthetic improvements, and the overall optimization of the user experience. I began testing with light mode and moved to dark mode with anonymous photos and then later incorporated video, sound, and zoom avatars.

User Testing with a script 

i. User test script 
ii. Feedback from testing ; iii. User Tester

Key Takeaways 

User feedback played a crucial role in shaping our approach to gesture representation. The feedback highlighted the importance of clarity and the avoidance of any potential ambiguities. 

As a result, we focused on creating a gesture library that not only aligns with users' natural understanding but also minimizes the likelihood of miscommunication or misunderstandings, ultimately enhancing the user experience in ZoomConnect.

Style Guide

Ensuring visual coherence and a cohesive user experience, I extensively examined Zoom's interface and visual language in the development of the Zoom Connect Style Guide. 
This visual representation of the brand encompasses the incorporation of recently designed gesture controls while adhering to Zoom's established design principles to maintain consistency.

Revised Design Decisions 

Through meticulous incorporation of feedback and multiple rounds of iteration, the finalization of my design endeavors aimed for seamless integration with Zoom's established brand principles. This refined version of gesture-controlled features is anticipated to significantly enhance the overall user experience, offering a more intuitive and streamlined approach to virtual meetings and potentially easing the virtual meeting experience for a wide audience.

Zoom Connect Unleashes a Smoother Experience

Update to 5.16.10 now

Reflections 

Reflecting on my journey in creating this project, I emerged as a more skilled designer with an expanded perspective of thinking beyond digital media. While this was an intensive and detailed project, the process was rewarding. 


I gained more confidence in my proficiency in After Effects by learning about smoke and mirror prototyping and green screen video production. 

I also gained proficiency in animating design prototypes with precise timing, particularly when working with live-shot videos to create a realistic gesture-based interface integrated into Zoom.

Next Steps 

In terms of the evolving technological landscape, Zoom serves as a universal platform that is revolutionizing the work-from-home culture. Improving its comfort is a worthwhile endeavor. I believe there is room for further refinement in my design, particularly in enhancing the smoothness of animations to ensure a more seamless user experience.
I also aspire to challenge myself by developing a more unified gestural language and integrating it with additional features in Zoom, aiming to create a comprehensive experience controlled seamlessly by both voice and gestures.