livelikevr.jpg

Interactive Media

— Interactive Media —


 Mobile and Environment Media Lab

Housed within USC’s School of Cinematic Arts, The Mobile & Environmental Media Lab’s design methodology straddles the cultures of visual storytelling, games, and interaction design. Accordingly, our work mashes together conceptual tools of storytelling (dramatic arc, character motivation, conflict, obstacles, and resolution) with the core concerns of game design (systems, procedures, constraints, objectives, resources, core mechanics).

At the MEML, I worked on the Virtual Design Workplaces research project. This research project explores how immersive design tools may impact the future of design and creative work. Insights from fieldwork with designers and engineers at Steelcase inspired a series of design fiction scenarios in room-scale VR.

We focused in particular on exploring how immersive interfaces could reshape practices of ideation, sketching, modeling, annotation, and prototype-review. We developed a methodology we call immersive design fiction to situate fictional embodied experiences in VR. As speculative interaction designers, immersive design fiction enables us to go beyond the simple prototyping of interfaces to also model the interaction rituals and surrounding social worlds that might envelope these experiences.


Walkabout

Parkinson’s disease is the second most common progressive neurodegenerative disease affecting approximately 1 million Americans and 60,000 Americans are diagnosed with Parkinson’s disease (PD) every year. This chronic disease impairs walking and balance, which 50-68% of people with PD experience one or more falls related to walking every year. The high risk of falls devastatingly affects the individual’s quality of life and independence. The cause of PD is unknown, thus physicians and therapists have focused on reducing parkinsonian symptoms such as walking disability. However, multiple clinics encounter some limitations where dynamic walking environments are difficult to simulate in a clinical setting.

Walkabout is an over-ground walking intervention in immersive Virtual Reality that can help players to practice walking in motivating environments while achieving dynamic walking skills such as obstacle negotiation and walking in a narrow path. Players experience a wide range of dynamical environmental challenges during walking as well as altruistic feeling of helping the virtual world. Accurate and efficient walking to a target is the key element to win the game. By using vive trackers for the feet, we were able to give useful feedback to patients about their walking skills.

Walkabout was created by a team of five including a current PhD candidate in Physical Therapy, a film maker and three MFA candidates from the Interactive Media and Games division at USC: Aram Kim, Patrick Bender, Daniel Batista, Wilson Stiner and Jivitesh Dhaliwal. Walkabout was created over a duration of 48 hours. It won the second prize at VRSC, a student organized festival at USC. Here is a link to the github repository for the project.


MAESTRO

Hack Music LA was the first ever hackathon organized by the LA Philharmonic to bring together 75 of the country's most creative musicians, engineers, artists, game makers, filmmakers, and entrepreneurs to create tools and experiences that expand access to music.

As a part of the Hackathon, my team created Maestro, a VR conducting application that could detect and analzye gestures of a conductor to control MIDI output in real time. Maestro won the Amazon Web Services and the Oculus prize for the best experience.


Livelike-VR

LiveLikeVR is a live streaming platform that allows broadcasters to stream regular or interactive VR content under their own brand with or without a VR headset. LivelikeVR successfully streamed major sporting events such as the French Open and the NFL SuperBowl. I had the great fortune of working with the young and talented team of LivelikeVR before joining the Interactive Media and Games Division at USC

I was given an interesting challenge to work on while at Livelike. We have a multiplayer feature where multiple people can watch a live game together in the same virtual space. However, due to bandwidth limitations, some users' video streams would lag behind others. This ruined crucial moments in the game for users, which was unacceptable to us as the creators of the interactive experience. I was tasked with finding a solution to this problem. With the help of some brilliant people working at Livelike, we managed to successfully resolve this challenge. 


TANSEN!

Tansen! is an interactive music project for art installations.

How would it be if another layer of expression were added to the experience of looking at art at an art exhibition?

This was the starting point for my interactive music for art installation project. I wanted the viewer to be drawn to the painting and while they were looking at it, to discover greater meaning in the painting through music.


MUSIC FROM YOUR HEART

Music From Your Heart is the more ambitious and potent successor to the Interactive Art Gallery project. Instead of relying on external factors, Music From Your Heart is based on the premise that a truly interactive experience should be channelled through emotions. Using expression recognition, this experimental application can change the music you hear to reflect the emotions you are experiencing. If look happy, you hear a more upbeat music, while if you look sad, the music is stripped away to just the melody. 
This project is my starting point in the exploration of emotions as a tool to craft more influential experiences. I want to use this project as a means to answer the most pressing question I have- How do I design moving stories for my players?

Try it out now on your Mac or PC. 

Instructions on how to install the app can be found here.

Music-From-Your-Heart.jpg

PAT-NAV: INDOOR NAVIGATION FOR PATIENTS

The Post Graduate Institute of Medical Education and Research, Chandigarh (more commonly known as PGI) is one of India's premier medical institutes. It is also one of the largest in terms of patient influx. For 2015-16, the OPD attendance stood at over 2.4 million patients. An overwhelming majority of patients belong to the economically weaker section of the society and are uneducated. As a result, they face numerous obstacles and discomforts in obtaining healthcare. One of these obstacles is navigating through the different departments at the institute. Doctors write down the name of the department the patients need to go to next, but not having prior experience and being uneducated, they are unable to. A lot of patients have a tough time finding their destination especially since they are ill. 

I was going through Professor Patrick Winston's lecture on A* search on MIT OCW to build my game programming skills and realized that it could be extremely useful for the current problem. Given a set of locations and distances between them, the algorithm could be used to plan the most efficient path. Since the architectural plan of PGI was easily available, the major part of the work was easily accomplished.

The next challenge consisted of figuring out a way to input the patient's locations. Since most patients are too poor to own smartphones, it was imperative that the solution also work on low cost phones (< $20). For this, a set of numbers were assigned to different locations and the patients could send a free SMS to the service with their destination and their current location (as numbers). Instructions to the locations come as both written and symbolic instructions. Patients with smartphones can use interactive maps and bar code scans to find the way.

Still under iteration, this project will grow in scope to include the main problem of helping patients know the time of their appointment. The process of registration at PGI involves the patients getting medical cards made. These cards are then submitted to the designated OPDs. Registrations end at 10AM, after which the cards are collected. However, since the cards are collected early in the morning, the patients form a mental map that their appointments will be scheduled soon. As a result, those that got their cards made later sometimes have to wait over 7 hours. To prevent this, I intend to add a 'time estimate' feature, where registered patients get a text message with the approximate time of their appointments.