Crowdpleaser - Red Bull AdrenaLAN 2019
Winner for Best VR/AR/MR Hack
Red Bull AdrenaLAN 2019
Fast forward to now, we live in a more connected world. With video games, we were accustomed to couch-co-op gaming. With the advent of multiplayer gaming in online infrastructures, online communities give us limitless opponents. Live streaming gamers have access to audiences wanting to view games in progress. Yet with this success, comes a downfall for live streamers.
Our team came together to create Crowdpleaser for the Red Bull AndrenaLAN hackathon. Crowdpleaser is our solution to solve what is hampering the gaming experience for live streamers and the audience. Crowdpleaser uses augmented reality (AR) to create a 3D audience, generated from an active chat room. As users in the chat room type their responses, the application uses Google’s Natural Language (NLP) to deploy application programming interfaces (API) to generate 3D model animations.
Research & Process
The sentiment is correlated to a 3D character model expressing that animation whether it be joy, sadness, disgust, or another emotion. The 3D character is animated and compiled using Unity 3D, to then be shown in an AR environment. The goal was to output the AR content into 2 streams. One that could be broadcasted on a live stream using Open Broadcast Software (OBS) onto Twitch. The 2nd stream would be available to the live streamer using a Crowdpleaser iOS app, powered by ARKit.
Architecture & Prototype
Sentiment Expansion – We were able to translate some of the chat using the NLP API. It was determined that to capture more sentiments in the chat feed, we would need more custom Natural Language API based on the game.
Cues – Such as clapping, cheering, jumping, and thumbs down. Ideally, facial animations on the 3D character models could be used to emote the sentiments.
Audience Analytics – We have created a way for the live streamer to see a visual representation of how the audience is engaging with the content they are streaming. There’s an opportunity to keep track of audience participation, feedback, and time.
Character Licensing – Opportunities for developers to lend out their game characters as AR avatars in Crowdpleaser. They could release custom skins, emotes, and animations for micro-transactions incentives.
Stadium Seating – We could explore crowd placement on Crowdpleaser, for future iterations of the platform. With many character models populating the screen, the live streamer could offer users to move up the front. By offering a micro-transaction (Twitch Bits), that live chat user could be featured more on screen.
Team Crowdpleaser was awarded 1st prize for Best VR/AR/MR Hack during Red Bull AdrenaLAN 2019. As a team, we were happy with what we were able to accomplish.
We felt Crowdpleaser could have been a contender as well for Best Streaming Integration Tool as well. We should have considered developing for the Twitch Extension platform at the time.
As we presented Crowdpleaser to our panel of judges, we were only able to show a mockup video of how in principle Crowdpleaser would function in the AR mobile app environment. We would have liked to showcase a live stream prototype using OBS integration.
Demonstrating AR/VR requires knowing the environment you are presenting your pitch well. There’s an expectation of wearing or holding an AR/VR device. There’s also to consider spatial design in case the solution requires a flat surface or space for movement.
For more case studies, click on one of the following below.