From our partners at Hearst: A look at the Hearst Immersive Hack

This past October 24th + 25th we co-presented and hosted the Hearst Immersive Hack, where different media and tech teams met for over 24 hours of innovative development in virtual reality, augmented reality and Internet of Things. Below is a recap and highlights of the hackathon written by Johann Rodriguez, General Manager Hearst Labs & Tech Community Manager at Hearst. To read the original blog post click here.

After many months of planning, “the weekend” finally arrived. On October 24 and 25, Hearst co-presented the Hearst Immersive Hack in Brooklyn, N.Y. For those unfamiliar, a hackathon is an event in which developers, designers and entrepreneurs collaborate intensively on software projects in competition with other teams. The hackers are incentivized with prizes and free food (and energy drinks!) as they code throughout the night, usually for 24 hours. Why would Hearst do this? The answer is simple: to spark innovation by letting smart people think outside the box.

For the Hearst Technology team, the benefits of a hackathon are very evident. We get hackers to use and provide feedback on Hearst Magazines’ APIs (in simple terms, APIs are sets of requirements that govern how one application can talk to another). For this hackathon, we teamed with Hearst VP of Engineering, Jim Mortko, to open up the typically-private Hearst APIs. Our APIs are full of content—28 titles, 1.1 million articles, 100 thousand galleries and 3.8 million images.

The Hearst Technology team is constantly looking for ways to innovate and provide unique experiences to consumers. Right now, the most exciting consumer experiences come from immersive technology, specifically Augmented and Virtual Reality. In fact, some industry experts forecast this new industry of immersive tech will reach $150 billion by 2020! Keeping in mind that this tech is still in its early stages, there remains a great deal of untapped potential for innovation, making it the perfect theme for this hackathon.

The Hearst Immersive Hack was not the company’s first public hackathon. In February 2013, the Hearst Fashion Hackathon was a great success, so we had a high bar to live up to. To maintain our high standards for hackathons, we also introduced three new things to set this hackathon apart—a new collaboration, inviting startups to participate and providing immersive gear to the hackers.

The Made in NY Media Center by IFP is a transmedia storytelling co-working space and community based in Dumbo, Brooklyn. The Media Center is a membership of producers, filmmakers and technologists, so we saw it as the perfect space to connect hackers.

One of the big goals of the Hearst Immersive Hack was to give early startups a chance to participate. This would set our hackathon apart and also increase the likelihood of being able to work with the established startup following the event. With that in mind, we created two challenges for this hackathon—the Fresh Code Challenge (for hackers) and the Startup Challenge (for, you guessed it, startups). In addition, there would also be a big grand prize open to both hackers and startups.

Given the state of this new, immersive tech, most of the hardware being used is still in development and not that easy to acquire. Through partnerships, we were able to gain access to several types of technology—IBM Bluemix, HP Haven OnDemand, Littlstar, Presto, WNYC Radio, Leap Motion, and, through some great connections by our friend J Dakota Powell (VR enthusiast and true rockstar), we got Oculus to sponsor the event as well. In addition, we also acquired (and borrowed) lots of gear for the hackers and startups to use, including Oculus Rift, Gear VR, Leap Motion and Google Cardboard 2.0.

With all of these pieces in place, the Hearst Immersive Hack was a huge success. In the course of this 24-hour hackathon, we had over 100 hackers, five startups, 25 teams and 22 projects submitted, as well as over 315 Meerkat livestream viewers! The 22 projects were judged by four esteemed industry experts, including Hearst Chief Technology Officer Phil Wiser, Liza Kindred from Third Wave Magazine, Jason Saltzman, CEO of AlleyNYC, and Omer Shapira, director of VR at FakeLove.

The winning hack for the Fresh Code Challenge was Beat Reader, an immersive reading experience which allows users to browse Hearst articles hands free and to the beat of the music. The app works on a regular iOS or Android smartphone and lets users browse Hearst articles in VR space—to the beat—hands free! Cover images and titles appear to the beat of music and float past users. If a user stares at an image for more than two seconds it’s saved for reading later. The demo was augmented with a SubPac subwoofer backpack, so the user could really feel the beat of music as they tried the product. This team took home five thousand dollars in prize money and a future invitation to pitch a hack to a panel of Hearst executives, including Wiser.

The Startup Challenge winning hack was Bytes, a news data visualization app for Apple Watch with the goal of redefining data consumption on wearables. This hack uses a natural language processing engine called Summarizer (developed by Agolo) to summarize the key contents of a Hearst article into one sentence/headline, while visualizing contextual data that enhances users’ understanding of the material. In addition to five thousand dollars in prize money and the future pitch invitation to Hearst executives, the Bytes team also received an Oculus Rift Dev Kit 2.

The Grand Prize was awarded to Allright Satellite, a mixed reality (AR+VR) educational platform that uses physical cards to teach children about the solar system. The experience turns kids into astronauts and requires them to use different cards to interact with the game. Each card triggers a single story, such as learning more about Saturn or following the path of the Mars Explorer mission. The team won 10 thousand dollars in prize money, the pitch invitation to Hearst executives, a community workspace membership to Made in NY Media Center by IFP and a chance to demo at NYC Media Lab’s event “Exploring the Future of Reality” on November 5.

Other highlights of the show include TheLibrarian, a new form of search aimed at discovering new topics through hand gestures and keyboard inputs, while navigating a virtual library space. The Akiles Smart Infrastructure with VR hack allows users to diagnose the status of a resource (e.g. pot holes on a road) and create a smart contract that would allow users to report or resolve issues. Other users would be able to access these contracts and invest on the resources while receiving residual passive income for their investment.

VCPR is a virtual CPR training web app using Oculus Rift and Leap Motion to create a one-on-one learning experience. The trainee gets to practice the correct pace for heart compressions and receives immediate feedback, an idea that could be extended to different emergency trainings. And for those of us who are not kitchen savvy, the Mocho hack could be a game changer. This AR app allows you to point your smartphone at pictures of recipes while the phone displays the pictures of the ingredients separately for easier identification. As an added bonus, it also shows you a map of nearby places to purchase the ingredients you need.

Another highlight was Evolving Scenery, a portal spanning the world of Hearst brands that allows users to traverse various Hearst Magazines-themed hubs and read Hearst articles, while applying force to a wireless foot controller, the VToe, to move around the virtual environment displayed thru the Oculus Rift headset

Another clever use of augmented reality came from One Book to Rule Them All, a hack where you could map any book or content onto an existing physical book. The user would have the experience of actually turning pages using gestures, while the pages virtually turn on the user’s smartphone screen.

Real estate showings and interior design are some of the use cases for SpaceWalk, a way to simultaneously walk in real and virtual spaces by hacking together iBeacons, Google Cardboard and a smartphone. Using low-cost bluetooth sensors to triangulate your position, location data is fed into your virtual headset. This hack allows users to experience virtual space in a way that has only been available with expensive motion-capture products.

So, what’s next? The hackathon winners will be invited to the Hearst Tower in Q4 2015 to pitch their hack to a panel of Hearst executives and thought leaders. The Hearst Technology team plans to continue building relationships with the NYC tech scene and exploring potential collaboration opportunities with the hackers, startups and partners.