VR Music Videos 2019

VR Music Videos, is a project/prototype/proof of concept collaboration between Abstract Worlds and Wellingborough's live looped electro-acoustic dance-punk contemporary R&B singer-songwriter Paul Strummer (who is also Paul Ireland from Abstract Worlds, Paul Strummer is a stage name).

360 music videos like Gorillaz Saturnz Barz 2017 are the current state of the art. Abstract Worlds wants to go beyond 360 videos, with realtime 3D virtual reality worlds you can move around (go forward, back and around the performer too). Combining web 3D, embedding multi-angle green screen chroma key video compositing performances of the artist (streamed from sites like youtube and vimeo), music and 3D audio visualisations, delivered in a standard web page viewable on mass market mobile and desktop devices, no VR headset required. An early proof of concept working prototype was created at the end of 2018. The main Javascript framework used was Three.js, along with some WebGL shading and other HTML5 components. Investigations are also underway for packaging the player up as a component, be it a standard HTML/Javascript component, or a React/Angular/Vue component.

By applying for a grant in November 2018, Abstract Worlds had hoped to secure UK Research and Innovation, Innovate UK, Industrial Strategy Challenge Fund, Audience of the Future, Production innovation for immersive content research grant funding to move forward with this project in a big way in 2019, but unfortunately the grant funding application was unsuccessful. Update: In July 2019 we applied for a second grant funding opportunity with Innovate UK, which also failed!

VR Music Videos Proof Of Concept Video and Live Web Link

VR Music Videos Proof Of Concept Video, created in November 2018 as par of grant application

(Unsuccessful funding application bid video. Perhaps the funding decision makers didn't like my hair at the time that prototype and video was created? I have cut my hair shorter since then!)

Live realtime 3D interactive proof of concept demo also available at:

www.paulstrummer.com/3d2

The demo should work on recent Android and iOS devices and on desktop browsers (Chrome, Firefox, Safari, Edge, IE).

When the page is loaded it will try to autoplay the video, but a lot of devices and browsers now prevent video autoplay on pages, so you will probably have to click, touch or double touch on the "Play!" link at the top.

On phones you should be able to move around the 3D world by touch dragging the screen, and zoom in and out using pinch zoom. On desktops/laptops, navigation is by mouse left click drag, and up down arrow keys and right click mouse drag (or mouse middle wheel) to zoom or if you have a touch screen, move by touch dragging the screen, and zoom in and out using pinch zoom.

Update February 2019: I am so happy to hear about DJ Marshmello's live virtual concert within the Epic Games Fortnite 3D game, which was attended by 10 million players worldwide. It was not easy receiving the bad news of the grant funding rejection email in January, but this game changing event in February was very welcome news as it completely validates where I've been going with the idea of placing performers in an immersive 3D world, and it dispells all the hype surrounding VR, people don't need VR headsets to enjoy VR. I only wish that the Marshmello Fortnite event happened a few months ago, because if it had, it would have made my grant application more compelling. There are important differences between the Fortnite approach and my approach. Fortnite requires the Fortnite game to be installed and has higher system requirements, whereas my approach just requires a web browser. The Fortnite approach is only open to the top music artist performers who Fortnite choose to work with, my approach democratises the technology and makes it available to any artist. The Epic Games team were wise to choose Marshmello as the performer as this made things more simple, he doesn't play an instrument and you can't see his face, which meant that motion capture applied to a 3D model was an easier way to go. This approach is more difficult with most other musicians since you need to see the singing face too, and guitarists can't play guitars wearing VR motion capture gloves. So I'm still very happy with the live video capture approach I'm championing!

Update July 2019: In July 2019 we applied for a second grant funding opportunity with Innovate UK, incorporating all the feedback for improvement we had received in the previous application. In early August 2019 we were notified that we had failed in this grant funding opportunity too! :(

×