GeorgeandJonathan.com is an NYC electro duo’s band page. On this page, they have these “music videos” that show you all the notes used in their songs. Using Three.js, they let you navigate these notes from a traditional bird’s eye view.

For now, We made a simple environment that makes sound (notes) when users pressed arrow key (left&right). And Whenever user presses the key and makes a sound, it creates a box behind the user like a tail.

And we connect with WebVR, so user can explore the space dynamically.

And we are still working on this project !

What comes next?

Technical Challenges

Supporting socket.io with a-frame

How to ensure a good framerate with other people making shapes?

Connect the Daydream Controller

Visual polish

The Music

Map pitch bend OR volume to a viewer’s z-position (up/down)

How to implement rests? (perhaps a jump?)

Quantize to the beat

Add different instruments

What did we learn?

WebVR with Three.js! A-frame is nice, but it doesn’t get us as far as three.js

We have a lot more design-wise to think through to refine the experience.