Biophilia – Crystalline by Björk
Creating one of the first interactive music app for the iPad…
In the fall of 2010 and early 2011, Sarah and I had the chance to work with Björk, and a few very talented artists to build an interactive version of her album Biophilia. I created the interactive version of Crystalline.
My friend Theo Gray, who invented the front end interface of Mathematica among many things, was one of the early secretive developer for the iPad. He released one of the first 14 application on the iPad called “The Elements”. It is still one of the most satisfying experience on the iPad 14 years later…
Björk and her team had seen Theo’s app, and they contacted him to see if he could help them build an interactive application for the upcoming album. After a few conversations, when he realized the scope of what they wanted, and the fact that they wanted something that felt like a game, he suggested that his company “Touch Press” could be the publisher, but that I could be the developer.
Sarah and I went on a trip to Iceland, where we spent a week with the team of developers: interactive artist and app developer Scott Snibbe, Björk’s longtime design collaborators M/M Paris, Drew Berry, Stephen Malinowski, and Max Weisel. Theo Gray and Matt Whitby from Touch press were there also with us.
Later Marks Danks, Sharon Hilbert, Lisa Ching also contributed to finish the “game”, and the album.
Here is the tutorial on Crystalline, which explains how to play the game, narrated by Scott:
I thought I would share the journey on how we got there, from a creative and from a programming point of view…
First we travelled to Iceland:
(me and Theo)
Sarah standing between two continental plates…
Continuing from Biophilia part-1…
The blue lagoon, on your way back to the airport
Björk wanted a tunnel game and express specific feelings from the music
in 2010 there were already many tunnel games, which used the orientation of the iphone to move your position. In general you had to avoid objects, sometimes hit booster pads. This was one of my favorite:
The goal of the game was to control which next segment of music would be played, along with unlocking new segments.
The song had been composed to be divided in the following 17 pieces, and this connection graph showed how they could be played:
I wondered how I could prototype this, and as usual started in Mathematica
I started to prototype tunnels, connected to one another:
which could grow like this:
and visualized it from the inside
I realized I could organize the space like this, where each piece would include a tunnel segment.
Made of blocks like this:
and in each block the connectivity would be like this:
( you enter from one side, and the curvature takes you to one of the other side.
Any cube would have only one of the curve)
And one trajectory while playing the song might look like this:
I would only create the tunnel in front of the user, and remove it behind the user, so the tunnel is actually very virtual, but gives the feeling of continuity. At anytime, there are only very few tunnel segment created.
In between segments there are connection rooms, and you can tilt the iPad to chose which tunnel you go to next.
oh!, and there is also a nebula…
one of the aspect of the “game” Björk really cared about, was that when you play one of the specific song segment (Galaxy), you elope from the tunnel, and are now flying in space, to come back where you started from.
I played around to find a suitable curve:
Manipulate[
pts = {{0, 0}, p0, p1, (p1 + p2)/2, p2, p0, {0, 0}};
F =
curveLength =
Total[Norm[#[[1]] – #[[2]]] & /@
Partition[Table[BezierFunction[pts][x], {x, 0, 1, 1/100}], 2, 1]];
Graphics[
{BezierCurve[pts],
Point[pts] },
PlotRange -> All,
PlotLabel -> StringJoin["Length=", ToString[curveLength]]
],
{{p0, {0, 2}}, Locator},
{{p1, {-3, 4}}, Locator},
{{p2, {3, 4}}, Locator}
]
I also started to create textures for the side of the tunnels. It turns out that one can create delauney triangulation on a cylinder, or a torus:
Oh, and then I started to code this in Unity..
( not knowing that I would end up working at Unity just 8 years later…)
I knew it would be easier to assemble something quickly, and maintain a good framerate, and add good lighting, transparency etc… I started to code this in Unity.
Some more information about the art direction
Valerie Zaroli, helped me put together some awesome sketches in drawings and in Maya to get a feeling for what things could look like:
Thank you Valerie for providing unbelievable art direction early in the project!!!
The unity images ended up very close to this…
Reading the music and adding crystals…
Björk’s team provided me with the midi file of the music, and that enabled the creation of crystals in the tunnel which are synchronized with the notes.
Then we added a game of collecting crystals on the side of the tunnels, different combinations “opening” new tunnel intersections in your space…
It is not a game you can really fail at, but if you do well, you acquire more freedom of choice.
Switching to Xcode !!!
We started developing all of this, thinking that there would be one main app (the album cover) selecting any of the other apps, but as Apple started to get involved in the project, they required that the whole album be only a single app!!!
This killed the use of Unity!!! because it did not seem plausible that we could teach all the developers Unity in time to finish the project…
I had to rewrite the game in Objective C, and find a way to re-implement the graphic effects in the much more limited graphic pipeline (no transparency). ARGH!!!
I asked Lisa Ching who I worked with at EA, if she could help me with the UI and the game logic, while I focused on the rendering of the tunnels. (Thank you Lisa!)
To make things more challenging, the team decided that Crystalline would be released first, while the other apps would be released a few month later.
Art direction from M&M
In the end, M&M (Michael and Mathias) who were the art directors for the whole album, provided clear direction to align all the teams. The tunnel had to be converted to line art (Thank you again Mark Danks, for solving that so quickly!!!)
The album got great reviews… and was even accepted into MoMA
Thank you all for an incredible experience… and specifically to Björk and Derek for trusting us to do this!
Again, big thanks to Mark Danks, Lisa Ching, Scott Snibbe and his team!
Thank you Theo Gray for inviting me to do this!
Luc