Posted on 2014 Jan 23 - Comment

A quick, yet long overdue, debrief about our game ‘Panflute Hero’, created during the Way Out West Hackathon 2013.

During last years (2012) WayOutWestHack, we made a multiplayer 3D space shooter, playable from inside Spotify desktop client, where you fly around in a spaceship and shoot your pals/foes using your favorite songs as ammunition. This year, we wanted to up the ante a bit and decided to add hardware into the mix. Except for one electronics course in college (where most of us flunked the tests a couple of times), none of us hardly know anything about hardware. Nonetheless, the idea of a guitar hero-esque panflute game was too stupid and silly for us to pass on.

All in all, getting all the hardware and piecing together the first flute took us about two weeks in total. All of us are full-time employed at various places, so finding time in everyone’s busy schedule was more challenging than actually building it. The core backend software was developed simultaneously, where most of the time we spent was on getting libspotify to play nice as well as setting up Lua bindings for the cross-platform gameplay framework. The bulk of the game engine and gameplay was implemented during the 24 hour time frame of the hack itself.

In the spirit of hacking and hackathons, our code really blows (get it?). You can look at it BitBucket and publicly shame us if you want (please don’t).

Tech used

• Flute controller hardware - Python running on a RaspberryPi, listening to the GPIO pins and forwarding “blow” events via TCP to the desktop.
• Game - GLFW was used for OpenGL-context creation and local input, and OpenGL is used for all rendering. SOIL/stb_image is used for texture loading. Almost everything is written in Lua running in luajit with FFI bindings for libraries. Some support code is written in C++ and exposed as a Lua module. libspotify for playing Spotify songs in the background using RtAudio.

Building the flute

The first step in the process was to find some kind of building material for the body of the flute. For this, we bought six meters of bamboo (for about 7 EUR / 9 USD) at a local flower shop in Gothenburg, and used about two meters in total for the two flutes (we still have four meters left if anyone wants to build a flute, or a small raft). We sawed out six times two pieces in descending lengths based on a golden-ratio-type algorithm we devised after hours of research:

1. Select the shortest acceptable length so that the raspberry is hidden from the front
2. Increase each pipe length by one or two cm until done

To capture when the player actually blows into each pipe, we bought and installed four Arduino sound sensors per flute. The sensors were positioned at the very tip of each bamboo pipe (glewed shut so no “real” sounds were generated from the pipes), wired to the GPIO pins on the RaspberryPi and in turn read back using the Python package RPi.GPIO.

Communication

Alongside reading the sound sensor inputs, we run a local TCP/IP server that acted as a controller hub. This mean we can serve multiple game clients the flute events at once (mostly for debugging). The server listens for events on the Raspberry PI GPIO ports, and forwards them over TCP/IP to all connected game clients that are listening. It can also be run locally together with the game to simulate a flute if the flute is being blown by someone else. The clients can also send commands back to the server to tell the flute when to turn on and off the non-existent lights (which was one of the planned, but canned, features).

Game Client

We used UFO, which is essentially a bundle of Lua binaries, modules and FFI-bindings, as a base for the game client. Our game engine was fully written in Lua, and used a fairly trivial game loop providing us with an update function for refreshing game logic and a draw function for drawing the game state. On top of this, each game state (i.e. “main menu”, “song selection” and “game session”) was represented as a “self-contained screen”, essentially a small unit responsible for updating and drawing itself when asked to. These screens was put into “stack of screens” where we could easily push and pop screens to navigate through the different game states.

Input events are polled every frame from both the Flute controller and the keyboard. The events are then translated into game events for the active screen to use.

To play some funky panflute tunes, we rolled our own Lua bindings for libspotify. This enabled us to access specific playlists at runtime, and in turn meant we could add songs outside of the game and they would automatically show up for everyone running the game.

Game rendering is very simple: first the four pipes are rendered into the framebuffer using a mix of lambert shading and pure nonsense. The pipes themselves never move, instead, the applied textures are sampled with a time-based offset to indicate movement. The colored ‘blips’ area, where a player need to activate each pipe, is a small section of the pipe-geometry that is rendered on top of the individual pipes. To achieve the wobbly effect, the base-color texture is sampled along the horizontal direction with a good old sinus function, modulated with a time-since-pressed value sent to the GPU as a uniform.

On top of the “fancy” graphics we implemented a basic dynamic environment backdrop, a sprite system, a bitmap font system with our own hand-drawn font and some fancy tweening for the fluterrific sprite pop-ups.

Look-and-feel

Since the whole concept is based around a well-known franchise of games, we wanted our version to resemble the aesthetics and characteristics closely, though with a more colorful and playful palette together with hand-drawn elements. Thankfully, unlike previous years, we had a dedicated graphics person on the team, which drew all assets and coordinated the overall look. We only used original content, except for the songs which were streamed through Spotify.

Things we learned

• It’s hard to come up with puns on panflutes other than ‘pantastic’
• Playing the flute require a set of lungs of athletes competing in the olympics
• There are A LOT of panflute covers on Spotify (they are all amazing)
• Sound sensors are not exactly in either supply nor demand

Features that was cut

We had big plans for cool features, but of course some had to be cut to meet the deadline.

• Player 2, a second flute - we broke one of the sound sensors a couple of the days before the hack. Got the last sensor the same day of the hack, but didn’t have enough time to finish it.
• More songs - it took quite a long time to build each track, so we only ended up having one (almost) complete playable song - a panflute cover of Moonlight Shadows by Mike Oldfield
• Automatic level generation - using data from the Echonest audio analyzer it might be able to generate some playdata automatically.
• Raindance-mode - we initially planned to have one player on the flute and one player doing a rain dance routine at the same time. The dance could be captured by Kinect where you had to match your hand and feet movement to a pattern on the screen. Scrapped quite early on..

B-Team

• Sven Andersson - @andsve
• Johannes Algelind - @jalgelind
• Filip Lundborg - @flundborg
• Jhonny Göransson - @jhonnygoransson
• Malin Sundelin - @kannibalin
• Ingemar Ådahl - @ingemaradahl

Aftermath

Our entry won us all a game of our choosing from VIAGame and a Jambox from Musixmatch as well as the heart and love of the audience. Our hack also was mentioned in the online edition of Wired, “The five best hacks from the Way Out West hack battle”.

Thanks to the kind folks at Kjell & Company here in Gothenburg for going the extra mile and ordering us some extra sound sensors from all around Sweden. :)

Posts by fellow hackers

• The Saddest Stylophone by Paul Lamere
• Discover new music browsing the globe through time and space with MusicMap by Rickard Olsson