A closer look at the interface design process for the development of a touch-screen-friendly skin for Liquid Rhythm.

There is an ongoing collaboration in the works between WaveDNA, SmithsonMartin, and music teacher Sina Fallah. Together, we are working toward building a uniquely powerful yet simplified interface for students to learn about creating music.

WaveDNA’s Liquid Rhythm has a robust interface for creating rhythm tracks. While it can be used with a touch screen, we wanted to utilize an interface design more appropriate for touch so that students would be able to use it very easily. To do so, we needed a couple of things. The first thing was the hardware. SmithsonMartin provided the touch-sensing technology, called G4 Overlay Touch Frame, from PQ Labs. (A “touch frame” is a large, empty frame.  The inside of the frame is lined with an array of infrared sensors that detect  fingers that pass through them.) We applied the touch frame to the frame of a 32″ TV. This worked great. We then enlisted the help of SmithsonMartin’s Emulator software. It’s specifically designed to act as the ‘middle layer’ between a hardware touch interface, and non-touch-optimized music software. Essentially, it’s a highly customizable ‘virtual touch interface’ that allows users to create buttons and sliders to send and receive MIDI.

After applying the PQ Labs’ touch frame to the 32″ TV, we began designing the Emulator skin. The goals for the interface design of the Emulator skin were twofold. 1) Provide a very easy workflow for beginner users to create rhythm tracks. 2) Facilitate the education of rhythmic structures by revealing visualizations provided by Liquid Rhythm’s interface.

The first step was to choose the features from Liquid Rhythm that we wanted to make available in the Emulator skin. Liquid Rhythm has a powerful suggestion system that suggests rhythm patterns dynamically according to the instrument selected in its canvas. We created eight big buttons in Emulator to insert and swap those bar-long note patterns into the arrangement. Since this widget is the primary way to building a rhythm pattern, we made the Emulator buttons for it big and arranged them in a grid.

Next up was Liquid Rhythm’s step sequencer. A traditional step sequencer inserts and removes notes one at a time, and this workflow can be represented effectively and simply with a row of buttons. However, the sequencer in Liquid Rhythm allows users to sequence multiple notes simultaneously, such as groups of eighth notes, sixteenths, triplets, and 32nds. This could be challenging to represent in a simple way. So, we decided to include only the simplest set of groups for new users to learn with – eighth notes and sixteenths. We created two rows of buttons – eighth notes and sixteenth note groups, and we labelled the buttons with dots to indicate the quantity of notes each one affects.

The last set of features we implemented in the Emulator skin are among the most fun to play with in Liquid Rhythm – the Randomizer and the GrooveMover. To entice users to play with these buttons, we made them very big, and gave them bright colours, using Emulator’s colour palette. The interface design was coming together smoothly.

Finally, all of these features perform actions on regions in the arranger, and users must select the bars to target. Emulator has a very clever feature that allows the designer to literally cut a hole in the Emulator skin, revealing a portion of Liquid Rhythm below it. This allows for very flexible and creative interface design.  We positioned this cutout above the first bar of Liquid Rhythm’s arranger canvas. When users begin to build rhythm patterns, they will be able to select the region by tapping on the location, and they’ll be able to see Liquid Rhythm’s Music Molecule MIDI visualizations directly. This exposure to the visualization, in turn, will help students understand rhythm structures.

The design and development of this Emulator skin for Liquid Rhythm was very satisfying and fun. We presented our Emulator+Liquid Rhythm+TouchFrame system to Sina, and he was extremely happy with the results. Here is what he had to say:

I started my collaboration with WaveDNA in August of 2013. I was interested in their latest creation, a rhythm-­‐generating software named Liquid Rhythm. My plan is to discover the potential use of this software in the health industry, particularly in the rehabilitation sector. Rhythm is not only the most fundamental element in music, but it’s also a very essential element to a number of human behaviours, especially those pertaining to motor movement, such as limb control and speech. I foresee that, as technology becomes a more integral part of everyday life, that rhythm exercises composed on Liquid Rhythm which are played in a loop can have many beneficial effects for cases of human exceptionality wherein there is some disruption in controlled motor movements and it is my goal to find out if these exercises can enhance these disruptions through repeated rehearsals.

We look forward to seeing how this progresses!

Written by: Saro Migirdicyan, UX Manager @ WaveDNA