Notes On the Margins

onthemargins.com

I’ve always been kind of a control freak, especially when it comes to motion graphics. I went into OpenGL with C++ because I felt that it gave me more more control than other ways of making music animations.

Control can come at a price - and right now its at the expense of output. Before I can make something, I feel like I have to restructure the code. But I know that would be getting too involved in the details of refactoring rather than creating something new. It would be a long process and I’m sure my creative drive would dampen along the way.

Enter Quartz Composer.

I have recently come across QC while watching some WWDC videos at work, and I can’t believe I haven’t given it much attention before. Its a visual/programming environment, similar to Max/Jitter and Isadora in its patch-and-cable interface. “Patches” are basically units of inputs and/or outputs that can be hooked together to eventually render something to screen. There are three classes of patches: Providers (of data of some sort), Processors (of that data), and Consumers (renders the result).

What is most appealing to me about QC is that, besides the default patches, it is also extensible with Objective-C. Lucky for me I have been working with Cocoa for the past year!

Even though I would still be coding custom patches, I can see the GUI  saving me a time and energy. I can experiment with different configurations and orders of data, processing, and rendering by rearranging patches, not by messing with function calls. I had often thought of the different objects in my own programs as units that hook up to each other, and I wanted to make this more obvious by defining protocols (interfaces) that the different types of objects (sound provider, animation classes) would have to follow. This all seems to be thought of in QC.

So I can’t wait to really get into it. I’ve been watching some tutorials to see how to get it working with sound. Theres a music visualizer template that actually provides audio spectrum data, which was a big challenge for me when creating my own program. One of the videos showed the Audio Spectrum patch output.  I was instantly reminded of my debug logs when developing Noesys.

Which makes me wonder if in porting my animations to QC, I would just throw out most of the FFT code. I don’t want my previous work to seem like a waste of time. A comforting thought is that at least I can write better processors to go with the Audio Spectrum patch now that I have gone through understanding and implementing my own.

I must admit this discovery is bitter sweet. I’m a little shocked that my investigation into competitors and frameworks at the time of my thesis didn’t lead me to QC sooner - or that none of my knowledgeable professors or peers knew about it to point it out. But then again these videos on QC I am watching are from 2007 and on, and I finished Noesys in Spring 2006.

My next steps are to look at the tutorials on the web and the posts on Create Digital Motion to catch up on what people have been doing with QC. The WWDC video also briefly describes how to use QC for interactive prototyping, so I think the first project might be somewhat work related, using mouse events. More to come…

Differential dynamics is a kind of motion - periodic motion - that was originally described by John Whitney Sr. in his book Digital Harmony. One can’t really say he discovered the phenomenon, but he did discover a proper application for it in visual music. Whitney believed the cyclical tension and resolution of differential motion was a fitting match to musical structure.

Here is a demo of differential dynamics I mocked up using a sample program he outlines in his book.

I previously brought up the idea of a taxonomy for musical visualizations. Maybe taxonomy is not the right word, but I do feel that each of our endeavors at expressing music visually can be placed between two poles. These two poles are also present in visual art - there is that which is highly abstract - consisting of shapes and colors that do not refer to anything outside their own forms - and works that are much more suggestive, figurative, they refer to elements in the world around us.

With these two extremes there is another polarity - visualizations that are directly tied to a musical quality so that they are affected by a change in that quality, and those that are influenced “indirectly”, relying on the artist to create an effect in the visual in response to their perceptions of the music. I am not trying to make a claim right now that one is better than the other, although a direct tie between musical and visual expression would make the visual more of an information visualization. It would be based on something quantifiable.

I recently read a post on Create Digital Music about the makers of a new virtual audio synth called Circle. What the makers at Future Audio Workshop (FAW) aim for with this design is to keep you in flow while you produce your music and sounds. In fact, they take care to make this reference to Mihály Csíkszentmihályi’s concept of Flow in their product description:

“Circle – stay in flow

Flow refers to the state of mind in which you are at your most creative and productive, a place where hours pass like minutes and musical ideas become reality. Staying in flow becomes increasingly difficult however, when dealing with the complexities of modern software synthesizers.”
- Taken from FAW’S site

Flow refers to a sweetspot when it comes to humans doing things. Symptoms of Flow include time passing quickly, being focused, not distracted, feeling adequately challenged, feeling in control and not stressed, enjoying yourself. These are the characteristics of a person in Flow, and an interface can play a big role in helping a person reach this state in one’s activity. Perhaps Flow can be somehow related to the “unmonitored” feeling that is necessary for authentic creation - (think about component #3 - A loss of the feeling of self-consciousness).

Back to Circle and the interface:

I would love to experience how this interface measures up to Ableton Live in usability. What I have noticed with past midi sequencing software such as Logic, Cubase, Reason is that, even ignoring the fact that the learning curve is high, the interactions are somewhat longwinded or nondescriptive. Longwinded in that they take too many steps, involve disparate controls or menus etc. I know that much software is not intended for performance, but even if you had all the time in the world to tweak a loop which was never to be performed live, you would still want the process to be intuitive and feel hands on so as not to drain on your creative energies.

I would love to see these principles of workflow and usability at the center of more sound software, especially djing programs like Traktor and Serato, where flow in performance is key (haven’t been too happy with either). Information design/visualization can also be utilized in conjunction to surface what is happening digitally to the sound - and to give different forms of feedback.

The new audio-visual software instruments that will be developed now and in the future need to take these principles seriously if they wish to be considered more instrument than computer program.

I guess I should put my $ where my mouth is and post my ideal djing interface soon.

In the face of the challenge of making our creative interfaces more transparent, I do think that there is a limit - and that ….

Interfaces are necessary for the creative process

Aren’t they? I mean, we have all thought about plugging our brains in directly to manipulate sounds or images, to get total control without requiring interface (um, haven’t we all?) But the best interfaces are actually engaging to deal with and add to the final creation, shaping the output and allowing us to be submerged in a process.