A Short History of Colorcoder
Before the artist Ellsworth Kelly became a pioneer of hard edge painting he
was a member of the 23rd Headquarters Special Troops unit, also referred to
as the ‘Ghost Army’. Deployed in France during World War II, the Ghost Army
was a U.S. military tactical deception unit assembled by actor Douglas
Fairbanks Jnr. It consisted of approximately 1,100 artists, actors, musicians,
and other creative types sequestered from art schools (Kelly was studying at
the Pratt Institute in New York at the time) and advertising agencies to design
camouflage and to construct fake military installations, rubber tanks, and to
fabricate soundscapes (acoustic techniques that were later used in the first
Gulf War, when speaker systems placed behind sand dunes, dictated the
movements of the battlefield by transmitting imaginary battle sounds).
The directive of the Ghost Army was to saturate the Nazi’s with duplicitous
acoustic signifiers of mass troop and artillery manoeuvres (mounted on
armoured vehicles, large speaker systems transmitted soundtracks in covert
locations so that the presence of mass numbers of soldiers and equipment
was fabricated) and to more generally transmit disinformation about the
numbers, plans, and whereabouts of the allied forces. More specifically Kelly
was part of the 603rd Camouflage Engineers within the wider unit alongside
fashion designer Bill Blass and photographer Art Kane. For Kelly’s part, he
was predominantly engaged in the creation of dummy and inflatable vehicles,
air-fields, and troop encampments (so that Nazi reconnaissance aircraft would
see them) and in the production of camouflage. You can see in his post-war
work how he begins to deviate from conventional camouflage, gradually
morphing into The Meschers of 1951, then taking the quantum leap in
abstraction to produce the 8ft squared Colors for a Large Wall (in the same
year) before finally honing in on this style to create 9 Squares.
In the early 50’s Kelly started imbuing his Color Field paintings with coded
messages relating to experiments and research carried out during WW2.
Covert information that holds clues to the composition of weaponised
frequencies that were deployed by Ghost Army operatives (the first ‘battle djs)
via 3 turntable set-ups. In response, AUDINT’s 1st wave of Bill Arnett,
Hippolyte Morton, and Walter Slepian (who also previously served in the
Ghost Army) start to mimic the 64 square formula of Colors for a Large Wall by
using inverted record covers and illustrating them with the pictorial and written
references that would aid their research into the painting’s meanings.
AUDINT’s current sound work is an App called the ‘Colorcoder’, which is
programmed to sonically decrypt Kelly’s paintings, extending the research
undertaken by Arnett, Morton, and Slepian.
Colorcoder Functionality
Each unique colour used within the painting has a palette of associated
sounds that span the audible range and can be layered in various ways to
create different sonic textures. Moving the AUDINT turntable icon around the
painting – using the mouse or other interaction modes – triggers sound that is
”representative” of Kelly’s colour palette.
These various ways of moving around the painting – ‘drawing’ a freehand path
from square to square using the mouse; selecting specific squares in a
particular order using the grid; bouncing around the painting at different
speeds and angles to cut across the squares in an irregular fashion; or
following a constant horizontal path back and forth between successive rows
or vertical path up and down between successive columns – equate to the
different ways our eyes might scan across it, taking it in.
A demo video explaining how to use the app can be located here:
https://vimeo.com/150266114
Overview of the interface:
The ‘HSB COLOUR, WAVELENGTH & FREQUENCIES’ section shows how
an area of the painting, determined by the size of the AUDINT turntable icon,
is sampled and averaged and displayed as the corresponding hue, saturation
and brightness values. It also shows the values of the “equivalent” wavelength
and frequency of light and the scaled down audible frequency.
Adjusting the ‘ICON SIZE’ dial(s) changes the dimensions of the AUDINT
turntable icon.
The ‘INTERACTION MODES’ section allows you to select from five different
ways of moving within the painting – through the mouse; an interactive grid; a
‘bouncing ball’ mode (mouse left click-drag-release anywhere within the
painting); and preset horizontal and vertical paths – with their respective
‘DRAG’ and ‘SPEED’ controls.
The ‘SONIC TEXTURES’ section allows you to layer up to 8 ‘OCTAVES’ of
sound from the lowest audible frequency derived from the hue. Selecting new
‘OCTAVES’ will load the associated audio files if they’re not already active –
“loading…” will appear above the ‘AUDIO ON’ toggle and any automatic
movement of the AUDINT turntable icon will freeze for a few seconds while it
completes.
Different combinations of octaves can be activated through a set of ‘OCTAVE
PRESETS’ which load (bottom row) and save (top row) these custom
configurations (we’ve filled the first four presets to get you started). When the
application is closed any changes you’ve made to these presets are saved for
the next time you open it.
This is Creative Software
This is creative software… not commercial software. Colorcoder was
developed in the C++ creative toolkit openFrameworks v0.9.0. While we are
happy to share it with others to play with, we do so with the proviso that it has
not been rigorously tested across a range of hardware/OS versions and may
not run on machines lower than the minimum spec. The application is provided
“as is” and under the terms of the BSD licence below.
Minimum Spec
Colorcoder is currently only available for OS X.
It was built on a MacBook Pro (Retina, 15-inch, Late 2013), 2.3 GHz Intel Core
i7, 16 GB 1600 MHz DDR3 running OS X El Capitan Version 10.11.1 and has
been tested on several Apple Mac computers down to the lowest spec of a
MacBook Air (11-inch, Late 2010), 1.4 GHz Intel Core 2 Duo, 2 GB 1067 MHz
DDR3 running OS X Yosemite Version 10.10.5 (which worked fine bar the
screen resolution).
We suggest running Colorcoder on Apple computers with the following
minimum spec:
•
•
•
•
multicore 64-bit processor;
min 2 GB RAM (4 GB recommended);
OS X 10.9 and above.
min screen resolution of 1280x800px
BSD Licence
Copyright (c) 2015, MONOMATIC
All rights reserved.
Redistribution and use in binary form, with or without modification, are
permitted provided that the following conditions are met:
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
FAQ
1. Running Colorcoder for the first time.
We’ll assume you know how to find the ‘Colorcoder_OSX_final’ ZIP file on your computer once
you’ve downloaded it. You may want to move it to a more convenient location such as your
‘Desktop’ folder. Double mouse left-click on the ZIP file and it will expand creating a
‘Colorcoder_OSX_final’ folder with the application and a ‘README’ text file inside. If you then
launch it in the usual way via a double mouse left-click on the application icon, OS X is likely to
open a dialogue box:
“Colorcoder_OSX is an application downloaded from the Internet. Are you sure you want to
open it?”
You do – and you won’t need to confirm this again. Colorcoder can take up to 30 seconds to
load – be patient.
To close the application make sure it’s in focus (mouse left-click on the title bar or anywhere in
the app window) and press the ‘esc’ button on your keyboard – or mouse left-click on the red
dot at the top left of the title bar.
2. How does what I’m hearing relate to a colour in the painting?
In computational terms we:
•
•
•
•
•
•
use the HSB colour model to sample and extract the hue, saturation and brightness
values of a single pixel – or the average of an adjustable square area of pixels – under
the AUDINT turntable icon;
use a simple function to calculate the wavelength of light with a colour “equivalent” to
this hue value;
convert this wavelength to it’s corresponding frequency;
44
pitch this frequency down by a factor of 2 – which effectively translates it to the
bottom of the audible frequency range;
‘stack’ 8 successive octaves up from this ‘base’ frequency;
and finally trigger audio files according to the extracted hue and the number of active
octaves.
For anyone interested we’ve included some of the code for this in the ‘Can I see the source
code?‘ FAQ below.
3. When I make the AUDINT turntable icon large it does’t seem to work properly in
'MOUSE' and 'BOUNCE' modes?
This is because the larger the icon the more pixels are being averaged to extract the hue. You
can see this in action in the two squares at the top of the ‘HSB COLOUR, WAVELENGTH &
FREQUENCY’ section – the left square displays the ‘subsection’ of the painting being
sampled, the right square the averaged colour. If the resulting average colour doesn’t closely
match the colour of a particular square in the painting it won’t trigger. This isn’t an issue for the
‘GRID’, ‘HORIZONTAL’ & ‘VERTICAL’ modes where the icon is always fully within a square at
some point… but it is for the ‘MOUSE’ and ‘BOUNCE’ modes where it might never wholly be
within a single square. Reduce the icon size to very small and it works much better in these
modes.
Also see the ‘What are the ‘MIDI Out’, ‘MIDI Panic’ and other toggles in the ‘SONIC
TEXTURES’ section for?‘ FAQ below. ‘ICON SIZE’ is certainly more responsive when
Colorcoder is used as a MIDI device.
4. What are the ‘MIDI Out’, ‘MIDI Panic’ and other toggles in the ‘SONIC TEXTURES’
section for?
These are ’surplus to requirements’ for most users and can effectively be ignored. They have
no effect on Colorcoder when it’s being used in ‘AUDIO ON’ mode. MIDI or Musical
Instrument Digital Interface is a technical standard that describes a protocol, digital interface
and connectors that allows a wide variety of electronic musical instruments, computers and
other related devices to connect and communicate with one another. Earlier iterations of
Colorcoder used the same colour to frequency computational process above but rather than
trigger audio files it converted these frequencies into MIDI Note and pitchbend data (while also
adding “expressive” controller data from the saturation value, volume from the brightness
value and pan from the relative position of the mouse) to trigger a sound patch in the Native
Instruments Massive software synthesiser.
The audio you hear in this version is the recorded output from Massive using a patch that we
considered both behaved and sounded ‘best’ with the output MIDI data. For those with a
knowledge of MIDI we’ve decided to keep this functionality in Colorcoder – it actually works
far more effectively as a MIDI device – although we provide no further explanation of how to
use it other than set the MIDI input port of your sound source to “Colorcoder MIDI Out”. We
also include a link to a ZIP file of some of our shortlisted Massive sound patches (although you
could use any MIDI software synth or hardware sound module).
5. Is Colorcoder going to be developed further?
A version for Windows 10 is in development and should be ready shortly. Over and above this
– possibly. Future iterations of Colorcoder may see the app developed into an interactive
gallery installation or a live performance tool.
6. Can I get the source code?
Colorcoder is not currently open source – but here’s some of the coding we developed to
convert a colour to MIDI which may be of interest.
7. Do you offer any support for Colorcoder?
No. We just don’t have the resources to offer any ongoing support. See the ’This is Creative
Software…’ section above…