When Designs Flicker

Florian Cramer

April 30th, 2006

In 1976, developmental psychologists Harry McGurk and John MacDonald dubbed a video recording of a person prouncing the word "ka" with the a sound recording of a person uttering the syllable "ma." Unlike in movies dubbed into a different language, listeners actually "heard" the syllable formed by the lip movements as opposed to the one that was spoken on the audio track. Alternatively, the audio-recorded syllable "ba" synched with the lip pronounciation of "ga" results in a perception of the word "da."¹ Various demonstrations of this phenomenon exist in the web, for example on the site http://www.media.uio.no/personer/arntm/McGurk_english.html, proving not only that the effect works, but that it functions even when the video has only a poor resolution and image quality below older analog home video standards. The "McGurk effect" (as it is officially called) demonstrates that what we see can actually alter what we hear. It also shows that different sense perceptions can not be isolated, but interfere, thus rendering classifications and separations of single "media" more problematic than it conventionally seems. The effect is also shaking our common assumptions of the physicality of sense perception, showing how perception is neither passive, nor objective, but literally means to make sense. Feeding those makings of sense back to the senses, Dragana Antic's Sounds of the Qualia is a precise reflection of this. When user of her installation hear their own footsteps anew, as unfamiliar sounds, behavioral automatisms get disrupted and exposed. Just like in one's puzzled observation of the McGurk effect, perceptional awareness results not only in heightened critical awareness of oneself in relation to one's environment. The deconditioning also reveals the subconscious and imaginary.

*0.8! Figure

Figure 1: Robert Fludd, Picture from Utriusque cosmi historia

A famous illustration in the Art of Memory chapter in the book Utriusque Cosmi Historia ("History of Macrocosm and Microcosm") of the 17th century hermetic philosopher Robert Fludd shows a man "seeing" in his mind images of among others an obelisk and the Tower of Babel through his "eye of the imagination" (oculis imaginationis) - depicted in an itself imaginary way as a third eye, located on the person's forehead.² It is a powerful early depiction of how imagination (literally) means to shape images in one's mind that don't exist in front of one's two actual eyes. This renders Fludd's illustration an emblem of the fantastic, utopian, (literally) visionary and eccentric. Remarkably, the visions depicted include architectural designs. It becomes indeterminable whether they are pre-existing forms and signs shaping the imagination, or new forms shaped by the imagination. What's more, the question of whether they fit the conventional category of "art" or "design" as in "applied arts" becomes entirely obsolete.

Our consciousness, it follows, is intrinsically interwoven with designs. Constructed objects and spaces map a mental territory. They have, as Fludd's illustration shows, an inherent dialectic of both constraining and opening up our imagination: Limiting it - likely, but not necessarily - to Euclidian dimensions, anthropomorphic measures and cultural archetypes on the one hand, elevating it beyond the confines of the ordinary on the other.

*0.39! Figure *0.39! Figure

Figure 2: El Lissitzky, two pages from About 2 [Squares], 1920

The children's book About 2 [Squares], created by the Russian constructivist El Lissitzky in 1920, shifts such imaginary designs to a scenario of reinventing culture through redesign. Similar to a comic or flip book, it tells, in six consecutive full page images, the abstract story of a red and a black square flying through space, landing on a planet that is governed by a "storm" of black objects; the squares explode them and build a new order.³ The book conceives of itself as a loose instruction code by telling its young readers "not to read," but act out the story with paper, sticks and bricks. It thus unifies imagination and objects, the abstract and the concrete, and becomes itself a building block of something new. Cheryl Gallaway's Open Wardrobe functions in a very similar way. It, too, mobilizes art and design to make them infrastructures, understanding them as world-making in a literal, not only metaphorical sense. However, the building blocks - garments - are not conceptualist, but anthropomorphic and intimately physical, microcosmic and not macrocosmic. By this, and by subjecting its social software code ultimately to the community, it expresses a humble scepticism not only regarding, but also in its design.

*0.49! Figure *0.49! Figure

Figure 3: El Lissitzky, Beat the Whites with the Red Wedge!, 1919; Design for The Little Goat, 1917

Just like Fludd's illustration, Lissitzky's story of the squares collapses traditional differentiations of "art", "design" and speculative thinking. Being not humble however, he writes a straight-forward program of revolutionary modernism that, with its lurking missionary militance and unification ideology, is admittedly problematic. In a city like Rotterdam whose architectural modernism is the historical result of Germany having bombed down the city in the Second World War, Lissitzky's story of objects flying from the sky to explode old architectures becomes highly ambivalent. Yet, the iconography of Lissitzky's story is complex and has an anti-fascist background. The fourth image, a red square vertically crushing the old order of things, paraphrases a billboard he had designed in the year before, the famous Beat The Whites with The Red Wedge. The latter, propaganda for the Red Army in its fight against the monarchist white troops, specifically addressed the Jewish population of Russia: Subverting the slogan Beat The Jews, it reminded viewers of the antisemitism of the right-wing forces. In addition, it reused the Jewish iconography of his 1917 illustration to the Haggadah children's story The Little Goat, showing an angel who punishes slaughter on earth by pulling his sword in heaven and striking it down to earth.⁴

Lissitzky's subsequent abstraction of the motif not only secularizes it. The image turns into a general reflection of how that which in Fludd's depiction was limited to imagination could be made material designs and spaces which would recursively allow new imaginations to develop. Lissitzky's designs could therefore be seen as even more speculative and fantastic than Fludd's imaginary architectures. They render themselves no less intricate figures of reflection despite their shift from metaphysics to design.

Postmodern art criticism, set off among others with Jean-François Lyotard's Postmodern Condition in 1979 and Rosalind Krauss' The Originality of the Avant-Garde and Other Modernist Myths in 1981,⁵ nowadays has degraded into a routine bashing of 20th century avant-garde modernism and an ostensive contempt among curators and artists for problematizing form and design. What tends to be overlooked in avant-garde modernism, and lost in a superficial understanding of design, are its experimental and speculative projects to not merely comment upon, but actually adjust reality. Radical reinvention of languages was a key program of the 20th century avant-gardes. It is encouraging to see a student project like Sasson Kung's Love Language System no longer refraining from such an endeavor, not restraining itself to merely criticizing codes. The Love Language System solves its design issue, the utopia of a universal language, in a humble and playful way. Even if it were less modest, dismissals of language reinvention has become cheap talk, too, that deconstructs itself in its own metaphysical assumption that language, and codes, wouldn't be cultural constructions that could be altered, and constructed differently.

*0.39! Figure *0.39! Figure

Figure 4: Design of Brion Gysin's Dreamachine, David Woodard and William S. Burroughs in front of a Dreamachine

Reality adjustment thus is made on the very level of the signs and objects that make up a world. Tsila Hassine's Ctrl-F(r)eader takes a reciprocal approach to Sasson Kung's synthesis of letters and words by constructing an analytical device that shapes our perception of written language, and thus effectively the language itself. By showing how words in the Internet achieve their meanings through different contexts, the Ctrl-F(r)eader maps language as a social product. If the use of the software conversely influences human understanding and usage of language, a strange feedback process results, triggered by the program's filtering politics.

The McGurk effect shows that no third eye like in Robert Fludd's image is needed to impose subjective over physical reality, but that the interference already happens with the two eyes we have. Another examples is the flicker effect, artistically employed in Brion Gysin's Dreamachine and Tony Conrad's experimental films from the 1960s, triggering color and visual form hallucinations when a stroboscope light meets a frequency of around 30 Hz. Yet another is the fact that we can hear sounds from a sound source that isn't physically capable of reproducing them - such as low musical notes from a cheap transistor radio - because our brain automatically reconstructs them solely from their overtone spectrum. Lossy audio and image compression codecs such as MPEG and JPEG are based precisely on such "psycho-acoustic" and psycho-visual phenomena.

*0.8! Figure

Figure 5: Peter Kubelka in front of the laid-out footage of his 1960 film Arnulf Rainer

If perception doesn't match physical laws, it also means that synthetic composition methods in art and design based on strictly physical parameters are founded on false aesthetic premises. This in turn affects 20th century formalist modernisms: constructivist art with its foundation on the square grid for example, and avant-garde music from Arnold Schönberg's dodecaphony to Karel Goeyvaerts', Pierre Boulez' and Karlheinz Stockhausen's serialism. The latter boiled down to a total physical parameterization of sound and its subsequent arrangement into permutation rows. It resulted in a complexity of polyphonic patterns that could be grasped only through formal analysis of the score, but rarely or not all by the human ear. Likewise, Peter Kubelka's abstract film Arnulf Rainer from 1960, composed only of monochrome black and white frames according to a serialist method, is a visually impressive composition and design on the photographs that show the film laid out as a vertical grid on a wall. But this quality is lost, and the film appears like flicker, when its projection dissects it in time rather than unfolding it in contiguous space.⁶

Today, this film reads as an early radical example of digital art in which a binary code functions as both source and visuals, structure and perceivable result. It coincides, both historically and aesthetically, with the first manifesto of permutational art of the French information theoretist Abraham M. Moles from 1962.⁷ This pamphlet, published originally in German and later expanded into a book Art and Computer (Art et Ordinateur,⁸ proposes to refound the arts upon the algorithmic combinatorics of their particular elements and physical parameters: serial and aleatory composition in music, word permutations in experimental poetry, constructivist abstraction in painting. According to Moles, the objective is to "narrow down and exhaust the field of possibilites accessible through a `set' of rules" for the composition of the material.⁹ It was, in other words, a radically formalist variant of the belief that the medium is the message. Along the lines of Claude Shannon's information theory, Moles proposed to conversely conceive of aesthetic perception as computational. Aesthetic criticism should therefore be based on a reverse-engineering that "determines the redundancy of artistic messages through the coding rule of its combinatorics".¹⁰

With this, Moles set a classical agenda of what first was called cybernetic, later electronic and eventually media arts and design. But already in his writing, the formalisms become eccentric and border on phantasmagorias. He takes, for example, Sade's novel The 120 Days of Sodom as a model of a computational eroticism, and thus transgression and reinvention of culture. It took three decades until the later 1990s when such eccentricity, the surrealism of formalisms, found its expression in the digital arts, in the works of Dutch artists (and former Piet Zwart media design research fellows) jodi, or in the frequent pastiches and remobilizations of abstract art, constructivist graphic design and typography in the pop cultural design of record covers, flyers and web sites.¹¹

When binary information turns into flicker, causing hallucinations, it reveals wider implications of aesthetics and aesthetic designs that disprove easy critiques of "aestheticism." The McGurk and flicker effects are so profoundly disturbing because they reveal cracks in our reality and the shakiness of cognition. In turn, they provide hooks for designs to hack conditionings. Since the Greek word "aisthesis" means "perception," such effects are simple yet radical examples of how aesthetics concerns the human condition.

Aesthetic theory has addressed this dimension since its very beginning. Kant's notion of the dynamically sublime refers to a force that exerts a vortex-like power upon us while, being a perceptive phenomenon, leaving the freedom of reflection and thus raising human self-awareness.¹² Friedrich Schiller expanded Kant's notion into a modality of artistic designs by conceiving of the sublime as a result not only of natural, but also human-made phenomena.¹³

Designs of systems or `media' to change perception and cognition of ourselves, our environment and communities: This could be called the common yet radical denominator of the audio installations, information filters, alternative language systems and garment economies in the graduation projects of Dragana Antic, Tsila Hassine, Sasson Kung and Cheryl Gallaway. Interestingly, they all apply designs onto industrially designed objects, attempting to reprogram and reappropriate the latter: audio processing to footsteps in shoes that were typically designed to avoid noise, linguistic algorithms and linguistic type design onto the Internet, and community web sites onto clothing, carving out physical intimacy - feeding back body motion, expressing love, wearing stuff - and conditions of thought. May these designs flicker!

References

[Bir73]: Alan C. Birnholz. El Lissitzky and the Jewish Tradition. Studio International, 186(959):133, 1973.
[Kan90]: Immanuel Kant. Critique of Judgement. Oxford University Press, Oxford, 1978 (1790).
[Kra81]: Rosalind Krauss. The Originality of the Avant-garde and Other Modernist Myths. MIT Press, Manchester, 1985 (1981).
[Lis91]: El Lissitzky. About 2 [Squares]. MIT Press, Cambridge, 1991.
[Lyo79]: Jean-François Lyotard. The Postmodern Condition. Manchester University Press, Manchester, 1984 (1979).
[MM79]: Harry McGurk and John MacDonald. Hearing lips and seeing voices. Nature, 264:746-748, 1979.
[Mol62]: Abraham A. Moles. erstes manifest der permutationellen kunst. Stuttgart, 1962.
[Mol71]: Abraham A. Moles. Art et Ordinateur. Casterman, Paris, 1981 (1971).
[Sch93]: Friedrich Schiller. Of the Sublime, 1986 (1793). http://members.aol.com/abelard2/schiller.htm.
[TJ95]: Peter Tscherkassky and Gabriele Jutz. Peter Kubelka. PVS Verleger, Wien, 1995.
[WPD03]: Künstlerhaus Wien, Norbert Pfaffenbichler, and Sandro Droschl, editors. Abstraction Now. Edition Camera Austria, Wien, 2003.
[Yat65]: Frances Yates. The Art of Memory. Routledge & Kegan Paul, London, 1965.

Footnotes:

¹[MM79]

²[Yat65], fig. 16

³[Lis91]

⁴[Bir73]

⁵[Lyo79], [Kra81]

⁶[TJ95]

⁷[Mol62]

⁸[Mol71]

⁹[Mol62], p. 3, my translation

¹⁰[Mol62], p. 2, my translation

¹¹The catalogue Abstraction Now, which includes jodi and former Piet Zwart media design research fellow Peter Luining, documents this tendency, [WPD03]

¹²[Kan90], §28

¹³[Sch93]