% Options for packages loaded elsewhere
\PassOptionsToPackage{unicode}{hyperref}
\PassOptionsToPackage{hyphens}{url}
%
\documentclass[
  a4paperpaper,
]{article}
\usepackage{amsmath,amssymb}
\usepackage{iftex}
\ifPDFTeX
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
  \usepackage{textcomp} % provide euro and other symbols
\else % if luatex or xetex
  \usepackage{unicode-math} % this also loads fontspec
  \defaultfontfeatures{Scale=MatchLowercase}
  \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1}
\fi
\usepackage{lmodern}
\ifPDFTeX\else
  % xetex/luatex font selection
\fi
% Use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
\IfFileExists{microtype.sty}{% use microtype if available
  \usepackage[]{microtype}
  \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\makeatletter
\@ifundefined{KOMAClassName}{% if non-KOMA class
  \IfFileExists{parskip.sty}{%
    \usepackage{parskip}
  }{% else
    \setlength{\parindent}{0pt}
    \setlength{\parskip}{6pt plus 2pt minus 1pt}}
}{% if KOMA class
  \KOMAoptions{parskip=half}}
\makeatother
\usepackage{xcolor}
\setlength{\emergencystretch}{3em} % prevent overfull lines
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\setcounter{secnumdepth}{-\maxdimen} % remove section numbering
\ifLuaTeX
  \usepackage{selnolig}  % disable illegal ligatures
\fi
\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}
\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available
\urlstyle{same}
\hypersetup{
  pdftitle={The Kaleidoscope Constraint: The automation of arts, seen from its dead ends},
  pdfauthor={Florian Cramer, Willem de Kooning Academy, Rotterdam},
  hidelinks,
  pdfcreator={LaTeX via pandoc}}

\title{The Kaleidoscope Constraint: The automation of arts, seen from
its dead ends}
\author{Florian Cramer, Willem de Kooning Academy, Rotterdam}
\date{}

\begin{document}
\maketitle
\begin{abstract}
Since Renaissance Lullism, the history of computational and
computer-generated literature has been a history of dead ends, reflected
among others in Jonathan Swift, Georges Perec, Italo Calvino, Franz
Josef Czernin, and more recently in the fields of Internet and machine
learning-based literature. These crises and impasses are rooted in what
I would propose to call the kaleidoscope constraint of literary
computation. The historical development of computational methods and
technologies, from simple combinatorics to Markov chains to neural
networks, has not removed this constraint, since it has only brought (to
borrow from finance) a quantitative easing of a qualitative problem. --
In the course of sketching this history, I would also like to put some
question marks behind common understandings of ``digital'' and
``creativity''.

Seit der frühneuzeitlichen ars combinatoria ist die Geschichte der
generativen und computererzeugten Literatur eine Geschichte von
Sackgassen und Krisen, die u.a. von Jonathan Swift, Georges Perec, Italo
Calvino und Franz Josef Czernin beschrieben wurden und sich in jüngster
Zeit auch auf Netzliteratur und machine learning-basierte Texterzeugung
erstrecken. Diese Krisen haben ihren Ursprung in dem, was ich
``kaleidoskopischen Formzwang'' berechneter bzw. generativer Literatur
nennen möchte. Dieser Formzwang wurde in der historischen
Weiterentwicklung von Rechenmethoden und -techniken -- von der einfachen
Kombinatorik über Markov-Ketten bis hin zu neuronalen Netzen -- nicht
aufgehoben, sondern (in der Sprache des Finanzmarkts) nur quantitativ
gelockert. Im Zuge dieser skizzierten Geschichte möchte ich auch
Fragezeichen hinter gängige Begriffe des ``Digitalen'' und der
``Kreativität'' setzen.
\end{abstract}

(lecture at the conference \emph{Automation and Creativity: Practice,
Aesthetics and Reception of the Digital in Music and Literature}, TU
Braunschweig, 8-10-2020)

\hypertarget{dead-ends-and-crises-in-computational-poetics}{%
\section{Dead ends and crises in computational
poetics}\label{dead-ends-and-crises-in-computational-poetics}}

The likely reason for my invitation to this conference is my previous
work on the automation of literature, including my 2006 dissertation, in
which I attempted to write a history of computational literature from
Renaissance ars combinatoria to contemporary Internet art, including its
antecedents in ancient and medieval poetry and mysticism. I have only
sporadically revisited this topic since then, because my thesis ended in
a dead end: that throughout its long but often overlooked history,
computational poetry has been characterized by a structural discrepancy
between the speculative imagination of the powers and potentials of
computational language and the fragmentary and often disappointing
results of these experiments.

In other words, the poetics of automated literature was more interesting
than its poetry. With Renate Lachmann, I therefore read this history and
genre of literary writing as fantastic literature, implicitly as
belonging to a literary anthropology of speculation (to borrow a term
from Gert Mattenklott). The speculative surplus of this poetics meant
that the rather simple act of shuffling words in a poem took on very
different meanings in different centuries and different poetics: as
ecstatic mysticism or as technocratic constructivism, as rules or as
randomness, as formalist regimes or as aleatoric anarchy, as satanic or
as Christian, as classicism or as anti-classicism, to name but a few.

\hypertarget{lullism}{%
\section{Lullism}\label{lullism}}

The speculative poetics of medieval and Renaissance ars combinatoria and
lullism are summarized in Jonathan Swift's \emph{Gulliver's Travels} as
follows:

\begin{quote}
``The first Professor I saw was in a very large Room, with Forty Pupils
about him. After Salutation, observing me to look earnestly upon a Frame
{[}\ldots{]} by his Contrivance, the most ignorant Person at a
reasonable Charge, and with a little bodily Labour, may write Books in
Philosophy, Poetry, Politicks, Law, Mathematicks and Theology, without
the least Assistance from Genius or Study. He then led me to the Frame,
about the Sides whereof all his Pupils stood in Ranks. It was Twenty
Foot Square, placed in the Middle of the Room. The Superficies was
composed of several Bits of Wood {[}\ldots{]}. They were all linked
together by slender Wires. These Bits of Wood were covered on every
Square with Papers pasted on them; and on these Papers were written all
the Words of their Language {[}\ldots{]}. The Professor then desired me
to observe, for he was going to set his Engine at work.''\footnote{Swift,
  Jonathan. \emph{Gulliver's Travels}. Edited by Herbert Davis, Basil
  Blackwell, 1965.}
\end{quote}

Swift's satire is effective because it does not actually exaggerate
Renaissance Lullism, but describes it very factually. In the 17th
century, the German polymath Johann Joachim Becher had constructed an
almost identical apparatus for machine translation from Latin into
modern languages,\footnote{Becher, Johann Joachim. \emph{Character pro
  notitia linguarum universali}. Frankfurt, 1661}, and the poet and
mystic Quirinus Kuhlmann had envisioned nineteen combinatorial and
automated arts and sciences, including an automated ``ars magna librum
scribendi'' that would be ``of such perfection that no mortal being
would be able to write a book that wasn't already contained in our Ars
scribendi``.\footnote{``tanta perfectione, ut nullus Mortalium librum
  edere posset, quem nostra Ars scribendi non comprehenderet'', Quirinus
  Kuhlmann, \emph{QUIRINI KUHLMANNI PRODOMUS}. Amsterdam: Lotho de Haes,
  1674.}

The poetic and scientific Lullism of the 17th century failed because its
machines never amounted to more than theoretical sketches or prototypes
that overpromised but underdelivered. By the early 18th century, when
scholasticism was replaced by empirical science and, in the arts,
rule-based poetics by the new paradigm of aesthetics, the Lullist
project of automated creativity had logically come to an end. Swift only
had to summarize it to make fun of it. In a classic paradigm shift (as
defined by Thomas Kuhn), ars combinatoria and automated creativity were
marginalized and lived on in the niches of games, children's books,
humorism, and the occult.

When Raymond Queneau revived the literary ars combinatoria in 1961 with
his \emph{One Hundred Thousand Billion Poems}, this was still in the
spirit of ludics and children's games.\footnote{Queneau, Raymond.
  \emph{Cent Mille Milliards de Poèmes}. Gallimard, 1961; English
  translation in Mathews, Harry, and Alastair Brotchie, editors.
  \emph{Oulipo Compendium}. Atlas Press, 1998.} The poems initiated the
Oulipo group of literary writers and mathematicians, including Georges
Perec and Italo Calvino. The group formally operated as a section of the
Parisian Collège de Pataphysique, which was modeled on Alfred Jarry's
poetic-absurdist science of pataphysics. The new automation of
literature in the twentieth century thus did not erase Swift's epitaph,
but confirmed his framing of literary combinatorics as organized
silliness.

The Oulipo began its collective work in the same years that Marshall
McLuhan's book \emph{Understanding Media} was published.\footnote{McLuhan,
  Marshall. \emph{Understanding Media: The Extensions of Man}. 1964.}
McLuhan defined media, broadly synonymous with technology, as
``extensions of man''. This discourse has since dominated the discourse
of media and technology where new technologies are understood as
progressive function updates. The poetics of Oulipo, however, was the
opposite, understanding techniques and technologies, including
computation and algorithms, as ``constraints'' rather than extensions.

In 1968, Georges Perec wrote a radio play for the German Saarländischer
Rundfunk, ``Die Maschine'' (``The Machine'') in which a simulated
computer processes Goethe's \emph{Wanderer's Nightsong} using the very
formal text processing methods developed in the Oulipo.\footnote{Perec,
  Georges. \emph{Die Maschine}. Stuttgart: Reclam, 1972.} This
processing anticipates much of the algorithmic text processing done
today in the Digital Humanities. But it ends up as an ultimately futile
and pointless exercise in constraining a literary text.

Since 1959 and parallel to the Oulipo, the \emph{Stuttgarter Schule}
(``Stuttgart School'') of experimental poets around Max Bense, Helmut
Heißenbüttel und Reinhard Döhl had created computer-generated poetry
which it called ``artificial poetry''.\footnote{Lutz, Theo.
  ``Stochastische Texte.'' Augenblick, vol.~4, no. 1, 1959, pp.~3--9.}
The dead end in Perec's \emph{Machine} ended up becoming the dead end of
artificial poetry, too. According to Döhl, Perec's radio play had such a
crushing effect on the \emph{Stuttgarter Schule} that it gave up
computer generative poetry:

\begin{quote}
``A radio play, which {[}\ldots{]} seemed like a preliminary end point
to us, who had come from writing to computers. {[}\ldots{]} We didn't
pursue these approaches further at that time, except in lectures and
discussions, but rather broadened our interest in artistic production
using new media and writing systems in other directions''.\footnote{``{[}E{]}in
  Hörspiel, das {[}\ldots{]} uns, die wir ja vom Text zum Computer
  gekommen waren, wie ein vorläufiger Schlussstrich erschien.
  {[}\ldots{]} Wir haben diese Ansätze außer in Vorträgen und
  Diskussionen damals nicht weiter verfolgt, sondern unser Interesse an
  künstlerischer Produktion mit Neuen Medien und Aufschreibsystemen in
  andere Richtungen ausgedehnt''. Döhl, Reinhard. ``Vom Computertext zur
  Netzkunst. Vom Bleisatz zum Hypertext''. \emph{Liter@tur: Computer --
  Literatur -- Internet}, Hansgeorg Schmidt-Bergmann and Torsten
  Liesegang (eds.), Bielefeld: Aisthesis, 2001, 27--50.}
\end{quote}

In his 1967 lecture \emph{Cybernetics and Ghosts}, Oulipo member Italo
Calvino saw computer-generated literature at a similar impasse when he
concluded that ``the style of a literary automaton'' and ``its true
vocation would be for classicism'', in the sense of ``traditional works,
poems with closed metrical forms, novels that follow all the
rules''.\footnote{Calvino, Italo. ``Cybernetics and Ghosts.'' \emph{The
  Uses of Literature}, Harcourt, 1982, pp.~3--27.} In the same year, the
American novelist John Barth criticized experimental poetry in general
as a \emph{Literature of Exhaustion} which exhausts itself in the
material creation of proteic variations rather than more simply
imagining them, as for example in the prose of Jorge Luis Borges -- or,
one might add, Jonathan Swift.\footnote{Barth, John. ``The Literature of
  Exhaustion.'' \emph{The Friday Book}, 1984, pp.~62--76.}

The history of literary computing and generative poetry thus can not
only be told as a history of impasses, but perhaps must even be told
from the perspective and vantage point of its periodic crises since the
collapse of the ars combinatoria at the turn of the 17th and 18th
centuries. More recent examples include the attempts and ultimate
frustration and failure of the Austrian experimental poet Franz Josef
Czernin to write computer-aided poetry with the experimental language
processing software ``POE'' in the 1990s,\footnote{Czernin, Franz Josef,
  and Ferdinand Schmatz. \emph{Teller und Schweiss. Gedichte aus POE}.
  Pakesch \& Schlebrügge, 1991} the short-livedness of various schools
of electronic literature since the invention of the World Wide Web,
including German-language ``Netzliteratur'' and its collapse in the
early 2000s. Contemporary electronic literature suffers from the problem
that its scholars either outnumber its writers or, as Álvaro Seica has
analyzed, are even identical with them.\footnote{ Seiça, Álvaro.
  ``Digital Poetry and Critical Discourse: A Network of
  Self-References?'' 2182-8830, Centro de Literatura Portuguesa Imprensa
  da Universidade de Coimbra, 2016. Many scholars formerly active in
  this area have moved on to such fields as computer game studies,
  digital humanities and cultural studies of computing and technology.}

It remains to be seen whether the current boom of artificial
intelligence text generation using neural networks and machine learning
will produce similar frustrations. A recent ``Guardian'' op-ed article
written by the currently most advanced artificial intelligence language
generator GPT-3, suggests that the technology does not resolve the
issues stated by Calvino and others.\footnote{https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3
  . Not only leaves the text much to be desired, it also turned out that
  it needed to be heavily edited by human editors:
  https://www.theguardian.com/technology/commentisfree/2020/sep/11/artificial-intelligence-robot-writing-gpt-3}
Realistically, this form of automated creativity is destined to take
over major parts of vernacular text production (from articles to reports
and helpline chat bots) and thus find indirect ways into literary
writing.

\hypertarget{structural-issues}{%
\section{Structural issues}\label{structural-issues}}

From Lull to GPT-3, there have been two, maybe three major modes -- or
techniques -- of language computation and machine-generated literature:

\begin{enumerate}
\def\labelenumi{\arabic{enumi})}
\item
  synthetic computation of pre-defined elements according to pre-defined
  (or ``hard-coded'') rules, in a period from the 13th to the 20th
  centuries -- with historical origins in the medieval Kabbalah and
  Lullist combinatorics and continuing up to so-called Symbolic
  Artificial Intelligence (``Symbolic AI''). Examples include the word
  permutation poems of the Renaissance period, the machine described by
  Swift in \emph{Gulliver's Travels}; computational poetry of the Oulipo
  including Raymond Queneau's 100.000 Billion Poems; but also more
  recent machine translation systems such as the Internet service
  \emph{Babelfish}.
\item
  analytical computation of arbitrary text input by pre-defined rules or
  algorithms, in the 20th and 21st centuries. It began with Andrei
  Markov's invention of Markov chains and his processing of Pushkin's
  \emph{Eugene Onegin} in 1906, continued with Tristan Tzara's 1920
  instructions to create Dadaist poetry by cutting out and shuffling
  words from any newspaper article, Brion Gysin's and William S.
  Burroughs' cut-ups in the 1950s, and literature written with the help
  of Markov chains such as Max Bense's \emph{Der Monolog der Terry Jo}
  (1968), Jackson Mac Low's ``diastic'' poems from the 1980s, Charles O.
  Hartman and Hugh Kenner's \emph{Sentences} (1995) and Ray Kurzweil's
  \emph{Cybernetic Poet} from 2001.\footnote{Hartman, Charles O., and
    Hugh Kenner. Sentences. Sun and Moon Press, 1995.}
\item
  analytic computation of arbitrary input through dynamically adapted
  rules -- such as in machine learning -, with the present-day example
  of GPT-3.
\end{enumerate}

While language computation has arguably become more sophisticated with
each new technique, none of them has changed the fundamental structure
and constraint of language computation: mathematical shuffling of
predefined (textual) material. This is what I propose to call the
\emph{kaleidoscope constraint} of generative literature, and of
automated creativity in general. But before I explain it in more detail,
let me first digress on the notions of ``digital'' and ``creativity''.

\hypertarget{digression-digital-creativity}{%
\subsection{Digression: digital,
creativity}\label{digression-digital-creativity}}

In the history I have presented, electronic computer technology plays an
important but not decisive role, since most literary computations do not
fundamentally depend on it. But this is not to deny digitality.
Digitality simply needs to be placed into a larger historical
perspective. Since the standardization of alphabets and the equation of
letters with numbers in the Greek and Hebrew antiquity, writing in
Western culture has been digital; not in a metaphorical sense, but in
the literal scientific definition of ``digital'' as information encoded
in discrete, countable units.\footnote{This is also how Goodman, Nelson.
  \emph{The Languages of Art}. Hackett, 1976, defines digital as opposed
  to analog.} The early digitization of language into alphabets was a
necessary precondition for any combinatorial poetics, including the
calculation of names as numbers in the Kabbalah and older schools of
mysticism and magic.\footnote{As described in Dornseiff, Franz.
  \emph{Das Alphabet in Mystik und Magie}. Teubner, 1925.}

Second, with Oulipo's concept of the constraint along with the
observation that automated literature boils down to shuffling linguistic
units, one might ask whether computational poetics is \emph{uncreative}
rather than creative. Here I borrow from the poet Kenneth
Goldsmith.\footnote{Goldsmith, Kenneth. \emph{Uncreative Writing:
  Managing Language in the Digital Age}. Columbia University Press,
  2011.} Conversely, the notion of ``automated creativity'' risks
reinforcing a problematic romanticist, and nowadays
neoliberal,\footnote{Such as in the policymaking concepts of the
  ``creative class'' and ``creative cities'' in Florida, Richard.
  \emph{The Rise of the Creative Class} Hachette UK, 2002 and Landry,
  Charles. \emph{The Creative City: A Toolkit for Urban Innovators}.
  Earthscan, 2012.} concept of art as creativity.

\hypertarget{kaleidoscope-constraint}{%
\section{Kaleidoscope constraint}\label{kaleidoscope-constraint}}

Constraints, exhaustion and dead-ends of generative systems are not only
a matter of poetics, but above all of aesthetic experience. This
experience kicks in soon enough when reading permutation poems or robot
journalism, watching algorithmically generated visuals and listening to
computer-generated music.

Whatever the technique or technology, `automated creativity' not only
boils down to remixing, but to remixing as an aesthetic end in itself --
unless these computations are being tactically used outside their own
logic.\footnote{As for example by the YesMen for their spoof of the
  World Trade Organization website: Bichlbaum, Andy, et al.~\emph{The
  Yes Men: The True Story of the End of the World Trade Organization}.
  Disinformation, 2004.} The constraint of any generative system is that
the recombination of elements means almost infinite possibilities in
theory, but quickly becomes dull and pointless in practice.

As a result, the promise of extension conflicts with the reality of
constraint. This concerns not only automated literature, but any
programmed system. It is the same aesthetic effect as watching the
visual shapes generated by a kaleidoscope -- an instrument whose early
history is part of the ars combinatoria of the 17th century since it was
co-invented by the Lullist polymath (and later correspondent of Quirinus
Kuhlmann) Athanasius Kircher. Machine-generated art and automated
creativity are, effectively, glorified kaleidoscopes. Therefore, I
propose the concept of the ``kaleidoscope constraint'' for computational
poetics and aesthetics, and use it to explore a poetics that does not
exhaust itself in it.

The history of computational poetics could be told as a history of
quantitative easing of the kaleidoscope constraint: that is, burying a
structural problem by throwing more resources at it. Since the emergence
and institutionalization of computer-generative arts on a larger scale,
this quantitative easing has masked a much smaller degree of qualitative
(e.g.: aesthetic and hermeneutic) ``gains''. In doing so, it has also
perpetuated the McLuhanian and techno-Hegelian narrative of extension
rather than constraint, and fostered a collective failure to think of
any extension as simultaneously a restriction.

\hypertarget{postscript-on-machine-learning}{%
\section{Postscript on machine
learning}\label{postscript-on-machine-learning}}

What are the foreseeable limits of neural-network-based ``machine
learning'' algorithms (which are now often, but problematically,
understood as synonymous with the larger field of ``artificial
intelligence'')?

First and foremost, they are pattern recognition systems; algorithms
that find regularities in unstructured data sets. They are therefore
primarily perception systems, not creation systems. However, when using
their pattern recognition for information synthesis, they reconstruct
information that is not actually contained in a data set, such as object
details in a blurry photograph or, hypothetically, missing words in a
fragmented text.

Machine learning algorithms are therefore likely to become part of
ordinary word processing software, where they could, among other things,
auto-complete sentences that consist only of a beginning or a few
keywords. Their ``learning'' is based purely on heuristics --
probabilities found in the training material. As a result, these
algorithms construct the present from the past; or, more simply, they
constantly stereotype.

An advanced machine learning system trained on nineteenth-century
realist novels as a data set would forever write novels in the style of
nineteenth-century realism (or rather, remixes of those novels), but it
would never be able to write \emph{The Making of Americans}, to take one
example. Italo Calvino's diagnosis of classicism, or conservatism and
conventionalism, probably applies even more to machine learning than to
the combinatorial poetics he studied with the Oulipo in the 1960s.
{[}Added in 2023:{]} Paradoxically, the extension of the capabilities of
generative systems seems to have led to even greater aesthetic
limitations than fifty years ago; just like the kaleidoscope, which --
having ``matured'' into today's AI image generators -- no longer
generates abstract shapes but photorealistic images.

\end{document}
