A tissue of citations

I’ve just completed an essay for a special edition of Architectural Theory Review in honour of my friend and former colleague Adrian Snodgrass, who passed away last year. Adrian introduced me to structuralism, semiotics and hermeneutics.

In part of that essay I reference the writing of the literary theorist and cultural semiologist Roland Barthes (1915-1980). I’ll present some of that material here in so far as it relates to the ethos of large language models.

In his seminal essay The death of the author, Barthes wrote, “the text is a tissue of citations, resulting from the thousand sources of culture” (53). He was here referring to a particular text by the novelist Honoré de Balzac (1799–1850), but attributed such profusion of sources to all textual production. The things that get written are derived from other writings to the extent that the author, or the concept of the author, diminishes in favour of a pan historical literary collective that spans across generations.

Authors are multiple and are co-complicit in the creation and interpretation of works of writing. Barthes’ account of language falls within studies that focuses strongly on texts rather than some independent grounding in empirical reality or external logic and mathematically-based rationality.

Such insights were extended by philosopher Jacques Derrida (1930-2004) who famously asserted that “there is nothing outside the text” (158). According to this statement any attempts to ground spoken or written assertions in some authoritative reality beyond what people say or write inevitably takes us on a journey through yet more texts, ad infinitum.

The acclamation by these humanities scholars of the power of texts thereby offers surprising support for techniques of automated natural language processing systems, specifically large language models (LLMs) as explained in other posts.

LLMs have not lived lives

I think I have said this before. As well as the consummate skill of countless human LLM systems innovators, developers and programmers, and the engineering, mathematical and linguistic innovations on which they build, LLMs emphasise the power of language: the vast repositories of literary source material on which they are trained and the armies of language users who “tune” the performance of these models.

LLMs have not lived lives or experienced worlds, but they have processed vast quantities of extant texts written by human beings about ideas and life experiences. The texts carry those insights, and/or it generates them.

The original source of a piece of writing is elusive, or at least from a hermeneutical standpoint contingent on your reasons for inquiring about the source. Authorship concerns not only the origin of an idea but attribution, authority, and the cultural context and practices by which society identifies and even constructs individuals and ascribes responsibility to them.

In fact, for the hermeneutical scholar, the infinite regress of multiply authored textual sources identified so dramatically by Barthes and Derrida can be translated more prosaically in terms of the author or reader’s background, tradition or horizon — all those influences, textual and otherwise that constitute who we are, our thought patterns and the bases of our judgements.

That said, for an AI-based large language model (LLM) there really is “nothing outside the text.”

Bibliography

• Roland Barthes, “The death of the author,” in The Rustle of Language, ed. and trans. Richard Howard (New York: Hill and Wang, 1984).
• Jacques Derrida, Of Grammatology, trans. Gayatri Chakravorty Spivak (Baltimore, Maryland: Johns Hopkins University Press, 1976).
• Gadamer, Hans-Georg. Truth and Method. Trans. Joel Weinsheimer, and Donald G. Marshall. London: Bloomsbury Revelations, 2013. Originally published in German in 1960.

Note

Roland Barthes challenged the idea that the author of a work is the final arbiter on what a text means. He also valorised the reader as the generator of meaning in relation to the text, thereby democratising or socialising the author-reader relationship. It’s a philosophy against the idea of an origin, a settled truth.

Barthes goes further. However we ascribe a work to a particular author, the work is always sourced multiplicatively. He asserted that “The model is not the supposed singular author but of readers and audiences, who come in multiples: there is one place where this multiplicity is collected, united, and this place is not the author, as we have hitherto said it was, but the reader” (54).

The reader (audience) doesn’t operate in isolation but is in the company of a whole community of interpreters. It seems that as much can be said of authorship and creativity: it’s a multiplicity. In media parlance, the author-audience relationship is “many to many.” According to Barthes, “We know now that a text consists not of a line of words, releasing a single ‘theological’ meaning (the ‘message’ of the Author-God), but of a multi-dimensional space in which are married and contested several writings, none of which is original: the text is a fabric of quotations, resulting from a thousand sources of culture” (52-53).

I like the fact that he described such meaning spaces as “multidimensional,” which reminds me of the multidimensional feature space of GPT models. He also explained: “Here we discern the total being of writing: a text consists of multiple writings, proceeding from several cultures and entering into dialogue, into parody, into contestation; but there is a site where this multiplicity is collected, and this site is not the author, as has hitherto been claimed, but the reader: the reader is the very space in which are inscribed, without any of them being lost, all the citations out of which a writing is made; the unity of a text is not in its origin but in its destination, but this destination can no longer be personal: the reader is a man without history, without biography, without psychology; he is only that someone who holds collected into one and the same field all of the traces from which writing is constituted” (54).