Share your expertise

https://richardcoyne.com/wp-content/uploads/2024/11/SharYourExpertise.m4a

Google’s NotebookLM enables a user to upload and then interrogate several files at once following the RAG (Retrieval-Augmented Generation) methodology incorporating local document search. See previous post: Smashing the context window. NotebookLM is designed to answer questions, i.e. respond to prompts, with reference to the corpus of texts you have uploaded.

As a further indication of the consequential impact of LLMs in education, the University of Edinburgh has now adopted its own large language model (LLM) built on the OpenAI ChatGPT platform for use by students and staff.

This ELM (Edinburgh Language Model) currently responds to prompts in general, without referring to specific uploaded texts, though it can incorporate texts individually and one at a time with each prompt.

With both applications open at once I was able easily to switch between NotebookLM and ELM, garnering the benefits of both. I discussed with ELM the implications of the NotebookLM strategy of interrogating large numbers of files, seamlessly, as it were.

An AI expert consultant

Up until the adoption of the RAG and similar context-expanding methods, if you want to create an LLM for specific applications, such as consultation about building codes, design guidance or course notes, then the LLM has to be trained and tuned for that application. That can involve training on specialised texts and styles of interaction. See post: Fine tune your AI. Training and fine tuning are expensive, time consuming and difficult to update. It involves entering into a contract with OpenAI and payment (https://openai.com/api/pricing/).

On the other hand, any user of NotebookLM need only upload relevant documents to create an LLM “expert” in that area. ELM helped me identify several use cases in architecture and construction: building regulations and codes (centralised access for professionals to ensure compliance and streamline project planning), design guides and manuals (reference for best practices and standards in design and construction), course notes and textbooks (educational aid for students and trainees in architecture and engineering). ELM proffered other examples from law and health care.

But uploading my own written outputs was the easiest way for me to test how effective such an upload would be in capturing the “expertise” embodied in a collection of documents. To this end I uploaded drafts of most of the books (11) and a few articles I have published, as well as a PDF of the entirety of this blog site. That came to 4,823 pages or 1,886,235 words (60.5Mb).

What can I glean from an LLM so primed with my own outputs in this way? What can I learn from interrogating my own work, or what could another reader learn were I to share it?

Sharing expertise

A reader new to the collection would have to know what it is about before prompting and probing the content. Notebook LLM provided a helpful summary of my collection, explaining the overall thrust of the work, while highlighting distinctive features. Here’s the summary it provided.

The sources provided are a collection of excerpts from books, articles, and blog posts written by Richard Coyne. The collection showcases Coyne’s work on the intersection of architecture, design, and digital technology. A recurring theme across these works is the exploration of semiotics, the study of signs and symbols, and its application to the built environment. Coyne examines how digital technology shapes our understanding and experience of space, particularly in the context of urban environments. This exploration extends to concepts like place, non-place, and the digital uncanny, drawing upon the works of philosophers like Heidegger, Derrida, and Freud.

Coyne’s work also considers the impact of digital technologies on creativity, design practices, and social interactions. He discusses themes such as digital utopias, cybernetic rapture, and the role of the trickster in the network economy, frequently drawing analogies between online and offline experiences. The sources demonstrate an engagement with diverse theoretical frameworks, including hermeneutics, postmodernism, and psychoanalysis, to critically analyze the cultural and societal implications of digital technology. Notably, the collection reflects an ongoing investigation into the evolving relationship between nature and digital technology, examining how our perception of the natural world is mediated and transformed through digital platforms and experiences. The sources also feature an interest in cryptography and its role in shaping the smart metropolis, referencing specific cryptographic concepts and their application in contemporary urban design.

Probing my content

In conversation with NotebookLM I was able to elicit commentary on overarching themes, including opinions about what might be missing from the research. It suggested I could have written more from a political angle, discussed the body in architecture, and the ethics of architectural communication. It answered direct questions, such as “What does the author say about metaphor?” It also answered queries requiring inference, such as “Is the author a relativist?” (I’m not.)

On request, NotebookLM was also able to identify minor themes, such as content about animals, interplanetary aliens, Trump, and cartoons. It could also proffer advice about pursuits external to the content, such as house renovation and gardening, providing responses and advice consistent with the content of the texts, and with explanations of the connections. It also provided citations to the relevant parts of the texts.

For example, I asked about rainwater harvesting. It responded: “While the sources do not explicitly discuss rainwater harvesting, several themes and concepts from Cryptographic City and Reflections on Technology, Media & Culture relate to your interest in this sustainable practice.” It related the theme to matters that I had discussed in the source texts, such as the sonic properties of water, emotional landscapes, underground infrastructures, sustainability and fine tuning one’s environment.

Prompts

The user can save the responses to prompts as notes in a digital notebook. NotebookLM will then provide a summary, as a well as a study guide, which is a series of questions based around the questions I asked and the responses it generated.

I asked NotebookLM to provide a series of prompts for probing the uploaded sources and its implications. I stated that these prompts do not have to be questions. Nor do they have to relate specifically to the content of the texts

It responded with some potential prompts that probe the content of the sources and their implications. Here they are:

Compare and contrast the concepts of ‘intertextuality’ in Derrida and ‘hypertext’ in Landow. Consider how both concepts challenge traditional notions of authorship, linearity and the boundaries of a text. How do these concepts illuminate or problematise our understanding of digital media and online platforms?

Explore the ethical implications of using AI for ‘disrupting the semiotic coding of the city’, as suggested in the ChatGPT response. Who has the right to redefine the meaning of urban spaces? How can AI be used responsibly in urban design and planning?

Reflect on the nature of conversation and inquiry in the context of human-AI interaction. How does the use of AI tools like ChatGPT shape our understanding of knowledge, learning and the nature of ‘convincing conversation’? How do these insights relate to the ideas of ‘question and answer’ and ‘dialogue’ discussed in relation to hermeneutics?

Consider the concept of ‘mood’ as a lens for interpreting digital media. How do specific digital platforms or technologies cultivate particular moods in their users? How can we design digital experiences that foster positive and engaging moods? How do these ideas relate to the concept of attention as a key element in developing LLMs?

‘The world could be its own tag’. Discuss the implications of ubiquitous tagging and ambient data collection for urban environments. How do these practices affect our experience of public space, privacy and the relationship between the physical and digital realms? How do they relate to the affordances of the city and the semiotic insight that urban elements communicate possibilities for action?

Reflect on the role of narrative and storytelling in shaping our understanding of technology. How do different narrative genres (e.g., science fiction, utopian/dystopian narratives) influence our expectations and anxieties about AI and digital media? How do these narratives relate to the concepts of ‘progress’ and ‘technoromanticism’?

‘Everything must be treated as potential evidence for something’. Discuss the role of curiosity, inquiry and interpretation in understanding complex systems, such as urban environments or digital networks. How do AI tools help or hinder our ability to ask meaningful questions and engage in critical reflection?

These prompts are useful for the human user (student, client, researcher) to contemplate, but can also be fed back in part or as whole paragraphs to generate a response from NotebookLM. One set of prompts caught my eye.

Imagine you are designing a game based on the concept of the ‘multiverse’. How would you represent the experience of transitioning between different worlds? What kind of challenges and opportunities would this concept present for game mechanics and storytelling?

I prompted NotebookLM with this paragraph. The AI volunteered the following, along with citations to the uploaded texts:

Transitioning Between Worlds in a Multiverse Game

The sources offer a rich array of concepts and examples that can inform the design of a multiverse game, particularly in terms of representing the experience of transitioning between worlds and the challenges and opportunities this presents for game mechanics and storytelling.

Representing Transitions

Portals and Thresholds: The sources discuss the concept of “portals” as a means of transitioning between different worlds. These portals can be physical objects, like spherical mechanisms, or symbolic thresholds, like magical doors or a snap of the fingers. In a game, these portals could be represented visually, requiring players to solve puzzles or complete tasks to activate them. The act of passing through a portal could be accompanied by visual and auditory effects, signifying the shift between worlds.

Geometry and Visual Effects: The sources emphasise the role of geometry and visual effects in representing spatial experiences, particularly in the context of video games and virtual reality. These techniques can be used to create a sense of disorientation and wonder as players transition between worlds, using techniques like distorted perspectives, shifting camera angles, and surreal imagery. For example, transitioning between worlds could involve a visual sequence where the familiar geometry of one world dissolves and reassembles into the unfamiliar structures of another.

Narrative and Psychological Shifts: The sources highlight the role of narrative and psychological states in framing our understanding of different worlds. Transitioning between worlds in a game could be accompanied by shifts in narrative perspective, tone, or even the player’s emotional state. For instance, moving from a bright, cheerful world to a dark, oppressive one could be reflected in the game’s music, dialogue, and visual design.

Challenges and Opportunities

Maintaining Coherence and Consistency: One challenge in designing a multiverse game is maintaining coherence and consistency across different worlds. Each world should have its own distinct rules, aesthetics, and narrative logic, but these elements should also connect in a way that feels meaningful and believable to the player. The sources suggest that the concept of “levels” in games can be helpful in this regard. Each world could be presented as a level with its own set of challenges and rewards, while also contributing to the overall narrative arc.

Exploring the Nature of Reality: A multiverse game offers unique opportunities to explore philosophical questions about the nature of reality and perception. The sources discuss how digital media can challenge our understanding of reality. The game could present players with situations where the boundaries between different realities blur, forcing them to question their assumptions and interpretations. For example, players could encounter characters or events that exist in multiple worlds simultaneously, raising questions about cause and effect, identity, and the nature of choice.

Enhancing Player Agency and Engagement: The concept of the multiverse allows for a high degree of player agency and choice. Players could be given the ability to choose which worlds to visit, how to interact with different realities, and even influence the course of events across multiple universes. The sources discuss how games rely on player engagement and interactivity. A multiverse game could offer multiple storylines, branching narratives, and alternate endings, allowing players to shape their own experience and explore the consequences of their actions.

The sources also touch upon the concept of “meta-games”, where players manipulate the rules or systems of a game to achieve their goals. This concept could be incorporated into a multiverse game, allowing players to discover hidden connections between worlds, exploit glitches or inconsistencies in different realities, or even manipulate the fabric of the multiverse itself.

From this outline it’s apparent that LLMs will translate theoretical critique into design outcomes, or at least leverage the transition to design. A further benefit for me of this technology is that it is able to make connections that I had not detected, even within my own writing. Potentially, this makes it a very valuable tool for writers and researchers. See post: Too much feedback.

Bibliography

• Mistelbacher, R. (2024), ‘A Guide to Using Notebooklm by Google (How to + Examples)’. Fresh Van Roor: AI for Creatores, 2 October. Available online: https://freshvanroot.com/blog/notebooklm-google/ (accessed 16 November 2024).

Note

• Image is from ChatGPT: “Here is a steampunk-inspired visualisation of transitioning between worlds. If you’d like any further refinements, feel free to share your thoughts!”