Too much feedback

Google has introduced an AI-driven document analysis platform called NotebookLM. You upload a document to the NotebookLM server via its website (notebooklm.google.com). The LLM generates a summary of the entire document and suggests some questions of the kind you see in textbooks that test a reader’s comprehension of the content. These are not generic questions, but fit the content of the document. The platform also provides answers to the questions, which the reader can supplement with their own interrogation.

I thought I would use an 85,000 word draft of my manuscript (AI and Language in the Urban Context: Conversational Artificial Intelligence in Cities) to test NotebookLM.

Here are some of my questions about the manuscript and NotebookLM’s responses. The answers are on point. I include a few comments of my own in italics.

I asked, “What is the book’s most controversial proposition?” NotebookLM responded:

The book’s most controversial proposition is its focus on semiotics as the primary framework for understanding AI’s impact on the city. This emphasis on the linguistic and communicative aspects of urban life challenges traditional, empirically driven approaches to urban planning and architecture.

• The book champions a “linguistic turn” in urban AI research, prioritising meaning, narrative, and interpretation as crucial factors in understanding AI’s role in the city.
• This perspective posits that cities are not merely collections of physical objects and systems, but rather spaces where meaning is constantly created and contested through language, symbols, and social interactions.
• The book argues that AI, rooted in language and communication, is inextricably intertwined with the city’s historical and ongoing evolution as a communicative entity.

This semiotic approach has drawn criticism for potentially:

• Oversimplifying urban complexity: Critics contend that reducing the city to a system of signs overlooks the intricate interplay of social, economic, and political forces.
• Overemphasising language-based communication: Critics point out the potential for neglecting other forms of communication, such as nonverbal cues and visual expression.

The book acknowledges these criticisms, suggesting that a balanced approach is necessary, combining semiotic insights with other perspectives to form a comprehensive understanding of AI’s role in the urban landscape. [I don’t think I propose a “balanced approach”. It strikes me that is the language of LLMs tuned to be even-handed and conciliatory.]

The book’s emphasis on semiotics contrasts with cybernetics, another prominent framework for understanding cities and AI. While the sources [actually, the manuscript] acknowledge cybernetics’ value in managing physical systems and infrastructure, they argue that it falls short in addressing the crucial aspects of meaning-making and interpretation that are central to human experience in the city. The book ultimately positions semiotics as a more suitable framework for grasping the profound implications of AI for urban life, as AI technologies increasingly engage with and influence the communicative fabric of the city.

I asked, “Does it provide a plausible account of how LLMs work?” Here’s the reply:

LLMs and Human Language: A Plausible Account?
The sources [a single source, the manuscript] offer a detailed account of how LLMs function, drawing parallels between their operations and aspects of urban life and human cognition. While not directly validating the plausibility of these accounts, the sources suggest that LLMs simulate human language competence through several key functions [these are correct, and as listed at several moments in the book]:

• Pattern Recognition:  LLMs are trained on vast amounts of text data (corpuses) to identify and manipulate patterns in language use. This mirrors how humans acquire language by absorbing and replicating patterns in speech and writing.
• Tokenization: LLMs break down text into smaller units called tokens, which can be words, parts of words, or punctuation marks. This process resembles the way humans break down language into meaningful components for comprehension.
• Semantic Embeddings: LLMs represent word meanings mathematically using vectors that capture relationships between words based on their proximity in the training data. This allows them to understand semantic similarities and generate coherent text.
• Context Windows:  LLMs operate within a limited context window, considering only a specific number of preceding tokens when generating responses. This simulates human short-term memory and the ability to focus on relevant information within a conversation.
• Positional Encoding: LLMs utilize positional encoding to account for the order of words in a sentence. This method uses mathematical functions to represent the position of tokens within a sequence, enabling LLMs to understand the importance of word order for meaning.
• Attention Mechanisms:  LLMs use attention mechanisms, particularly multi-head attention, to identify the most important parts of a text for generating responses.  This resembles how humans focus their attention on key words and phrases to understand a message.
• Fine-Tuning:  LLMs undergo fine-tuning to refine their responses, addressing biases and tailoring them to specific domains. This process involves adjusting the model’s parameters to improve its performance on specific tasks and ensure that its responses are aligned with human values and expectations.

Parallels with Urban Life

The sources [i.e. the manuscript] draw intriguing parallels between LLM functions and aspects of urban experience, suggesting that both are shaped by patterns, relationships, and contextual influences:

• Cities as Corpuses:  Just as LLMs are trained on vast text corpuses, cities can be seen as repositories of information and patterns. Urban spaces contain a wealth of data embedded in their physical structures, social interactions, and cultural expressions.
• Urban Tokens: The sources compare the use of tokens in LLMs to the representation of urban features on maps. Buildings, streets, and other elements can be seen as tokens that contribute to the overall meaning and organization of the city.
• Urban Attention: The sources highlight how attention mechanisms in LLMs parallel the ways humans navigate and experience cities. Just as LLMs attend to specific parts of a text, individuals focus their attention on certain aspects of the urban environment based on their needs and interests.
• Tuning the City:  The concept of fine-tuning in LLMs is linked to the process of urban design and planning. Both involve making adjustments to optimize performance and achieve desired outcomes.

Beyond Simple Simulation

The sources [the manuscript] emphasise that while LLMs effectively simulate human language competence, their internal processes differ significantly from human cognition. Their reliance on statistical pattern recognition and massive datasets contrasts with the more nuanced and context-sensitive learning processes of humans.  This highlights the need for a nuanced understanding of LLM capabilities, recognizing both their strengths and limitations. [I don’t think I emphasised the human above the machinic. I thought that this is too obvious a point to make. Perhaps the LLM response reflects its fine tuning to prefer an emphasis on the human.]

A Note of Caution

The sources also raise concerns about potential biases and ethical implications of LLM technology.  They stress the importance of human oversight in fine-tuning these models to mitigate harmful biases and ensure responsible use. They also caution against over-reliance on LLMs, recognizing that human expertise and critical thinking remain essential, especially in complex fields like urban planning and design. [That’s a minor point in the text and I see my commentary on “cautions” as more a report on what other commentators say rather than something I advance. Again, the LLM may have been “tuned” to give critical interpretations of AI.]

Conclusion

The sources offer a plausible, though arguably simplified, account of how LLMs work by outlining their key functions and drawing parallels with familiar urban concepts. They emphasize the importance of understanding the technical underpinnings of LLM technology while acknowledging the need for critical engagement with its potential impact on society.

[END OF REVIEW]

Reviewing the AI reviewer

I think that for an author, this kind of review and writing assistance exceeds that available from human sources, particularly during the writing of a book or thesis. The performance of NotebookLM is very impressive.

The downside for me of its integration into the writing process is that the AI might overwhelm the writer with too much feedback, analysis and commentary. Such relentless feedback can freeze the writing process, as I worry more about what the reviewer will say that plowing my own furrow. The AI feedback is also invariably encouraging and positive, which can instil overconfidence in one’s own writing.

The process also suggests that if the reader has access to the NotebookLM platform, then they have no need to read the source document. The platform might in turn influence the writing. As long as what you write has its own internal coherence then you can assume the reader will use an LLM tool to interpret and simplify it, and in a way that is tuned to the reader’s comprehension level.

What might be the long-term effects on writing, reading and language usage of all this automated writing and reviewing … and AI reviews of AI-assisted writing and reviewing — and feedback!

To illustrate feedback overload, here’s what the AI feedback feature in WordPress said about this blog post:

• The introduction effectively presents NotebookLM’s capabilities, but could benefit from clarity about its specific features.
• When discussing your manuscript, consider providing more context for how your specific questions to NotebookLM directly relate to its functionality.
• Ensure that the terminology used, especially concerning AI concepts, is consistent and accessible to a broad audience.
• The feedback on the LLM responses is insightful, yet could be more concise to maintain reader engagement and focus.
• Towards the conclusion, consider expanding on the implications of AI feedback on the writing process and reader engagement, as this could provoke further thought in your audience.
• Address the balance between AI feedback and personal voice, as it’s essential for writers to retain their unique style despite using such tools.
• It may also be valuable to include practical examples or anecdotes that illustrate your experience using NotebookLM, which can enhance the relatability of your insights.

Note

• Featured image is the library at Wells Cathedral, England.