In natural language processing LLMs, the “context window” is the range of text around a target word or phrase that constrains the sequence of text processed in calculating positional and attentional encoding, and hence prediction of the text that follows. A larger context window allows the AI model to capture more subtle relationships and dependencies…More
Tag Archives: LLM
Easy reading
I continue a discussion of my manuscript AI and Language in the Urban Context: Conversational AI in Cities (Routledge). See previous post: Too much feedback. I asked Google’s NotebookLM, “Would the average reader have any difficulty following the technical/mathematical explanations?” The AI responded: Readability of Technical/Mathematical Explanations The sources [actually, the manuscript] suggest that the…More
AI as critic
I listened recently to a radio program in which a collection of prominent entrepreneurs discussed some of the key examples this year of overhyped marketing. As an aside, one of the participants mentioned the disturbing error rate in ChatGPT’s responses to questions of fact, e.g. name ten distinguished alumni of the Fitzwilliam College, Cambridge? It’s…More
Derrida on AI
The radical philosopher Jacques Derrida challenged the conventional belief that speech precedes writing, arguing that writing is fundamental to language’s structure and meaning. Current large language models, reliant on text, support Derrida’s theory, emphasizing the enduring and analyzable nature of written language compared to the ephemeral and context-dependent qualities of speech. (Word count: 50)More
Conversation-centric computation
In the previous post I alluded to some of the challenges of encouraging large language models (LLMs, e.g. AI chatbots such as chatGPT) to communicate with one another in ways that are oriented to some task, and that don’t settle on merely exchanging platitudes. In spite of their instant access to parameters trained on vast…More
The human in AI
As critics contemplate the artificial in artificial intelligence, it’s worth considering the extent to which human agency plays a role in this technology, particularly in LLMs (large language models) and conversational AI. These technologies are invented, developed and improved by human beings, and they are fuelled by gigabytes of human generated texts. That much is…More