The ease with which I can be seen, heard and otherwise monitored has increased since January 2014. That was the arbitrary date of my post #180 titled Showing Off. Thanks to digital networks and displays it’s easy to put yourself “out there” as part of a strategy of personal presentation (e.g. as an online influencer),…More
Tag Archives: attention
Beyond attention
I’m still reviewing my blog posts from 2013. In keeping with the unstructured nature of blogging, I didn’t plan sequences of posts to follow particular themes. However, a thread does emerge from this sequence of six posts — that of attention. Soft fascination (138) introduces the theory that sustained concentration on a task induces “attention…More
Meaning and attention
Understanding and misunderstandings in conversation often stem from emphasis. People tailor their responses based on where their conversational partners place emphasis. Attention distributions play a crucial role in inflecting responses in dialogue. Emphasis influences what comes next in a conversation, shaping the interaction between speaker and listener. Text-only conversational exchanges rely on context without additional cues. Attention, as demonstrated by LLMs, significantly affects the platform’s text generation capabilities and conversation continuation.More
An AI focus group
Before widespread digitalisation, I recall borrowing books from the library that were graffitied with penciled underscores and multi-coloured hi-lighter markings indicating what various past readers thought important. In terms of my previous posts about attention in NLP models, such markups constitute a compelling record of human-based “multi-headed attention.” I described multi-head attention in my previous…More
Multi-head attention
My fascination with electronic sound production began when my father brought home a Grundig TS 340 reel-to-reel stereo tape recorder. Like many other recorders it had separate heads for recording and playback, which meant that you could play a recording, add new sounds and feed the combination back through the recording head in real time.…More
Attending to the city
In the book Network Nature, I explored how people attune themselves to the natural world — or at least, how we attune to that part of our spatial experience that we are inclined to describe as “natural.” We also attune to artifice, such as urban environments. Cities present a spectrum of stimuli that shape our…More
Attention scores
One of the important techniques in the new wave of highly successful generative natural language (NLP) models is the use of attention scores in neural network (NN) training. Here I continue the investigation I started in previous posts into how NLP works. To recap: a word in NLP models is represented as a point in…More
Who’s listening?
In every mind there is a higher function watching or listening to the inflow of sense data and monitoring its own thought processes. In his seminal book on the philosophical challenge of consciousness, Daniel Dennett argues convincingly against this proposition. He directs his criticism against those who presuppose that somewhere, conveniently hidden in the obscure…More
Attention is everything
Attention is a key element in cognition. At our most thoughtful we direct attention to features in our environment that are most important to us at that moment. Attention can wander, of course, we daydream, and we can pay attention to inexistent things, memories and objects of the imagination. A lecturer will come to the…More
Inattention and power
Stanley Kubrick’s film 2001: A Space Odyssey (1968) provides one of the most enduring depictions of machine intelligence, a spaceship that exhibits sentience. HAL, the onboard computer provides an interface to the ship’s functions. In his chapter “Toward the sentient city,” Mark Shepard identifies the conversational aspect of HAL’s interface, “symbolized by his iconic and…More