Tokens NLP (natural language processing) systems don’t necessarily consider only words as the basic components of prediction. They may break words into smaller units roughly corresponding to syllables, punctuation marks and even single characters. The general term for these textual units is “tokens.” An NLP system will include in its algorithmic workflow the construction of…More
Nothing beyond text
In explaining the political philosopher Jean-Jacques Rousseau’s (1712-1778) autobiographical book Confessions, the continental philosopher Jacques Derrida observed that there is nothing to support Rousseau’s claims about his relationships with his family members, other than what the text discloses. Derrida generalises this observation to the controversial proposition that any attempts to ground spoken or written assertions…More
Attention is everything
Attention is a key element in cognition. At our most thoughtful we direct attention to features in our environment that are most important to us at that moment. Attention can wander, of course, we daydream, and we can pay attention to inexistent things, memories and objects of the imagination. A lecturer will come to the…More
Learning to transduce
Human beings at their most rational are able to generalise from examples. If you stand under the shower head before turning on the tap then it is likely you will be dowsed with cold water before it gets to a temperature you are happy with. That if-then rule is a generalisation borne of a few…More
My tutor is an AI
In constructing this post I’ve been cross checking some of my explanations about neural networks with OpenAI’s chatGPT-3, which is a highly responsive chatbot available at the OpenAI.com website (free at present). I’m most familiar with spreadsheets. So that’s where I’ll start. You can think of a neural network as a matrix or grid (a…More
Intentional systems
In his book Consciousness Explained, the philosopher and cognitive scientist Daniel Dennett says that a study of hallucination, “will lead us to the beginnings of a theory — an empirical, scientifically respectable theory — of human consciousness” [4]. I’ve explored what some philosophers say about hallucination in previous posts and tried to relate that to…More
Chatting with an AI about urban inexistence
“Intentional inexistence” is a philosophical term adopted by the nineteenth century philosopher Franz Brentano (1838-1917) to indicate the commonplace human capacity to imagine things. We may suppose that inexistence refers to things that don’t exist (nonexistent things) or that are under demolition, but that presumes too much — or too little. As explained by Linda…More
Cryptographic City
The description of my next book is out with the cover design on the MIT Press website. The graphic designer says that the cover is in code, though I have yet to decipher it. Cryptography’s essential role in the functioning of the city, viewed against the backdrop of modern digital life. Cryptography is not new…More
Cryptographic index
To introduce cryptography as a theme in my Media and Culture class I provided each student with a customised string of code containing the name of an AI-themed film. The key to decode the secret message was the student’s unique 7 digit university identification number. If the encrypted string is uaroulojz and the student key…More
Of what do AIs dream?
Midjourney is an AI platform that deploys machine learning and processing techniques similar to Dall-e. The backend to the Midjourney platform drew (and continues to draw) on a vast repository of images and their associated texts. That’s as much as I dare surmise about its operations. I have yet to read the definitive academic articles…More