In the book Network Nature, I explored how people attune themselves to the natural world — or at least, how we attune to that part of our spatial experience that we are inclined to describe as “natural.” We also attune to artifice, such as urban environments. Cities present a spectrum of stimuli that shape our…More
Hiding one AI generated picture inside another
Steganography is the art of concealing one image inside another. I discussed the basics of the technique in a post: Hiding one surface inside another. It involves bit-shifting. In an 8-bit image each pixel is represented by an integer in RGB (red, green and blue). With 256 shades of RGB that provides over 16 million…More
Evaluating your AI
I’ve been implementing small scale trials of automated natural language processing routines that deploy the same methods as ChatGPT, i.e., implementations of the so-called Transformer architecture. That requires training on sequences of words in an original source document, equating each word to a semantic encoding (i.e., a long vector of 30 floating point numbers sourced…More
NLP sequences and cycles
In spite of its esoteric mathematical intricacies, automated natural language processing (NLP) as in conversational AI, draws on at least one primal everyday phenomenon. I’m referring to the concept of periodicity, i.e., cycles, periods, rhythms, repetitions, etc. Periodicity is a major principle through which we understand time, temporality, ordering, and sequencing and permeates so much…More
Robot probes city grid
Sentences are of course sequences of tokens (or words), and one of the major tasks of large language models (LLMs) is to predict the plausible continuation of sentences from some start condition, such as a prompt in a chatbot dialogue box, taking account of the previous flow of tokens within a given context window. Several…More
AI learns ABC
Neural networks (NNs) as deployed in automated language processing (NLP) are good at identifying and reconstructing patterns in data. So a NN trained on a corpus of texts can identify words that are commonly grouped according to their proximity within sentences, e.g., we wouldn’t be surprised to find words (tokens) such as “building,” “services,” “construction”…More
Chatting with multi-modal AI
ChatGPT now supports image upload, which means that I can chat about pictures, a step change in its application to architecture and design. Here’s my first such conversation, with chatGPT responses shown as quotes. The platform has information about my interests in urbanism and AI via the Custom Instructions settings. So responses reference my interests.…More
Non-NLP models
As an NLP neophyte I thought the mechanisms of automated natural language processing, such as ChatGPT, esoteric and arbitrary. At best, the general pre-training transformer (GPT) model is structured as a series of mathematical functions that apparently work in simulating human language, but bear little relation to the processes by which language is actually spoken,…More
By the same token
Automated natural language processing (NLP) of the kind deployed in conversational AI typically breaks blocks of text not just into words, but into tokens. Tokens are strings of characters extracted from a corpus of texts — the set of texts an NLP system is trained on. Tokens are more efficient than storing just whole words.…More
The neural neighbourhood
In a previous post (AI makes AI) I described an an auto-associative neural network with an example using made-up data. Here’s a more sophisticated version with actual urban data. Consider this fragment of a city map. I overlaid a 100 metre grid carving the neighbourhood into 72 cells. Within each of these cells lies a…More