Here is output from my first home grown generative language model. I trained it on my own writing (19,000 words), it extracted my vocabulary and it was able to produce nonsensical but better than random text, considering the simplifications and assumptions in the language model I used. “Unveils ancient daydream charles regulatory demands interactive aggregate…More
AI makes AI
One of the benefits of (and concerns about) large language models (LLMs) such as ChatGPT and its successors is that it can write computer programs, even AI programs. The concern is that it might start to do so autonomously without human intervention, but that’s for another post. GPT writes code I took advice from ChatGPT…More
Find yourself in AI
Guest post In the vast expanse of the digital universe, AI, particularly large language models (LLMs) like ChatGPT, have found a home, influencing our lives in countless ways. A lingering question, however, is whether we are inadvertently influencing these models back. In other words, could our writings have been part of the training data for…More
What’s next
Industry, education and everyday users are making increasing use of large language models (LLMs) driven in part by the prominence of ChatGPT and other AI tools. The technology is developing at a pace. The analysis of commentators, critics and legislators also gain traction as they evaluate the implications of the technology and seek to influence…More
Meaningful, imaginative, fluid
The recent escalation of interest in text-based AI coincides with our students completing their project work, dissertations, research degrees, etc. So it’s been a useful period to test the capabilities of platforms such as ChatGPT4 as a tutor — or perhaps as co-tutor. As yet there are no constraints in place that seriously impede its…More
Training your AI
The training data for a GPT model such as ChatGPT4 consists of many (hundreds of billions!) tokens harvested in sequence from various online sources, such as Wikipedia amongst many others. This source data is processed as a continuous stream independent of document boundaries. Tokenize The initial training task is to tokenize the entire corpus, identifying…More
More on automated recollection
Conversational AI platforms such as ChatGPT generate predictions for what should come next after a user inputs a question, statement, paragraph, or other text prompt. In early text prediction software, a simplistic language model might calculate the most likely word to follow a given input like “door,” based on pre-calculated statistical analysis of word co-occurrences…More
The human in AI
As critics contemplate the artificial in artificial intelligence, it’s worth considering the extent to which human agency plays a role in this technology, particularly in LLMs (large language models) and conversational AI. These technologies are invented, developed and improved by human beings, and they are fuelled by gigabytes of human generated texts. That much is…More
Why a neural network forgets
Conversational AI, such as ChatGPT, has limited capacity to recall the content of earlier conversations. OpenAI does not disclose all the details of its operations, but scholars estimate that ChatGPT4 can process and recall up to 10,000 words in a single session or thread. That’s a substantial improvement on earlier models, but it doesn’t ensure…More
AI textsurfing
Before digital text markup I would highlight key words in a difficult document with a coloured highlighter pen. Students with a more methodological inclination developed this into an art, with colour coding and supplementary markings and marginal comments. (I see that Staedtler imply that their pen users are textsurfers.) Now my markup practice is fairly…More