Industry, education and everyday users are making increasing use of large language models (LLMs) driven in part by the prominence of ChatGPT and other AI tools. The technology is developing at a pace. The analysis of commentators, critics and legislators also gain traction as they evaluate the implications of the technology and seek to influence…More
Meaningful, imaginative, fluid
The recent escalation of interest in text-based AI coincides with our students completing their project work, dissertations, research degrees, etc. So it’s been a useful period to test the capabilities of platforms such as ChatGPT4 as a tutor — or perhaps as co-tutor. As yet there are no constraints in place that seriously impede its…More
Training your AI
The training data for a GPT model such as ChatGPT4 consists of many (hundreds of billions!) tokens harvested in sequence from various online sources, such as Wikipedia amongst many others. This source data is processed as a continuous stream independent of document boundaries. Tokenize The initial training task is to tokenize the entire corpus, identifying…More
More on automated recollection
Conversational AI platforms such as ChatGPT generate predictions for what should come next after a user inputs a question, statement, paragraph, or other text prompt. In early text prediction software, a simplistic language model might calculate the most likely word to follow a given input like “door,” based on pre-calculated statistical analysis of word co-occurrences…More
The human in AI
As critics contemplate the artificial in artificial intelligence, it’s worth considering the extent to which human agency plays a role in this technology, particularly in LLMs (large language models) and conversational AI. These technologies are invented, developed and improved by human beings, and they are fuelled by gigabytes of human generated texts. That much is…More
Why a neural network forgets
Conversational AI, such as ChatGPT, has limited capacity to recall the content of earlier conversations. OpenAI does not disclose all the details of its operations, but scholars estimate that ChatGPT4 can process and recall up to 10,000 words in a single session or thread. That’s a substantial improvement on earlier models, but it doesn’t ensure…More
AI textsurfing
Before digital text markup I would highlight key words in a difficult document with a coloured highlighter pen. Students with a more methodological inclination developed this into an art, with colour coding and supplementary markings and marginal comments. (I see that Staedtler imply that their pen users are textsurfers.) Now my markup practice is fairly…More
Can I use AI in academic writing?
I’ve been coaching masters dissertation students as they complete their final projects. I’m interested in large language models (LLMs) and their applications. At the moment, it’s easy to include the ChatGPT platform as a participant in one-on-one discussions with students. The subject matter of their projects is digital media, so familiarity with the applications, strengths,…More
Attending to more than one thing at a time
In an earlier post (Attention Scores) I considered how automated natural language processing (NLP) models attempt to simulate the way a listener or reader will focus on key words and groups of words in a sentence to decide how to respond to the sentence. I won’t repeat the calculation here. But recall that the automated…More
Confidential documents and conversational AI
Confidentiality is key in any profession, especially as it related to client-consultant relationships. I’m hard pressed to find confidentiality foregrounded in architectural codes of practice, but it is crucial in law and financial services. The Handbook of the Financial Conduct Authority, for example, states that a financial advisor (a “skilled person”) “may not pass on…More