Tag Archives: transformer

Extending large language models

I’m in the process of identifying parallels between urban semiotics and large language models (LLMs), arguing that core facets of language competence parallel aspects of urban life, experiences and processes. We can identify eight core functions that contribute to the success of the Transformer model of LLMs, as deployed in ChatGPT and other chatbot platforms.…More

NLP sequences and cycles

In spite of its esoteric mathematical intricacies, automated natural language processing (NLP) as in conversational AI, draws on at least one primal everyday phenomenon. I’m referring to the concept of periodicity, i.e., cycles, periods, rhythms, repetitions, etc. Periodicity is a major principle through which we understand time, temporality, ordering, and sequencing and permeates so much…More

Words in order

Neural network researchers invented several methods that store and make inferences about the order of words in a sentence. The main method I will present here provides one of the components that undergirds the recent impressive performance of natural language processing (NLP) models known as transformer models.  The method also resonates with my prior investigations…More