Post
In 1957, the linguist J. R. Firth observed that 'you shall know a word by the company it keeps'. The principle that words sharing contexts share meaning underlies generative AI, from early Latent Semantic Analysis to today's largest Transformers. This post traces that lineage with three interactive LSA-to-PCA visualisations in R, built from Reuters newswire, State of the Union addresses and IMDB reviews. They show where simple co-occurrence models succeed and where they fail, and the post explains how scale and the Transformer turned a modest insight into the technology behind ChatGPT. It then examines why LLMs, optimised for fluency, cannot be rid of hallucination entirely, and argues that careful prompt engineering is the best tool we have for steering a fundamentally heuristic machine.