We’re all familiar with the standard XY graph. It shows us a point on 2 dimensions.
AI does a similar thing except that it has millions, and more recently, trillions, of dimensions.
Those dimensions are defined by the words we write into the instructions, built upon the base of raw data to which the machine has access.
The output from AI is a function of the data that the particular AI tool has been ‘trained’ on and accesses to respond to the instructions given.
Every letter, word, and sentence, generated is a probability estimate given what has been said previously in the database of what the next word, sentence, paragraph, chapter, and so on, will be.
Generative pre-training of digital models goes back into the 1990’s. Usually it was just called ‘machine learning’, which plays down the ability of machines to identify patterns in data and generate further data-points that fit those patterns. The revolution came with the word ‘transformer’, the T in ChatGPT. This came from the seminal AI paper written inside Google in 2017 called ‘Attention is all you need’.
The simple way to think about a transformer, is to imagine a digital version of a neural network similar to the one that drives our brains. We make connections, based on the combination of what we see, hear, and read, with our own domain knowledge history and attitudes acting as guardrails. A machine simulates that by its access to all the data it has been ‘trained on’, and applies the instructions we give it to then assemble from the data the best answer to the question asked.
The very first paper on AI was written by Alan Turing in 1950 was entitled ‘Computing machinery and intelligence’. He speculated on the possibility of creating machines that think, introducing the concept of what is now known as the ‘Turing Test.’
The original idea that drove the development of the transformer model by Google was a desire to build a superior search capability. When that was achieved, suddenly the other capabilities became evident.
Google then started thinking about the ramifications of releasing the tool, and hesitated, while Microsoft who had been also investing heavily through OpenAI, which started as a non-profit, beat them to a release date, forcing Google to follow quickly, stumbling along the way.
Since the release of ChatGPT3 on November 20, 2022, AI has become an avalanche of tools rapidly expanding to change the way we think about work, education, and the future.
Header cartoon credit: Tom Gauld in New Scientist.
