Rockstar AI chatbot that’s good for innovation, for work, but is it good at all?
By Michael Grant, chief technology officer at Synthesis
The public release of ChatGPT by OpenAI has taken the world by storm. There are two notable components to this: the first has been the staggering rate of adoption and the second is just how unreasonably good it is.
The Synthesis Labs team has been following this phenomenon, which has already provided some interesting take-aways.
Adoption
When we think about new innovation it is convenient to use the “diffusion of innovation curve” when trying to describe a target market. There is a small fraction of risk-amenable consumers who want to try something new for the sake of it being new (2.5 percent, called the innovators), a slightly larger fraction (13.5 percent, called the early adopters) who want to adopt new technologies because they can see the value, followed by the early majority (34 percent) who follow shortly after the early adopters as value is proven. Gaining market access to the early majority is often referred to as “crossing the chasm” and a good measure of getting there (in the consumer space) is reaching one million users.
How long does this take? Well, it took Netflix 3.5 years; Twitter took two years; Facebook took 10 months; Spotify took five months.
ChatGPT took five days
Although the underlying AI technology is revolutionary, we think this unusually high adoption is due to the correct level of sophistication of the interface. The majority of technology users are not sophisticated, where new technical product features are complex to access and tricky to meld with your lifestyle. The ChatGPT conversational interface is so simple and natural – we have hundreds of discussions a day and the style of interaction and discussion is so natural that it takes very little effort to adopt it.
The unreasonable goodness
There have been many detailed investigations to find and understand the limits of ChatGPT, and of course many have been found. The challenge about trying to protect against a set of edge cases is that the edge-case-surface is infinite and so it is impossible to protect against all of them. Of course one can put guard rails in play, but all of this adds in friction and increases the level of sophistication required to engage meaningfully with the technology.
I suspect much of the improvement is that while building on the architecture transformer-type models that made GPT-2 and GPT-3 so powerful, ChatGPT was trained differently: using a reinforcement policy exploration based on human feedback and reward – a departure from the previous error minimisation/gradient methods used to train the previous GPT instances.
So, what’s next?
While the current ChatGPT model has some well documented flaws, the team at OpenAI was working on successive versions of their GPT family of models before the November release was even made public. Termed “GPT-4”, this model has even more dimensions, and so we are expecting a massive increase in the size of the language embedding and bandwidth of context transfer – although there is much speculation around the specific dimensions and architecture of this new version. We expect the set of already publicised improvements to reduce many of the type-2 errors currently present in ChatGPT.
Type 2 errors are really interesting problems to solve: they’re well embodied by “unknowingness” and so moving more information forward, question to question, will result in noticeable improvements.
What is clear is that this is a phenomenon worth tracking. It has upended the tech space and will continue to do so, one way or another.
