OpenAI to Release Commercial Version of Famous Text Generation Model

The artificial intelligence research lab OpenAI will be making its famous text generation tool GPT-3 available for purchase, marking the first commercial product produced by the machine learning non-profit.

In February of last year, OpenAI made their text generation model GPT-2  open source. GPT-2  was able to generate impressively coherent text when provided with a short prompt, and the model was regarded as a significant step forward in the fields of natural language processing and artificial intelligence. In fact, OpenAI initially refused to release the GPT-2model, claiming that it was too dangerous for an open-source release, fearing misuse. However, after a version of the model became available for some time, and the nonprofit reported that they hadn't seen any evidence of malicious usage, a decision was made to make the model open source.  Just recently, OpenAI announced the follow-up to GPT-2, dubbed GPT-3.

The GPT-3  model is approximately 100 times larger than its predecessor, and it’s this model that they are offering as a commercial product. The GPT-2  model was comprised of approximately 1.5 billion parameters while GPT-3  is comprised of approximately 175 billion parameters. The sophistication and reliability of the GPT model series have caused the models to converge on becoming the standard for machine learning projects involving text. Much like how convolutional neural networks have become the default model for image-related projects, text related projects are increasingly utilizing GPT-2,  and perhaps soon GPT-3.

As reported by The Verge, OpenAI seems as if they are hoping to drive the adoption of a GPT standard forward by making their GPT-3 model commercially available. Currently, access to the GPT-3 API is invite-only. However, this is likely just a test run and it seems probable that GPT-3  will be made more widely available sooner rather than later. OpenAI has stated that it will be vetting customers of their models to prevent the abuse of technology for unethical uses like spamming, creating misinformation, or harassment.

It’s somewhat unclear exactly how GPT-3 is intended to be used by customers. This is arguably because GPT-3  is so flexible and has so many potential applications. GPT based models can be fine-tuned as necessary to generate text appropriate for specific tasks. The model is able to adapt to a wide variety of various prompts and input styles, meaning that it could be used it to create text summarization tools, answer simple questions, or act as the basis for a sophisticated chatbot. As OpenAI’s chief technology officer Greg Brockman explained to Wired:

“The big mental shift is, it’s much more like talking to a human than formatting things for a machine,” says Greg Brockman, OpenAI’s chief technology officer. “You give it a few questions and answers and suddenly it’s in Q&A mode.”

It’s unknown just how reliable the GPT-3 derived commercial models will be. While GPT-3 can generate syntactically correct, natural-sounding language, it lacks an intuitive understanding of the real world and how concepts relate to one another. This could prove to be problematic for uses where preserving context is especially important, such as chatbots designed for customer support.

Joe Davison, a research engineer at Hugging Face, expressed to Digital Trends that the sheer size of GPT-3 could limit its potential uses. Davison argued that while OpenAI has meaningfully advanced the state of the art when it comes to text generation, demonstrating that creating general-purpose models can reduce the need for task-specific data, the computation resources required to make use of GPT-3 make it fairly impractical for many companies.

OpenAI was previously a non-profit enterprise, but since 2019 the lab has shifted to a for-profit model. GPT-3 is the first major commercial product produced by the lab, and it remains to be seen if the switch to a for-profit model will shift OpenAi’s research priorities.