Why ChatGPT Enterprise Is Not Yet Ready for Business Use

Gartner Analyst Jim Hare Takes a Closer Look at ChatGPT Enterprise Brian Pereira (creed_digital) • September 15, 2023

The adoption of generative AI tools by both businesses and consumers has been on the rise, presenting new potential use cases. Gartner predicts that generative AI will make up 10% of all data produced by 2025, a substantial increase from its current share of less than 1%.

However, the business world continues to have apprehensions about the accuracy of outcomes produced by generative AI models. Concerns regarding data privacy, AI hallucinations and bias persist.

OpenAI's recent launch of the enterprise version of ChatGPT aims to address some of these challenges, with its enterprise-grade security and privacy features that offers improved data privacy and security.

In an email interview with ISMG, Jim Hare, distinguished VP analyst, Analytics & AI at Gartner, discusses the challenges and concerns around generative AI and to what extent these are addressed in the enterprise version of ChatGPT.

Edited Excerpts Follow:

What are the biggest challenges CIOs face with generative AI technologies such as ChatGPT?

The biggest challenges that CIOs face are data privacy, hallucinations/accuracy, and security. Other challenges include misuse and lack of user skills to use the tools correctly to get the best responses to prompts.

To what extent does the enterprise version of ChatGPT address all these challenges?

ChatGPT Enterprise appears to address just two of the key challenges: data privacy and security. While the Enterprise version offers a bigger window to process 4x longer files prompts and faster speed, this does not address the accuracy and hallucination issues since it is still based on the underlying GPT-4 model.

What about privacy of enterprise data? Will OpenAI use this to train its models (without consent)? What kind of control do enterprises have?

OpenAI claims they won’t train their models on an enterprise’s data conversation (inputs/outputs). However, there are no real controls in place other than OpenAI’s promise on their website.

What needs to be done to check data bias in generative AI models?

Eliminating bias in generative AI is almost impossible but most of the very large models have been trained on vast amounts of data from various sources (websites, books, social media, conversational data) that reflect a biased world. In fact, biased generative AI models can still be useful in certain contexts or applications, as long as users are aware of their limitations and take them into account when making decisions. The key to using generative AI tools responsibly is to ensure that users have a clear understanding of the potential biases and limitations, so they can make informed decisions about whether, and how to, use them in different contexts.

Some ways to address the issue of bias include:

Transparency: Model providers should be transparent about the methodologies, data sources and potential biases of AI models, providing users with the necessary information to understand the factors that may influence the model’s results.
Awareness and education: Provide resources, training and support to help users better understand the potential biases in AI models and how to account for them when making decisions. This may involve creating guidelines, best practices, or other educational materials that explain the implications of bias in AI and how to navigate it responsibly.
Context-specific use cases: In some limited cases, biased generative AI models may be viable where the known biases align with specific case. Experts should be employed to carefully evaluate the appropriateness of using biased models in these situations, taking into account the potential risks and benefits associated with their use, and active governance to recognize, quantify and mitigate biases. It is important to communicate the biases and assumptions to users of these applications.
Domain-specific models: Domain generative AI models are ones that have been optimized for the needs of specific industries, business functions or tasks. They are aimed at improving performance, reducing bias and the need for advanced prompt engineering, compared with general-purpose models, for a narrower set of use cases. Through more targeted training, these models have the potential to lower bias and hallucination risks associated with large models.
Continuous monitoring and evaluation: Regularly assessing the performance of generative AI models in real-world contexts, monitoring their impact on users and affected communities, and making adjustments as needed to mitigate any biases or unintended consequences that emerge over time.

What impact will data privacy laws and regulation have on generative AI adoption in the enterprise?

Generative AI introduces a variety of data privacy concerns due to its ability to collect and process personal data, and generate sensitive information. Personal information, such as names, addresses and other details, can be inadvertently collected during interactions with generative AI tools. The processing of personal data by generative AI models may result in unintended exposure or misuse of this information.

Enterprises that use generative AI will be on the hook by regulations such as GDPR to ensure that customers are aware and can opt-in/out of having their data used by the generative AI tools. This includes having the ability for customers to have their personal data deleted in training, validation and fine-tuning data sets. This opens a whole new level of complexity and compliance challenges, necessitating that privacy must be built into the use of generative AI by enterprises, called privacy by design, rather being an afterthought. A proactive risk-based approach helps enhance consumer trust, prevent violations (such as costly data breaches) before they occur, and reduce the damage from them if they do (such as fines or brand damage).

What kind of controls and checks do we need to have in place to ensure AI ethics? For instance, protecting against copyright infringement and stolen data? Are companies like OpenAI looking into this?

Right now, there is limited protection in place to avoid copyright infringement and stolen data (deliberate or unknown). In fact, training generative AI models using copyrighted materials is protected under certain conditions by the fair use doctrine of the U.S. copyright statute.

OpenAI is facing a number of lawsuits from Getty Images, the New York Times, and various plaintiffs over using copyrighted materials in the training data sets for its models. What is needed is better transparency from the model providers as to the data sources used to train its models. Most model providers obfuscate the sources deliberately because they can’t determine if copyrighted was part of the training data or because they are simply scraping data from any website they find. I expect enterprise will increasing prefer to use vendors who are transparent and guarantee that no copyrighted materials were used to train their models both ethic and legal protection reasons.

Hare is a distinguished vice president in Gartner's Technology and Service Provider Research and Advisory group. His research coverage includes AI, data science and analytics and BI. He advises tech providers on how to best adapt their product and go-to-market strategy to emerging technologies and trends, market dynamics and buyer preferences.