While LLMs have gone mainstream this year, the technology is still maturing and companies are taking a measured approach to adoption. The State of AI 2023 report, released by Retool, gives us a look into generative AI’s impact on job roles, and the practicalities of AI adoption in companies.
Despite the growing popularity and usage of LLMs like ChatGPT, most respondents (51.6%) find AI overrated. Interestingly, upper management tends to have a more favorable view, with VPs and C-suite executives rating AI closer to 'fairly rated'. In contrast, those involved in actual implementation, like entry-level employees and mid-to-senior individual contributors (ICs), lean more towards viewing AI as overrated. Seems like there’s a disparity in understanding and expectations of AI across different levels of a company.
The survey found over 75% of companies have started implementing AI in some capacity. However, nearly half described their efforts as fledgling – either in early pilot stages or limited to ad hoc use cases. About 16% have not begun adoption efforts at all and may not pursue AI soon. Internal AI use cases are more prevalent than external ones, ranging from automated code writing to customer support chatbots. Despite this growing adoption, concerns about AI’s accuracy and data security remain prevalent, especially in larger companies.
GitHub's Copilot led usage of AI coding assistants, which over half of developers reported using less since Copilot's release last year. A notable 57.6% of respondents have reduced their reliance on traditional platforms like StackOverflow, with 10% no longer using it at all, citing GitHub Copilot and ChatGPT as replacement resources.
Knowing how to use new AI tools like GitHub Copilot and ChatGPT is starting to matter more in job applications. Proficiency in tools like GitHub Copilot and ChatGPT is becoming a consideration in hiring, particularly in smaller companies and startups. Hiring practices are also evolving, with 60% expecting changes to technical interviews allowing AI assistance. However, there's still a degree of neutrality towards these skills, indicating an evolving landscape in technical hiring practices.
Most companies are currently consuming hosted AI services rather than building custom models. Where internal AI applications exist, ChatGPT is the predominant model used according to 80% of respondents. Self-hosting and customis
ation is still the exception for now. On the commercial side, Copilot and Grammarly led adoption of AI features embedded in products.
Accuracy and data governance were frequently cited challenges in applying AI responsibly. Output accuracy was the top concern, which can manifest in ways like hallucinations. Data security concerns increased almost linearly with company size. While about half of companies have explicit data policies governing AI use, nearly a third have no policy or remain unsure if one exists. The most common safeguards allow only anonymised data or limit specific classes of data.
As AI becomes more embedded in workplaces, the report points out the need for clear data policies and ethical guidelines around AI use. There's a notable portion of professionals using AI secretly at work, indicating a potential gap in policy clarity or development. Companies are also exploring customising AI models and tools, but this remains an exception rather than the norm.
"While generative AI has captured mainstream attention, we need to appreciate actual usage patterns to truly harness it," said Retool CEO David Hsu. "AI is reshaping technical jobs more than replacing them – and people want oversight into its ethical development."
While LLMs have gained significant traction, perceptions vary widely, and its full potential and implications are still being unraveled. The report underscores a critical period in AI development, where its integration into business and society continues to evolve amidst discussions on ethics, policies, and the future of work.
