Author: William Vorhies
Summary: Less than 9%? What this study really shows and what we should take away from it.
Wow. Less than 9%! Can this be true? Well according to a large scale survey study conducted by the US Census Bureau it’s actually a little worse than that since the 9% applies to a basket of advanced technologies some of which don’t involve AI, like RFIDs. So what’s the story?
Turns out the Census Bureau has a mandate to survey US businesses about their use of technology. This report of their most current work was presented in July at a National Bureau of Economic Research conference. There are several twists to this story but the main one is how do we reconcile this result with the many other surveys we all read that say that between 20% and 33% are implementing AI ‘at scale’ and that a much larger percentage are right behind. Even that estimate dates back to surveys conducted in 2018 and based on current literature you’d have to believe adoption is much, much higher.
About the Census Survey
The “2018 Annual Business Survey (ABS)” (survey data from 2017) sought to answer the degree to which US companies were adopting advanced technologies. They first queried about ‘digitization’ figuring that was a precursor to all advanced technologies, then about ‘cloud computing’, and finally about a basket of ‘advanced technologies’ some of which relate to AI and some which don’t.
The advantage the Census has over other surveys we’ve seen is that response is legally required. They mailed out about 850,000 surveys and received 583,000 responses or almost 69%. That’s much better than the 20% or 30% response rates of large business surveys we’ve reported in the past. This actually lets them project out over the entire US business population with some statistical accuracy without the curse of respondent bias.
The first twist, and it’s unclear why they elected this, is that the survey includes only “all private, non-agricultural sectors of the economy”. So the exclusion of public companies immediately excludes at least the largest 4,000 or 5,000 US companies, who logically, are leaders in AI adoption.
So the results of this survey are a kind of trickle down story about what the rest of us are doing, with most responses from very small companies.
- 1 to 9 employees 75%
- 10 to 49 employees 20%
- 50 to 249 employees 4%
- 250+ employees 1%
Unfortunately without access to the raw data we can’t further refine their findings to look at least at the largest of this group.
And the findings about ‘advanced technologies’ relate to these categories which fall within AI:
- Machine Learning 8%
- Voice Recognition 5%
- Machine Vision 7%
- Robotics 3%
- Natural Language 2%
- Automated Vehicles 8%
Versus these categories which don’t relate to AI according to the definitions given with the survey:
- Touchscreens 9%
- RFID 1%
- Augmented Reality 8%
If you’ve been quick to add these up, you’ll see the AI category totals a little over 10% but this includes overlaps. The real answer to utilization of AI is likely much smaller and more like 6% across this group.
Do Survey Respondents Even Recognize AI?
We’ve written in the past about how difficult it is to measure adoption. There’s no end of organizations conducting surveys. If a large company has implemented a chatbot in one operation do we give them credit for adoption (as many surveys do)? Do the folks who respond to these surveys know what’s going on in other parts of their companies if those organizations are large and dispersed?
In smaller companies like those surveyed you’d expect them to know if, for example, their operation used a robot. However so much of AI is now buried in applications these companies may be using they may be completely unaware.
A further confounding factor is that the survey specifically asked the respondents about the use of these technologies in “the production of goods and services”. Were they sophisticated enough in their understanding to include or exclude the many types of AI found in support systems like HR and finance that heavily incorporate machine vision and NLP. Probably not.
What’s the Degree of Adoption?
The survey did allow the respondents to provide some information about the degree to which they had adopted these technologies expressed as percentage ranges over their entire company.
What’s the Takeaway?
There are several.
Despite AI companies wanting to know about market size and penetration, the traditional voluntary survey of large companies suffers from low response rates and respondent bias making any statistical conclusions no better than guesses.
In smaller companies respondents are unlikely to have sufficiently deep understanding of AI applications to actually spot all the places AI may already be at use in applications.
In large companies the specific respondents answering surveys are unlikely to be completely aware of all the AI applications planned or used throughout a large dispersed corporation.
Particularly in large companies, do we give credit for AI adoption if there is one chatbot in customer service or one ML model embedded in a purchased-in recommender. The real metric ought to be total spend, internal and external on AI.
Increasingly, AI is disappearing into the infrastructure of all sorts of applications we purchase where users may be completely unaware. Computer vision, chatbots, NLP, and even ML algos are increasingly embedded in all sorts of day-to-day applications. It’s no longer relevant to ask about AI on the assumption that there has been a specific in-house project to develop and deploy an application that has captured significant attention and development effort.
Finally, of real concern is what government policy makers may make of this. Focusing AI policy or allocating resources to this end of the market without understanding the shortcomings of the survey may seriously misallocate funding and efforts.
It’s too much to hope that we’ll see an end to these ‘AI Adoption’ surveys. AI is here. You may not recognize all the activities you undertake that use AI, but that’s a good thing. As a smaller private company it’s not necessary to make a conscious decision to adopt AI. The vendors who provide your apps and services will see to that.
Other articles by Bill Vorhies.
About the author: Bill is Contributing Editor for Data Science Central. Bill is also President & Chief Data Scientist at Data-Magnum and has practiced as a data scientist since 2001. His articles have been read more than 2.1 million times.
Bill@DataScienceCentral.com or Bill@Data-Magnum.com