How does data utilization change when the reader is no longer solely human? And what is the role of humans when machines not only analyze but also act as active agents?
The fundamental formula for data utilization has long been clear: business generates data, data is stored, analytics produces answers to business questions, and business benefits. The basic structure has not disappeared, but an entirely new layer has been built on top of it. A few years ago, I wrote about the anatomy of data utilization. At that time, the picture was clearer. Now generative AI has changed the playing field in a way that deserves a fresh perspective.
The nature and storage of data
At its simplest, data is a unit of information that gains value only through context and combination. However, a significant change has occurred in what can be done with data. Previously, unstructured data—such as log entries, software code masses, documents, emails, contracts, and Slack conversations—always required heavy pre-processing stages before any value could be extracted. Now language models can more directly read, interpret, and summarize this data.
Structured data, such as sales figures, orders, and customer information, still resides in databases and data warehouses. Data dimensions continue to guide storage methods and tool selections. The change lies in how data is made accessible to AI. Company documents and guidelines can be indexed so that a language model retrieves context from them for its responses. The expanded context windows of language models—the amount of information a model can process at once—enable even large datasets to be fed directly to the model, although context size alone does not guarantee that the model will effectively utilize all the information provided. AI can also use tools and APIs through which it retrieves and updates data in real time directly from source systems.
A larger proportion of the data that a company produces and collects is now being able to utilize in one way or another.
What AI does with data
A conversational layer can be built on top of data, where users ask questions in natural language and receive answers that combine information from multiple sources. Dashboards, reports, and predictive models are still needed. However, the way people access these results is changing: a business user can ask “How does last month’s sales compare to the previous year?” and receive an answer without needing to write queries or navigate a BI tool.
In addition to traditional analytics, generative AI brings entirely new capabilities. It can draft customer communications and contract texts based on company data, summarize and compare documents, generate product descriptions, or automatically classify support requests. However, AI’s capabilities are not limited to data analysis or content generation. A language model can generate code, automate processes, and orchestrate workflows. It can serve as an interface between humans and a wide variety of systems.
Herein lies the essential change. AI does not merely analyze data but produces something new based on it and acts accordingly. In my original article, the end product of data utilization was always information, based on which a human made a decision. Now the end product can be an action, or an entire process in which AI combines information, produces content, and executes steps autonomously.
Agents: from data to action
AI agents take this development to its furthest extent. Without tools, a language model’s operation is limited to what it can produce in a single response. AI agents are language models equipped with tools: they can retrieve data, call APIs, write and execute code, and modify files. Crucially, an agent operates in a loop: it plans, executes, evaluates the result, and continues until the task is complete.
The true power of agents emerges when they work together. A complex task can be divided among specialized agents, each mastering its own domain: one agent monitors and interprets data, another can write and format reports, a third knows the company’s systems and can initiate actions within them. An orchestrating agent coordinates the whole, distributes tasks, and assembles the results.
In practice, at its best, this might look like a data agent detecting an anomaly in delivery times, an analysis agent tracing it to its root cause, a reporting agent preparing a summary for the operations team, and an action agent proposing corrective routing. A human sees the complete picture and decides whether to implement the proposal.
Challenges: old problems, new dimensions
The challenges from the original article have gained new dimensions, and entirely new ones have emerged alongside them. Many of these challenges are also interconnected in a way that makes managing the whole more demanding than solving individual problems.
Data silos remain a real problem. Language models can function as a kind of unifying query layer across silos, but this does not eliminate the need for data governance and integration. A model cannot use data it cannot access, and the importance of APIs is emphasized.
Data quality is not a new challenge, but its significance is heightened. A language model can produce a convincing answer even from poor-quality data, making problems in the output harder to detect. High-quality source data remains the foundation of everything.
Reliability is perhaps the most critical challenge. Language models can produce convincing but incorrect answers. This makes human evaluation even more critical. Here we encounter the paradox of automation: the more the level of automation is increased, the more the importance of humans in the process grows. But at the same time, the human ability to fulfill this role weakens when routine work no longer maintains competence. 1
And when reliability is uncertain, the problem of measuring quality arises. In traditional analytics, measurement is more straightforward: numbers either match or they do not, and there are established metrics for the accuracy of predictive models. But how do you measure whether a customer message generated by a model is good enough? Or whether a summary is comprehensive? Evaluation often requires human judgment, and building a systematic feedback loop is work that easily goes undone.
Cost and performance are practical realities that surprise many. Running language models in production costs money, and costs scale with usage. Latency may be too high for real-time use cases. These constraints can be managed through architectural choices, such as selecting a lighter model for simple tasks, but they must be considered when building solutions.
Behind these practical challenges lies a skills gap, which is currently the biggest bottleneck for many organizations. Who will build and maintain AI solutions? People are needed who understand both business and the possibilities and limitations of AI. Competence does not arise spontaneously, and its absence can prevent otherwise promising projects from progressing. Without skilled professionals, reliability and quality problems remain unsolved, and costs spiral.
Data ownership and governance takes on new nuances. Whose data is fed to the language model? How is sensitive information prevented from leaking to third-party models? Companies must take a position on these questions before AI is deployed at scale.
The human role shifts from participation to guidance
The most important message of the original article was that understanding of business domains resides in the company’s people, and therefore they should be involved in data utilization. This remains entirely valid, but the role is changing.
Previously, people’s participation meant that business experts were involved in defining metrics, interpreting dashboards, and guiding analysis. Now the role expands: the task of people is to guide AI, critically evaluate its outputs, and design feedback loops through which the system’s quality is improved.
The required competence is changing. “I can read a dashboard” is no longer sufficient. The ability to formulate good questions for AI, assess the reliability of answers, understand when a model works well and when it does not, and design processes in which humans and AI complement each other is needed.
This is not an IT project
Fully leveraging generative AI is not a single project that can be delegated to someone else to handle. It changes the way an organization works with data, makes decisions, and serves customers. Change of this magnitude requires commitment from top management: a clear vision of how AI will be used, allocation of resources, and readiness to change operating methods.
There is now a new, powerful layer between data and business. Guiding it requires people who understand both business and AI. This competence determines who will truly gain competitive advantage from data.
- Lisanne Bainbridge, “Ironies of automation,” Automatica, 1983. ↩︎
