AI: Assistance or Influence
Driven by recent advances in Machine Learning, the adoption of artificial intelligence in enterprises has been growing exponentially. According to Forbes, three out of every four organizations worldwide already use AI in at least one business application. This scenario has led analysts to predict that the market will quadruple over the next three years, reaching US$ 1.4 trillion by 2030.
Although the rapid adoption of AI brings real benefits, it raises concerns about the expansion of systemic risks, including algorithmic bias, lack of transparency, privacy violations, and even reputational damage. The result is a growing regulatory responsibility that organizations must address urgently, especially now, with the entry into force of the EU AI Act, the world’s first comprehensive legislation to regulate artificial intelligence.
Created by the European Union, this legislation establishes legal rules for the development, commercialization, and use of AI technologies. As happened with the GDPR (which inspired Brazil’s LGPD), the EU AI Act has extraterritorial effect, applying to any company in the world whose AI systems make products available or generate impacts within the European market, subjecting violators to heavy fines of up to 7% of their annual revenue.
The concern is legitimate. In his latest book, Reflections on AI Governance and Compliance — a work that masterfully dissects this subject — André Rizzo highlights: “What is at stake is not simply technological progress, but the gradual reconfiguration of how institutions perceive reality, allocate attention, and exercise authority.”
Machine Learning models do not merely help us modernize applications, automate processes, and create innovations. They primarily operate at the most basic and fundamental layer of any initiative — information — helping us extract, understand, summarize, categorize, and even generate it for subsequent use in complex decision-making chains. This interpretive framework, which previously relied on reality itself, now operates in light of training based on historical data.
A case widely reported by Reuters illustrates the dangers of this dynamic in practice. One of the world’s largest technology companies decided to put AI in charge of pre-screening résumés for its job openings. Over time, the organization’s HR team realized that the system prioritized almost exclusively male candidates. Upon investigating the problem in depth, they discovered that, by cross-referencing historical data on current employees — a population that had historically been predominantly male — the AI spontaneously inferred that there was an internal preference for individuals of that sex. As a result, the tool began systematically evaluating women’s applications negatively.
Although this project was discontinued, it demonstrates how algorithmic systems can inherit structural biases from the data used to train them. Rizzo reinforces this point: “When institutions rely on Machine Learning models trained on historical data, these models can reproduce outdated institutional patterns unless governance frameworks are used to actively examine how these patterns influence outcomes.”
Unlike other technological developments, AI is adopted without much fanfare or announcements. Without drawing attention, employees begin classifying data using newly available features in spreadsheets. Sales opportunities are automatically prioritized through assistants integrated into CRMs. Meetings are fully transcribed and summarized in collaboration tools, while emails, proposals, and contracts are drafted by autonomous agents. It is precisely this invisible infiltration that reinforces the urgency of raising governance maturity within organizations.
“Artificial intelligence does not require formal authorization to influence the institutional environment. As this process advances, the most critical governance challenge will not be controlling the technology itself. Instead, it will be ensuring that institutions remain capable of explaining how authority is exercised when intelligent systems participate in producing the information on which decisions depend,” warns Rizzo.
Like the European Union, jurisdictions such as South Korea, China, and some U.S. states — including Texas and California — already have their own regulatory frameworks for artificial intelligence. Meanwhile, Brazil, Canada, Japan, and Singapore are making progress in advancing similar frameworks through their legislative processes. In this scenario, multinational companies now face a true compliance puzzle. Mapping trends in the business environment is always complex, but one reality is indisputable: the advancement of AI within organizations is an irreversible path, which will require the inevitable maturation of institutional governance.