AI Is in Control: What Now?
Driven by recent advances in machine learning, the adoption of artificial intelligence in businesses has grown exponentially. According to Forbes, three out of every four organizations worldwide already use AI in at least one business application. This trend has led analysts to predict that the market will quadruple over the next three years, reaching $1.4 trillion by 2030.
Although the rapid adoption of AI brings real benefits, it raises concerns about increased systemic risks, including algorithmic bias, lack of transparency, privacy violations, and even reputational damage. The result is a growing regulatory burden that organizations must address urgently, especially now that the EU AI Act—the world’s first comprehensive legislation to regulate artificial intelligence—has come into effect.
Created by the European Union, the legislation establishes legal rules for the development, marketing, and use of AI technologies. Just as with the GDPR (which inspired Brazil’s LGPD), the EU AI Act has extraterritorial effect, applying to any company worldwide whose AI systems make products available or have an impact within the European market, subjecting violators to heavy fines of up to 7% of their annual revenue.
This concern is legitimate. In his most recent book, *Reflections on AI Governance and Compliance* —a work that masterfully dissects this subject—André Rizzo points out: “What is at stake is not simply technological progress, but the gradual reconfiguration of how institutions perceive reality, allocate attention, and exercise authority.”
Machine learning models do more than just help us modernize applications, automate processes, and drive innovation. They operate primarily at the most basic and fundamental level of any initiative—information—helping us extract, understand, summarize, categorize, and even generate it for later use in complex decision-making chains. This interpretive framework, which previously relied on reality itself, now operates based on training models grounded in historical data.
A case widely reported by Reuters illustrates the dangers of this dynamic in practice. One of the world’s largest technology companies decided to have AI handle the initial screening of resumes for its job openings. Over time, the organization’s HR team realized that the system was prioritizing almost exclusively male candidates. Upon investigating the problem in depth, they discovered that the AI, by cross-referencing historical data on current employees—a workforce that has historically been predominantly male—spontaneously inferred that there was an internal preference for male candidates. As a result, the tool began to systematically evaluate women’s applications negatively.
Although this project has been discontinued, it demonstrates how algorithmic systems can inherit structural biases from the data used to train them. Rizzo emphasizes that “when institutions rely on machine learning models trained on historical data, these models can reproduce outdated institutional patterns unless governance frameworks are used to actively examine how these patterns influence the results.”
Unlike other technological advancements, AI is being adopted without much fanfare or fanfare. Without making a big deal of it, employees are now categorizing data using newly available features in spreadsheets. Sales opportunities are automatically prioritized by assistants integrated into the CRM. Meetings are fully transcribed and summarized in collaboration tools, while emails, proposals, and contracts are drafted by autonomous agents. It is precisely this invisible infiltration that underscores the urgency of raising the maturity of governance within organizations.
“Artificial intelligence does not require formal authorization to influence the institutional environment. As this process moves forward, the most critical governance challenge will not be controlling the technology itself. Instead, it will be ensuring that institutions remain capable of explaining how authority is exercised when intelligent systems are involved in generating the information on which decisions depend,” warns Rizzo.
Like the European Union, jurisdictions such as South Korea, China, and some U.S. states—such as Texas and California—already have their own regulatory frameworks for artificial intelligence. Meanwhile, Brazil, Canada, Japan, and Singapore are moving forward with similar frameworks. Against this backdrop, multinational companies today face a veritable compliance puzzle. Mapping trends in the business environment is always complex, but one reality is indisputable: the advancement of AI within organizations is a path of no return, which will inevitably require institutional governance to mature.