drpontus

By Pontus Wärnestål

Generative AI has been deployed at scale before societies had time to understand its consequences. In only a few years, experimental models have rapidly become embedded in education, healthcare, media, public services, and daily communication. The dominating narrative calls this progress. In reality, it is a transfer of influence and control over information and decision-making to a handful of Silicon Valley-based technology companies.

That is why AI governance is the most pressing issue of the AI era. Without it, we are not shaping this technology. We are allowing a handful of private actors to shape our economies, institutions, and public discourse.

What AI Governance Actually Means

AI governance is often misunderstood as simple regulation or compliance. It is neither bureaucratic overhead nor a brake on innovation. AI governance is the system of rules, institutions, technical safeguards, and accountability structures that determine how AI is designed, deployed, evaluated, and controlled.

It includes:

• Legal frameworks defining responsibility and liability
• Technical standards for safety, robustness, and transparency
• Independent oversight and auditing mechanisms
• Ethical and democratic accountability structures
• Public infrastructure for trustworthy and sovereign AI development
• Labor, environmental, and cultural protections related to AI supply chains

In short, governance determines whether AI serves society — or whether society becomes a testing ground for tech companies.

The Governance Gap

The latest LLM-powered AI technology has advanced at a pace that far outstrips our ability to regulate it. Companies release increasingly powerful models into public and institutional use while the systems that should verify safety, evaluate societal risks, and assign responsibility remain fragmented or nonexistent.

This gap is structural. The current economic incentives of the AI industry reward rapid deployment, market capture, and scale. Safety, transparency, and accountability are viewed as slowing down that process. As a result, governance is treated as a secondary concern – something to be added after technologies are widely adopted and dependencies are already established.

History suggests this is the most dangerous phase of technological development. Industrialization, pharmaceuticals, aviation, and nuclear power all demonstrate the same pattern: early expansion without oversight creates systemic risk that later requires costly and reactive regulation. AI is following that trajectory, but at unprecedented speed and scale.

AI development has been driven by impressive demonstrations rather than proven reliability or societal readiness. The risk is not that AI exists, but that it is deployed in complex social systems before we understand how to control it.

AI Is Not Just Software. It Is Power Infrastructure.

Generative AI is often described as a productivity tool. That framing obscures its real impact. AI systems increasingly shape information flows, language use, economic opportunities, and decision-making processes. They are becoming a layer of societal infrastructure.

Infrastructure carries power. Whoever controls it influences communication, knowledge production, and public services. When AI infrastructure is controlled by a small number of private actors, governance is no longer just about technology. It becomes a question of democracy, sovereignty, and institutional resilience.

For smaller countries like Sweden, the issue is particularly acute. Reliance on external AI systems creates strategic dependency. If AI becomes foundational to education, public administration, healthcare, and communication, access and control over these systems becomes as critical as energy or telecommunications infrastructure.

AI sovereignty is therefore basic risk management and a resilience issue. It ensures that democratic societies retain the capacity to govern technologies that shape their citizens’ lives.

Alignment Is a Governance Problem

The concept of AI alignment is often framed as a technical challenge: how to make AI systems follow human values. But this framing avoids a fundamental question – whose values?

Alignment cannot be solved inside corporate research labs alone. Human values are negotiated through democratic processes, legal systems, and cultural institutions. Without governance, alignment becomes an internal corporate policy rather than a societal decision.

True alignment requires enforceable standards. It requires transparency about training data, model behavior, and deployment contexts. It requires independent evaluation and public accountability. Without these mechanisms, alignment becomes marketing language rather than a measurable outcome.

The Hidden Costs of Ungoverned AI

The urgency of AI governance is not hypothetical. The consequences of insufficient oversight are already visible across multiple dimensions.

Creative industries face structural disruption as AI models are trained on copyrighted work without consent or compensation. Creative workers lose income as their work trains models without compensation. Invisible global labor markets support AI systems through data annotation and content moderation, often under poor working conditions. Content moderators in Kenya and the Philippines develop PTSD filtering training data for poverty wages. The environmental footprint of large-scale AI – including energy consumption and water use – remains opaque and largely unregulated.

Linguistic and cultural diversity erodes as English-centric models dominate. The legal costs of deepfakes, defamation, and disinformation fall on individuals and governments while tech companies invoke terms of service to shield themselves from liability. These companies increasingly operate as publishers and information intermediaries while avoiding the accountability traditionally required of those roles.

These issues are systemic outcomes of technological scaling without governance frameworks capable of distributing risks and benefits fairly.

Governance Enables Responsible Innovation

Current AI development exemplifies what happens when technological capability outpaces social wisdom about appropriate use. We have created powerful tools for mass content generation without considering whether replacing human creativity with statistical pattern matching serves any purpose beyond reducing labor costs. We have built systems that can mimic human reasoning without addressing whether mimicry advances understanding. We have enabled unprecedented surveillance and manipulation capabilities without establishing boundaries around acceptable applications.

The halo effect that surrounds AI makes these questions difficult to raise. Skepticism about specific deployments gets conflated with opposition to progress. Calls for oversight get framed as obstacles to innovation. Concerns about harms get dismissed as “luddism”. This rhetorical strategy benefits those who profit from unconstrained development while silencing those who bear its costs.

The dominant narrative suggests that governance slows innovation. Evidence from other sectors suggests the opposite. Aviation safety regulations made commercial flight trustworthy. Pharmaceutical oversight made medicine reliable. Environmental regulation drove cleaner industrial technologies.

We would never allow the pharmaceutical industry to self-regulate, yet we permit AI companies to deploy systems affecting hundreds of millions of users without independent oversight, safety testing, or liability frameworks. The comparison is apt. Both industries produce products with significant potential for societal harm. Both require expert evaluation before mass deployment. Yet only one operates under a regulatory regime designed to protect the public.

Governance creates stable conditions for innovation by building trust, ensuring safety, and distributing benefits more broadly. Without governance, technological progress becomes fragile. Public backlash, legal uncertainty, and systemic failures eventually undermine the technology itself.

Responsible AI innovation depends on governance structures that are credible, transparent, and enforceable.

What Rigorous AI Governance Should Look Like – And What Organizations Can Do Today

AI governance is often discussed as something governments or regulators must solve. But governance is not only a legal framework. It is also a design discipline and an operational responsibility. Every organization that develops, deploys, or procures AI systems becomes part of the governance ecosystem.

For small and medium-sized enterprises (SMEs), public organizations, and design teams, responsible AI governance is not about building large compliance departments. It is about embedding accountability, transparency, and human oversight directly into how services are designed and delivered.

Six practical pillars can guide that work.

1. Independent Evaluation and Continuous Testing

Governance begins with knowing how AI systems behave in real conditions.

What SMEs and organizations can do:

• Test AI systems with real-world scenarios before deployment, including edge cases and failure situations.
• Involve diverse users in testing to identify bias, accessibility barriers, or unexpected outcomes.
• Document known limitations and communicate them clearly to users and employees.
• Establish internal review checkpoints when AI systems are updated or retrained.

What designers can do:

• Design services that anticipate AI mistakes and allow users to correct or override automated outputs.
• Include clear signals that show when content or decisions are AI-generated.
• Create interfaces that encourage verification rather than blind trust.

Good AI design assumes systems will fail sometimes – and ensures those failures are visible, understandable, and recoverable.

2. Transparency and Impact Awareness

Responsible AI requires openness about what the system does, what data it uses, and what risks it carries.

What SMEs and organizations can do:

• Inform customers and employees when AI is being used in products or decision processes.
• Map what data is being used and ensure it is collected and processed legally and ethically.
• Conduct simple internal impact assessments before launching AI-powered services:
  • Who benefits?
  • Who might be harmed?
  • What could go wrong at scale?

What designers can do:

• Design user journeys that clearly explain when AI is involved and what role it plays.
• Use plain language explanations rather than technical disclaimers.
• Provide users with meaningful consent and choice when AI is used.
• Contribute to standards by documenting and sharing your ML-driven design patterns.

Transparency builds trust. Hidden automation erodes it.

3. Accountability and Responsibility Structures

AI systems often blur responsibility between developers, vendors, and organizations. Governance requires clarity about who is accountable when things go wrong.

What SMEs and organizations can do:

• Assign internal ownership for AI systems — someone must be responsible for oversight and risk management.
• Create escalation procedures for reporting AI errors, bias, or harmful outputs.
• Carefully evaluate third-party AI providers and demand documentation of safety and performance practices.

What designers can do:

• Build feedback loops that allow users to report harmful or incorrect AI outputs.
• Design services that preserve human review for high-impact decisions such as hiring, lending, or healthcare recommendations.

Accountability means AI systems are never allowed to operate without human responsibility attached.

4. Public and Ethical Procurement Choices

Many organizations do not build AI – they buy it. Procurement is therefore one of the most powerful governance tools available.

What SMEs and organizations can do:

• Choose AI vendors that provide transparency about training data, model limitations, and environmental impact.
• Prefer providers that support open standards and data portability to avoid long-term dependency.
• Include ethical and sustainability criteria in procurement decisions, not just price and performance.

What designers can do:

• Advocate internally for selecting tools that support responsible data practices and user safety.
• Ensure service designs do not create unnecessary reliance on opaque AI outputs.
• Make sure you understand enough of the technological aspects of ML and genAI to be a part of the professional conversation around how “AI” actually works.

Every procurement decision shapes the AI ecosystem.

5. Labor and Creative Supply Chain Responsibility

AI systems rely on large amounts of human labor and creative content. Governance requires recognizing and respecting that human foundation.

What SMEs and organizations can do:

• Avoid using AI systems that clearly exploit copyrighted or ethically questionable training data when possible.
• Credit and compensate human creators when AI tools incorporate identifiable creative contributions.
• Ensure employees understand how AI affects their roles and provide training that empowers rather than replaces them.

What designers can do:

• Design workflows where AI supports human creativity instead of replacing human authorship.
• Highlight and preserve human contribution in AI-assisted services.

Responsible AI should augment human work, not erase its value.

6. Continuous Monitoring and Adaptive Governance

AI systems evolve over time. Governance must evolve with them.

What SMEs and organizations can do:

• Regularly audit AI performance, user feedback, and unintended consequences.
• Track how AI affects customer trust, employee workflows, and decision quality.
• Update internal policies and service design based on real-world outcomes.

What designers can do:

• Treat AI-enabled services as living systems that require iteration and monitoring.
• Design dashboards and reporting tools that help organizations observe AI behavior over time.

Governance is not a one-time checklist. It is an ongoing responsibility.

Governance as a Design Opportunity

For SMEs and designers, AI governance is not simply risk management. It is a competitive and ethical advantage. Organizations that design transparent, accountable, and trustworthy AI services build stronger customer relationships, reduce legal risk, and create more resilient products.

Responsible AI design also aligns with long-term innovation. Systems that users understand, trust, and control are more likely to be adopted sustainably.

Governance is therefore not only about avoiding harm. It is about designing technology that earns trust and creates lasting societal value.

Choosing Responsibility and Direction over Speed

Every major technological shift forces societies to decide what kind of future they are willing to build. Generative AI is no exception. It carries enormous potential: it can strengthen public services, accelerate scientific discovery, and expand access to knowledge. But it also carries the capacity to concentrate power, erode cultural diversity, destabilize labor markets, and weaken trust in information systems.

Governance determines which of these futures becomes reality.

Too often, the debate around AI is framed as a race – a competition between nations, companies, and institutions to develop more powerful systems faster than everyone else. But the real race is not technological. It is moral and institutional. It is the race between capability and responsibility.

Right now, technological capability is accelerating rapidly. Responsibility is not.

Dutch historian and author Rutger Bregman describes moral ambition as the willingness to dedicate talent, resources, and political will to solving humanity’s most urgent and complex problems. Moral ambition rejects the idea that the most capable actors should simply pursue profit, prestige, or technological dominance. Instead, it asks what those actors owe to society.

Artificial intelligence demands precisely this kind of ambition.

Developing systems that shape language, information flows, education, public administration, and democratic discourse is not a neutral technical exercise. It is an act that redistributes power across society. And power, when left ungoverned, rarely distributes itself fairly.

The question is no longer whether AI will influence our future. It already does. The question is whether we will take responsibility for guiding that influence.

Responsibility means acknowledging that technological progress does not automatically produce social progress. It means accepting that safety, fairness, sustainability, and democratic accountability must be designed into AI systems deliberately. It means building institutions capable of auditing, regulating, and shaping technologies that are increasingly embedded in everyday life.

Most importantly, responsibility means rejecting the idea that governance is an obstacle to innovation. Governance is what makes innovation legitimate, sustainable, and worthy of public trust.

History offers a clear lesson. The Industrial Revolution created unprecedented wealth and productivity – but it also produced exploitation, inequality, and social upheaval. The benefits society now associates with industrialization did not emerge from technology alone. They emerged from labor movements, democratic reform, public regulation, and collective demands for fairness and safety.

The same is true for AI.

If we want artificial intelligence to strengthen democracy, improve working life, preserve cultural and linguistic diversity, and contribute to a sustainable future, then governance cannot remain reactive or symbolic. It must be proactive, evidence-based, and democratically grounded. It must be built with the same seriousness and ambition that currently drives technological development itself.

Artificial intelligence may shape the future. But responsibility will decide whether that future is worth living in.