In an age where ambient computing (an expanding paradigm in which digital systems and intelligent services are seamlessly embedded into the environment) is no longer just a vision but a growing product category, decentralisation is becoming increasingly relevant. For ambient intelligence to be better aligned with privacy goals, it must be offline-first (avoiding the transmission of the content of your local tasks to remote servers) and private by default. It must not be reliant on someone else’s server in the cloud.
Inference (the act of ‘thinking’ by an AI model) is under growing pressure to happen locally, privately, and securely. While the term ESG itself is now contentious, AI increasingly carries the weight of all three elements. Environmental concerns make headlines, including the energy demands of training large models and the impact of water-cooled datacentres. Social questions follow, such as the risk of job displacement, or the communities affected by opaque algorithmic decisions.
Governance - encompassing everything from standards and regulatory bodies to internal oversight - remains the quietest part of the conversation, often overshadowed by more publicly discussed concerns, such as large companies unleashing artificial general intelligence (AGI). Yet governance is hardest to ignore in the centralised ecosystems currently shaped by dominant tech firms. These platforms consolidate power, obscure accountability, and raise essential ethical questions. Who decides what is acceptable? Whose values are built into the model? In this context, governance is not an afterthought, it is a critical area of concern.
Who, then, are the members of this new Promethean horde that aims to give the power of fire to many? Are they launching a collective jailbreak from the landscape dominated by a few large players, and if so, are they all escaping the same thing?
This group spans open-source groups such as EleutherAI and LLaMA-adapters, value-aligned companies like Hugging Face, and more ambiguous figures, such as Emad Mostaque, who has advocated for open-access diffusion models since his departure from StabilityAI. While his stance has sparked meaningful discussion around openness in AI, some observers question whether his motivations align fully with the broader decentralisation movement, highlighting the complex interplay between individual advocacy and institutional history.
In contrast, some of the dominant platforms acting as the incumbents offer risk to those who challenge them: the risk of reputational attacks, aggressive acquisitions, or quiet suppression through preemptive patenting. For the companies that are fully dependent on them to operate, the possibility of having them create a backdoor to observe what another company’s end customers are doing poses a plausible threat. And the possibility of having the account terminated as a retaliatory measure remains a persistent concern.
It is not surprising that today’s leading AI companies are based in the US and China, countries where alignment with national policy is frequently mandated by regulatory context. At best, corporate safeguards may conflict with commercial needs (as shown in debates over the perceived inauthentic outcomes of Gemini’s image generation). At worst, these systems become insufficiently transparent windows into user intent, creating a model-generated snapshot derived from their conversation history, context, and behaviour. This creates opportunities for privacy invasion and profiling.
As the initial AI race for technology wins and market shares led by these large firms begins to cool and the pressure to deliver profits increases, centralised offerings may lose their appeal. This shift is driven in part by to rising financial risk for users, even as the core product changes little.
At the same time, new governance approaches are emerging. Ideas such as “democratisation,” “sovereign AI,” and transparent tooling are gaining traction, but they all rely on a shared principle: standardisation and better tools are needed first in order for the current black boxes to be opened. Even some figures from the corporate labs of incumbents have tapped into this dissatisfaction, calling for decentralisation. Whether these calls reflect a genuine strategic shift or serve to maintain influence over the decentralisation narrative remains debated. Some analysts point to a blend of openness and commercial self-interest, suggesting the motives may be mixed rather than purely ideological.
Alongside this, the idea of self-hosted, on-device AI continues to surface. Sometimes it appears as a quiet technical milestone, and sometimes it arrives in tech’s headlines in a dramatic way. But these are often just demos of what is possible, without financial backing or the engagement of a community to carry it to its full potential. Hundreds of projects are working toward this goal, whether under permissive licences or led by OEMs. The motivation changes depending on the actor, but one motivation remains consistent: to challenge a technologically fragmented landscape but a commercially concentrated one. Just as crucial is the emergence of a new generation of engineers who, equipped with open research and accessible tooling, are no longer reliant on elite institutions or closed corporate labs to innovate. Their presence is contributing to a more distributed innovation ecosystem tackling the historical concentration of talent, long a key enabler of centralised control.
Elsewhere, individuals like David Heinemeier Hansson have sparked a wider reassessment of Big Cloud, as in his 2023 essay “*We have left the cloud*”. His decision to leave hyperscaler infrastructure has become a rallying point for digital sovereignty. That same mindset is now influencing the decentralised AI movement that sees the brain of software 2.0 (as Karpathy calls it) manageable on device. From alternative energy generation to federated runtimes designed for edge NPUs, the decentralisation landscape is expanding. Startups such as Modular which aims to be a ground-breaking platform, but leaves its blueprint, tools and research open for the world to see focus on serving efficiency, while projects like llama.cpp show that CPU inference is already viable.
The boundary between hardware and software is narrowing. New chip designs and SDK abstractions allow developers to be more productive by lessening the concerns around portability, or hardware limitations, even if those limitations haven’t disappeared entirely. Laptops with efficient modes, modular NPU upgrades (such as Raspberry Pi or LattePanda Mu), and cross-platform runtimes like ONNX are reshaping expectations around what commodity hardware is capable of when it comes to inference. Microsoft’s ONNX Runtime, initially developed for device standardisation, now supports open-source models running almost anywhere. Apple’s Neural Engine and the latest Copilot+ PCs offer NPUs capable of over 40 Tera Operations Per Second. That is enough to run models such as Stable Diffusion directly on-device. This vision is no longer speculative. Teams worldwide are working on democratising AI workstations that run full toolchains offline with no cloud tether, no API tokens, and no per-query costs. Sovereign computation is now a viable reality, even for hobbyists. Still, this path introduces its own risks. If decentralisation simply shifts control from cloud providers to chip manufacturers (Apple, Qualcomm, Intel, AMD), it may only replace one dependency with another, albeit more diversity in hardware involves lower lock-in risk. General models with acceptable performance are already running locally. Mistral, Phi-3, TinyLlama, and Gemma all are capable of running locally with usable speed and accuracy for basic text-based tasks. Distributed learning and edge inference are starting to challenge the assumption that AI must operate in the cloud. However, decentralised learning is not without cost. It requires new frameworks (e.g. Flower, FedML) for version control, trust, aggregation, and local evaluation. System-on-Chip (SoC) architectures may offer the most promising route to ambient AI, but they also carry trade-offs. These chips provide autonomy, yet their novel design introduces potential attack surfaces for new surveillance mechanisms.
Projects such as Semantic Kernel are enabling models to run directly within tools that developers can create. This supports new forms of interaction inside software products and allows tighter integration for local, offline use.
The darker side of decentralisation brings us back to the same letter G - once again governance - but this time noted for its possible absence, which should not be marked as a liberation. The possibilities of grey-market surveillance tools, shadow models for bioweapons, libertarian extremism dressed as digital liberty, all due to the illusion, oft-seen in politics that decentralisation is self-regulating.
Their goals are messy, loud, and contradictory, but maybe that’s the point. If fire is to belong to everyone, it must sometimes burn in strange hands.