Foundations: Living on the Edge

[This is a Foundations article. Less opinion, more textbook.]

I spend a lot of time talking to clients about where their physical AI should run. Here is the situation most of them are in. They have assets out in the field, anywhere from a few dozen to many thousands, and every one of them is wired with sensors. Those sensors have been streaming data back to a central repository for years, faster than anyone has been able to make use of it.

By most estimates, the large majority of that data is never looked at again. It gets stored, it gets paid for, and it sits there.

So the instinct is: we've already collected everything into the cloud, let's run the intelligence there too. It sounds reasonable. But the data didn't originate in the cloud. Physical AI is about the real world, and the real world is somewhere else.

That somewhere else is what we call the edge: broadly, wherever the machines and their sensors actually are. For a manufacturing facility, that is on the floor, on the machines themselves. For an oil drilling platform, it is at the drill site. For a fleet of trucks, it is on the truck.

Two more terms, because I'll use them constantly:

The term physical AI gets used loosely, so it's worth pinning down. I introduced it in an earlier article; here's the short version.

The AI most people think of is a chatbot, and a chatbot has two properties worth noticing. It works in transactions: you type a question, it sends back an answer, and then it sits idle until you ask again. And it has no location: it doesn't matter whether the computer answering you is next door or on another continent, because nothing about that exchange depends on distance.

Physical AI is intelligence applied to physical assets. Robots and self-driving cars are the famous cases, but the category is much wider: the HVAC system on the building, the pumps at a well site, the trucks on the road, the equipment on the manufacturing floor, the chillers in your data center. And it flips both properties.

It isn't a transaction. It runs continuously against a stream of data that never stops, processing it as it arrives so you can optimize how the pumps run, when the fleet gets maintained, and what it costs to cool the data center.

And it has a location. The answer has to act on a machine, and the machine is somewhere in particular. Where the intelligence runs relative to that machine is the question this article is about.

Machine sensing itself isn't new. We've had sensors on equipment for decades, and two ways of dealing with what they produce: haul everything back to a central place and run reports on it, or run a threshold or a rule on the controller right at the machine. Either way, what you could do with the data was narrow. Statistics, signal processing, maybe a simple model tuned to one machine. A vibration alarm that fired when a number crossed a line.

Physical AI is a different order of capability. For the first time we can look at the raw signal from a machine and understand what's happening inside it, not just whether a number is high or low. That's a huge unlock.

And because it's new, the question of where that intelligence should run is genuinely open.

When deciding where to deploy your physical AI, there are five things to consider.

1. Data gravity. Data is heavy. The higher the sampling frequency, the heavier it gets, and moving it around is hard and expensive. What AI does, in effect, is compress data. You go from an enormous volume of raw sensor readings to a single intelligence event: this bearing is failing, this compressor is running out of spec. An intelligence event is small: which asset, what's wrong, how confident the model is, and what to do about it. A few hundred bytes standing in for hours of raw signal. Once you see it that way, the question becomes obvious. Why move the heavy thing to the intelligence when you could move the intelligence to the heavy thing?

2. Latency. Your hand pulls off a hot stove before your brain has finished processing that it's hot. Some decisions in physical systems are like that. If the model is closing a control loop, adjusting a valve, slowing a motor, stopping a robot, the answer has to act right where the machine is, and a few hundred milliseconds of round trip isn't a performance issue. It's the difference between a valve that closes in time and a tank that overflows, or between a near miss and an injury.

3. Resiliency. A lot of physical AI runs where connectivity is unreliable or absent altogether. Underground. Offshore. Remote sites. And in the critical situations where you most need the intelligence, the loss of a grid, a backup generator kicking in, the network is often the first thing to go. You can't have the intelligence disappear at exactly the moment it matters.

4. Security and sovereignty. If physical AI runs where the data originates, the data never has to leave. There are regulations dictating which country certain data can physically sit in. There are other industry specific regulations concerned with the privacy of people and citizens that need to be considered.

5. Scope. Some decisions are about one machine. Is this bearing failing? Is there a person in that aisle? Others are about the fleet. Which turbines should the technicians visit this week? Is the fault we saw on one line showing up on the others? Fleet decisions need data from every asset, so they have to run somewhere that can see all of it. That's the reason to run intelligence centrally, and it's a good one. It doesn't have to conflict with data gravity, and we'll come back to how the two are reconciled.

The five considerations tell you where a piece of intelligence ought to run. Two more decide whether it can.

6. Capability. Every location has limits on space, power, and cooling. A model that needs a fast GPU won't run on a small sensor, and a box that fits in a pump house won't hold what a rack in the data center holds. The question is whether the location you want can carry the model you have, and whether you can upgrade it if not.

7. Cost. Bandwidth, storage, cloud compute, the hardware you'd have to put in the field instead, and the people who have to keep it running there. Which way it comes out depends on the fleet.

There isn't a binary choice between "in the cloud" and "on the device." There's a whole chain of locations in between, and each one is a legitimate place to run inference. One useful way to group them is into three bands: central, near edge and far edge.

Central: away from the machines, a handful of large facilities that can see the whole fleet.

Near edge: in between, compute that's regional or on the premises, close enough to be quick but big enough to serve many machines at once.

Far edge: the machines, their sensors, and the small boxes bolted next to them.

The three bands aren't alternatives. They nest, like an onion: central at the core, near edge around it, far edge on the outside.

Intelligence flows inward. Each band produces intelligence events and passes them to the next one in, so raw data gets compressed at every hop and the core only sees what it needs to. This is how data gravity and scope are reconciled: the raw data stays out, the events go in, and the center sees the fleet without carrying the fleet's data.

Not every band has to be occupied. A wind farm might run on the turbines and in the cloud with nothing in between. The bands you skip are as much a design choice as the ones you fill.

Seven questions can be used to decide where your physical AI should run. The first five are the considerations discussed, turned into questions. The last two are practical considerations.

Answer these questions and the location usually picks itself.

Here's the process run twice, on cases that land in different places.

Two requirements:

Most of the questions don't force anything here. The data is a few readings a second, not video, so it can travel. Telemetry isn't sensitive. The turbine already has a controller, and the control model is small.

Two questions decide it. What happens when the connection drops? The turbine has to keep running correctly regardless, so control has to live on the turbine, and the pitch loop can't wait for a round trip anyway. One asset or the fleet? Control is one turbine. Working out who needs a technician is every turbine, and that can only be seen from the center.

So it splits. Control runs on each turbine. Alerts go up to the operators as soon as there's a connection. The raw sensor history goes back to the cloud on its own schedule, where predictive-maintenance models run across the whole fleet and work out which turbines to visit first. Two requirements, two places, and the same raw data serves both.

In 2018 alone, 85 US workers were killed by forklifts, and OSHA has long estimated around 35,000 serious injuries a year. The leading cause of death is a pedestrian being struck.

Two requirements:

Here nearly every question points the same way. Dozens of cameras produce far more video than anyone could stream, or would ever watch. The alarm has to fire within a second or two, and the forklifts don't stop when the network does. Footage of your own workers is personal data, and in many places it can't leave the building. The installed cameras can't run a vision model, but a single new box on the floor can watch every camera in the building.

So the vision model runs on a box in the warehouse. What travels upstream is an event: which zone, what kind of near miss, when, how confident. Whether a video clip travels at all is a separate decision, and if it does, faces are blurred first. The safety team trends anonymized events across every site in the cloud. The raw footage never goes anywhere.

In my work with clients, deployments almost always start in the cloud. Very quickly, though, the goal becomes pushing inference out toward the asset while the platform that manages where models are deployed stays central.

One caveat. Running intelligence in the field brings its own problems: hardware that varies, environments that are hostile, models that drift. Each gets its own article.

The places to run physical AI already exist, from the cloud down to the sensor, and every one of them gets more capable each year. The question isn't whether you can put intelligence where the machines are. It's which intelligence belongs where.

Sisinio Baldis is head of solutions engineering at Archetype AI, where he works with teams deploying physical AI in factories, industrial sites, and other real-world environments. Physical AI from the Frontline is a field guide to what works, what doesn't, and why. New issues arrive every few weeks.