Cloud or onsite: where should AI run in a logistics operation?

When a company starts talking about AI, the question of where it runs tends to be answered by default. The AI providers are in the cloud, so the AI runs in the cloud. For a head office with a fibre connection that is fine. For a warehouse at the end of a long rural link, or a cold store whose internet drops twice a week, it is a decision that deserves more thought.

The honest answer is that some work belongs in the cloud, some has to run at the site, and the choice is made for each integration rather than once for the whole company.

A site is not an office

Offices have reliable internet, quiet networks and people who can wait a few seconds for a page to load. Sites are different. The link may be a single consumer-grade line shared with the scanners and the cameras. Distant sites often sit on slow, high-latency connections where every round trip to the cloud takes noticeably long. Some sites lose their connection during storms, during peaks, or whenever a truck parks in the wrong place.

Meanwhile the floor keeps moving. Trucks arrive, pallets are picked, and a cold room keeps getting warmer whether or not the internet is up. Anything the floor depends on from minute to minute has to keep working when the connection does not.

What can run in the cloud

Plenty of useful work is not minute to minute. A demand forecast for next week, a daily summary for head office, a review of yesterday's late orders, a suggestion to move stock between branches: these read data that is already in a central copy and produce a result that someone looks at later. If the link is down for an hour, nothing on the floor notices.

The cloud is also where the larger AI models live, and where reading documents, emails and images is easiest to do well. Turning client emails into order entries is a good example: the mailbox is already in the cloud, the result is checked by a person, and a short delay while the link recovers costs nothing.

What has to run at the site

Other work is tied to the moment. A temperature drift warning is only useful before the product is at risk, which means it has to run beside the sensors, not on the far side of a flaky link. A short pick alert that reaches the supervisor after the truck has left is noise. A dock camera that cannot be watched when the internet drops is a camera that stops working during the very weather that causes trouble.

For these, a small service runs on a server, a virtual machine or a PC inside the site's own network. It reads the local sensors, the local warehouse database and the local cameras, and it does its work there. Smaller AI models can run on modest hardware at the site; where a larger model in the cloud is needed, the local service queues its requests and carries on with what it can decide alone.

What happens when the connection drops

This is the question to ask of every integration before it goes live, because it will happen. There are three acceptable answers and one unacceptable one.

• The integration carries on at the site and catches up when the link returns. Right for anything the floor relies on.
• The integration pauses, and nothing on the floor depends on it until it resumes. Right for forecasts, summaries and reviews.
• The integration keeps a queue, so emails or readings that arrived during the outage are processed in order once the link returns and nothing is skipped.
• The unacceptable answer is that the floor stops, or that a person has to notice the outage and restart something by hand.

The floor comes first in every case. An integration that competes with the scanners for a thin link, or that leans on the warehouse database during the busiest hour, is doing harm even when it is right. It should ease off when the site is under pressure and catch up when the pressure lifts.

Choose per integration, not once for everything

Companies sometimes try to settle this as policy: everything in the cloud, or nothing leaves the building. Both are wrong often enough to matter. A cloud-only policy leaves the cold store without its drift warning during a storm. An onsite-only policy means the forecast runs on a PC under a desk that nobody backs up.

The better approach is to decide for each integration, based on three things: how quickly the result is needed, what happens if the link is down, and where the data it reads already lives. Write the answer down as part of the design, test the outage case deliberately during the pilot, and revisit it if the site's link or hardware changes.

Decide where each integration runs by asking what the floor needs when the internet is down, because one day it will be.

This is one of the decisions redfly makes with you during the design of each integration: where it runs, what it reads and what it hands back, planned for both the cloud and the site, with the results landing in the redfly Portal for your people when the link allows and the floor kept working either way.