Why AI pilots fail on a busy floor, and what a good one looks like

The demo was impressive. Emails became orders, the dashboard lit up, and everyone in the room nodded. Three months later the pilot is quietly switched off and nobody wants to talk about it. This happens often enough in logistics that it deserves a plain explanation.

The short version is that a demo and a floor are different places. A demo runs on clean data, in a quiet room, with a patient person at the keyboard. A floor runs on whatever arrived, during a peak, with a shift change halfway through.

Demos run on clean data in a quiet room

A demo is built to succeed. The sample emails are well formed. The stock table has been tidied. The appointment calendar has no double bookings. The person showing it knows which questions to ask and which to avoid.

None of that is dishonest; it is what a demo is for. The trouble starts when a company treats the demo as proof that the system will work on its own data, in its own building, on a Friday afternoon in November. It is proof of something much smaller: that the idea can work when everything else cooperates.

Floors have peaks, exceptions and shift changes

A real warehouse supplies three things a demo never sees. The first is volume at peak. An integration that reads fifty emails a day comfortably may read four hundred on the day before a holiday, and the warehouse system it writes into is at its slowest on that same day.

The second is exceptions. Carriers send the same order twice with a small change. A client attaches last month's PDF by mistake. A pallet arrives with no paperwork at all. A sensor reads minus forty because someone unplugged it. Floors are made of exceptions, and an integration that only handles the normal case is handling the minority.

The third is shift change. The person who understood the pilot goes home, and the person who arrives was not in the meeting. If the integration needs explaining, it stops being used at the first handover. Anything the floor is expected to act on has to make sense to someone who has never seen it before, on a screen they already look at.

What a good pilot looks like

A good pilot is not a bigger demo. It is a small piece of real work, done the real way, with a few deliberate limits.

• It runs on real work. The emails are this week's emails, the stock table is the live one, and the dock calendar is the one the coordinator actually uses. If the data has to be cleaned up first, that cleanup is part of the pilot, not a step before it.
• It starts with a small group. One shift, one client, one dock, or one product family. Small enough that a problem is contained; real enough that a problem would show.
• It is tested with a set of checks, agreed in advance. For each kind of input, what should come out, and what should happen when the input is wrong. The pilot passes when the checks pass through a busy week, not when the room nods.
• It keeps a person in charge. The AI reads, compares and suggests; a named person confirms the order, approves the appointment or decides what to do about the flagged mismatch. That person can see why the suggestion was made, in plain words.
• It is rolled out with someone watching. When the group widens, the people who built it are looking at what comes through, every day, until the floor stops needing them to.

The exceptions are the test

The pilot earns its keep on the strange inputs. The duplicate email, the missing paperwork, the unplugged sensor. A useful way to run it is to collect the exceptions from the first fortnight, write each one into the set of checks, and keep going until a week passes without a new kind of surprise.

The checks also settle the argument that kills most pilots, which is whether it is "working". With agreed checks, working means something specific, and a disagreement becomes a conversation about one input rather than about the whole idea.

A demo shows the idea can work; a pilot shows it works here, on a Thursday, with the usual chaos.

What to ask before you start

Before any pilot begins, three questions are worth asking the people proposing it. Which real data will it read on day one? Who on the floor will be in charge of what it suggests? What is the list of checks it has to pass before the group widens? If any of those answers is vague, the pilot is still a demo.

This is how redfly runs the pilot stage of each integration with warehouse and distribution companies: on real work, with a small group, against an agreed set of checks, with a person in charge throughout, and with our team watching as it goes live, with the results landing in the redfly Portal where your people already look.