Insane Robotic Warehouse Automation

[EEVblog]

WOW! 

[Bud]

Looks 3D rendering to me 

[EEVblog]

Looks 3D rendering to me 

Nope, it's real. I have parts from it in my Mailbag.
Turns out is burned down! Once of the robots caught on fire! 

https://www.bbc.com/news/uk-england-hampshire-48094801

[nali]

Well, it's been well & truly re-rendered now!



https://www.bbc.co.uk/news/uk-england-hampshire-49071456

Edit: Drat, beaten to it!

[Gyro]

Damn, the things that you assume the rest of the world knew all about. It was in the national news for days, on and off, here (it took a fair while to go out), Months, including the investigation.

It was something like a faulty charger setting fire to the plastic top of one of the robots, which then wondered off around the warehouse spreading the fire like a virus. 

[Cerebus]

It was something like a faulty charger setting fire to the plastic top of one of the robots, which then wondered off around the warehouse spreading the fire like a virus. 

Am I the only one with a mental image of a vaguely anthropomorphic robot running around in a panic, hands in the air, with its head on fire setting fire to everything it passes? Sounds like something that would happen to Futurama's Bender.

[I wanted a rude username]

Looks 3D rendering to me 

Modern video codecs perform weird optimisations to save bandwidth, such as arbitrary translation of regions of the frame, and these in particular make moving objects look fake. It's no longer enough to evaluate how the video "feels", because the codec is working against you. You need to look for details such as imperfections, reflections, etc.

[Gyro]

Irrc, the senior fire officer described it as 'not a human friendly environment'! 

[PA0PBZ]

"One of the pitfalls of a robot-only workplace: they can't smell the smoke when said workplace is on fire"

Well said!

[nali]

"One of the pitfalls of a robot-only workplace: they can't smell the smoke when said workplace is on fire"

Well said!

Heh... Humans are not a whole lot better!

The sprinkler system started operating 11 minutes later but was then turned off by Ocado engineers for five minutes which led to a "significant" growth in the fire.

Once staff realised the fire was not being extinguished they turned the sprinklers back on and finally dialled 999.

[Marco]

The only humans in the loop still found a way to be the weak link, impressive. Tried to save a penny in wasted stock, wasted 100 million warehouse and probably not much less due to loss of business.

[rstofer]

Looks like s few hundred lines of FORTRAN ought to cover it.

[thm_w]

Nope, it's real. I have parts from it in my Mailbag.
Turns out is burned down! Once of the robots caught on fire! 

https://www.bbc.com/news/uk-england-hampshire-48094801

What the hell.. hindsight being 20/20 I'm thinking if they spent the extra 20c per bin on flame resistant plastic this wouldn't have happened. I don't see much flammable material other than that thing.
Any kind of fire suppressant system would not be that desirable as it could ruin all of the stock.

[EEVblog]

It was something like a faulty charger setting fire to the plastic top of one of the robots, which then wondered off around the warehouse spreading the fire like a virus. 

PLEASE tell me they have CCTV footage of that?

[Cerebus]

What the hell.. hindsight being 20/20 I'm thinking if they spent the extra 20c per bin on flame resistant plastic this wouldn't have happened. I don't see much flammable material other than that thing.
Any kind of fire suppressant system would not be that desirable as it could ruin all of the stock.

I doubt that would have been sufficient. In fact I'd be very surprised if it wasn't already the case.

If you look at the contents of the average supermarket trolley it's chock-a-block with highly flammable materials (paper towels, cardboard, plastic film...) and backed up by stuff that burns really well once the former has acted as a fire starter (cooking oil, cornflakes, bread). Heck, if the Ocardo warehouse holds the same things as my local supermarket then there are cans of butane gas, boxes of matches and actual packets of fire-starters backed up by lots and lots of bottles of whiskey, gin, vodka... With those as a given it looks like an environment that's nigh impossible to properly fire-proof. All it's going to take is a spark in the wrong place, or a gob of sufficiently hot melted plastic hitting the wrong material (fire retardant plastic still gets hot, melts and even smoulders) and poof! up it all goes.

I think the only option is a VESDA (very early smoke detection) system that removes all power from the place (including remotely popping isolation relays on the robot batteries) backed up by a heavyweight fire suppression system that fires if the power cut off is ineffective. It might well ruin all the stock, but that's better than the stock, the whole system and the building going up in flames. Cleaning up has got to be better than rebuilding from scratch.

If they didn't already think of it, kitting out a few robots as localised fire fighters would be a good idea. Having them scoot to the location of any detected fire and start spraying fire suppression chemicals over a limited area would probably stop a fire spreading and limit the clean up to a comparatively small area.

And it would give the lady robots a source of jpegs to pin up on their scheduling tables - "Mr February, he may be small, but he packs a lot of watt-hours into his battery and has an impressive fire suppressant nozzle".

Edited to add:
Just reading the fire brigade (short) report. They did have VESDA, it failed to detect the fire - no information in the report as to why it failed. Fire was detected visual by a human 30 minutes after it had started.

[Cerebus]

It was something like a faulty charger setting fire to the plastic top of one of the robots, which then wondered off around the warehouse spreading the fire like a virus. 

PLEASE tell me they have CCTV footage of that?

[Gyro]

It was something like a faulty charger setting fire to the plastic top of one of the robots, which then wondered off around the warehouse spreading the fire like a virus. 

PLEASE tell me they have CCTV footage of that?

Apparently they do have CCTV footage but Ocado probably have no intention of ever releasing it.  Shame, it would have been something to behold, it would probably break some records on YouTube!

The closest we have is the Hampshire Fire and Rescue report summary, that makes reference to the video... http://democracy.hants.gov.uk/documents/s36357/Report.pdf

I did wonder how they tied down the cause among all that mess - it either had to be CCTV footage, or the poor robot's electronic screams echoing across the network! 

[Cerebus]

The closest we have is the Hampshire Fire and Rescue report summary, that makes reference to the video... http://democracy.hants.gov.uk/documents/s36357/Report.pdf

There's a second and later report here: http://democracy.hants.gov.uk/documents/s38626/Report.pdf

[Gyro]

Yeah, that one seems to be just a lot of warm and fuzzy words about what we should all do to collaborate better in future.

[Cerebus]

Yeah, that one seems to be just a lot of warm and fuzzy words about what we should all do to collaborate better in future.

Agreed. I found it (because it was promised in the earlier report you linked) and posted the link before I'd read it. The earlier report is more substantial and the latter adds nothing to it.

[DrTune]

Pretty obvious what happened; a resistance fighter arrived from the future (naked, in a flash of blue lightning), and torched the place. Due to a subtle error in a Node.js parser, the warehouse robots were on course to achieve sentience in August 2020.
We'll thank them later.

[I wanted a rude username]

a heavyweight fire suppression system that fires if the power cut off is ineffective. It might well ruin all the stock

They could have used a CO₂ system (which wouldn't harm products), if they had designed for it and accepted that the "robot space" would be segregated from the "human space", with people only entering under certain specified situations and only after having received training. That would increase operational cost a bit, but obviously would have been cheaper in hindsight.

[Cerebus]

a heavyweight fire suppression system that fires if the power cut off is ineffective. It might well ruin all the stock

They could have used a CO₂ system (which wouldn't harm products), if they had designed for it and accepted that the "robot space" would be segregated from the "human space", with people only entering under certain specified situations and only after having received training. That would increase operational cost a bit, but obviously would have been cheaper in hindsight.

It was a 500,000 m³ building. I suspect it'd need an active fire suppressant (one of the old nasty sort that nicks electrons and works at relatively low concentrations) rather than a blanketing system like CO₂.  Those are even more inimical to life than a CO² flood. CO₂ has a density of ~2kg/m³ at 1 atm, the standard concentration of CO² you need to acheive in a flood system is 34%, so 500,000 m³ * 0.34 *2 => 340,000 kg of CO₂ required for this building. That's 340 tonnes of compressed (liquid) CO² taking up around 300 m³ (a cube 6.7m on each side) plus the size of the pressure vessels around it. Obviously it would instead be hundreds and hundreds of individual cylinders, that was just an illustration of scale.  Basically, hugely impractical.

FM-200 works at 6.25% and is a 'clean' fire suppressant, and is people safe up to 9% concentration, and is chuffing expensive. You'd still need tonnes of the stuff.

So, on reflection I think safe, clean, sane fire suppressants are out and lots of water or aqueous fire fighting foam are in, and a huge mess is going to be made.

[coppice]

Why does anyone buy from Ocado? They seem to be the most expensive way to get your groceries.

[Domagoj T]

While the tech seems impressive, I don't understand how the robot takes individual items out of the basket? At one point the robot can be seen taking up entire basket up into itself, so what happens next?

[coppice]

While the tech seems impressive, I don't understand how the robot takes individual items out of the basket? At one point the robot can be seen taking up entire basket up into itself, so what happens next?

I think that's when an entire order has been accumulated in the basket, and its being taken off to dispatch.

[thm_w]

While the tech seems impressive, I don't understand how the robot takes individual items out of the basket? At one point the robot can be seen taking up entire basket up into itself, so what happens next?

Its either hand picked into plastic bags or taken to a packing station where its automated: https://internetofbusiness.com/ocado-robots-hands-packing-shopping/

I think this one is a fulfillment center (customer orders), but they can also do this process to restock retail outlets.
Every week you'd get an order from say "joes corner grocery", we need: 20x can pop, 50x cat food, 20x box rice, then you'd pack up those items and ship to their store via pallet or perhaps in these bins. I used to do this and everything was ultra low-tech, routing was entirely up to the picker, these robots would be much faster than me. Most of the time was spent moving from one end of the warehouse to the other.

[nosmoke]

This is one of SwissLog's "AutoStore" miniload systems, See a video here: and here:

SwisSlog seems to specialise in miniload systems, they have various different types, all moving and storing the small plastic totes.

The way these systems tend to work, is that the robot doesn't have anything to do with the contents of the tote, they just fetch a tote from the store and bring it to an operator, who picks items for a order from the tote, and the system takes it away again.

While very space efficient, the swisslog system shown, can only access a tote on the top of the stack - if you want one further down, then you have to "dig" for it, and put the ones on top, somewhere else, first. - Dig moves are "expensive" in any ASRS, so you try and store stock to avoid them.
 
The order picker has a conveyor in front of them carrying the boxes that they are placing items into, a screen tells them what item to take from the tote, and what box to put it into.
That is a so-called "Goods to Man" system, very common for small item picking.

Amazon recently bought Kiva systems that do similar system, where robots drive under sets of free standing shelves and take them to the order pickers.

Source: I used to work for one of SwissLog's competitors, they do large scale automated pallet stores, amongst other things.

[Domagoj T]

Oh, that makes much more sense.
Thanks

[m0]

Coming to a warehouse near you...
https://uk.reuters.com/article/us-ocado-coles-deal/ocado-teams-up-with-australias-coles-in-latest-partnership-deal-idUKKCN1R702B

Deals with Grocers such as Coles (Australia), Kroger (USA), Sobeys (Canada) and a few across Europe will see these warehouses all over the world!
list of clients here: https://www.ocadosolutions.com/

[schmitt trigger]

Nosmoke;
thanks for the insightful information.

The future has indeed arrived.

I am going to make a dumb question: In addition to unskilled laborers doing the few menial tasks that robots do not yet perform, what job opportunities exist in those warehouses?
Programmers?
Automation engineers?
Electronic Technicians?

Not for my benefit, but to provide advice to my grandsons.

[Doctorandus_P]

Are those red robos more fire proof than the ocado ones?
I like the idea of the ocado robots more, Each one takes up only a single space instead of 2.

I also do not understand why a vat of 300m3 CO2 would be "impractical". It is a very minor thing for an warehouse of this size.

But it does not tackle the hart of the problem.
Such a dense stack of flammable stuff as the warehouse itself should not be built.
Instead: Each robot runs on rails, and a set of rails is a few centimeters wide.
Those few centimeters are enough to add fireproof layers between each stack of goods.
In that case, if any fire breaks out, it will be contained in a single stack of containers, or at least spread very slowly which gives the firemen plenty of time to react.

The next step would be to make some kind of lid on each stack of boxes, which is opened by the robot as it dives on top of it (Some kind of extenable pin on the underside of each robot)
If the rest of the stack is air tight this will severely limit oxygen supply to a stack of burning goods.
The lid itself does not have to be air tight. It works with the same principle as the sigarette bins at the entrance of buildings.
Rising hot air of something burning prevents fresh air from going down the "chimney".

Also, the horror scenario of a burning robot driving around and setting the whole warehouse on fire is undoubtly a beginners error in this sort of warehouse and will be completely eliminated in the next version.

[Gyro]

^^^ Maybe you should email them with suggestions.  Ocado have just announced that they're rebuilding the Andover site - bigger! 

[Nauris]

Local supermarket co-op here build a big automated warehouse recently. It is from Witron, does not use robots but more traditional automation just lots and lots of it.

They have quite good video showing how it works from goods recieving to dispatch.

[MrMobodies]

I see they have some serial ports on them:

SAFETY OVERRIDE
COMMS

[Ed.Kloonk]

I wonder how much of all this could have been circumvented if only the shelf packers packed the shelves instead of smoking near the loading zone.

[Ed.Kloonk]

I see they have some serial ports on them:

SAFETY OVERRIDE
COMMS

Was wondering about that also.

[vk6zgo]

I know this is an old thread, but according to the BBC, the sprinkler system was  an award winning design.
In Oz, we have had plenty of warehouse fires, & in many cases, the old style "non-award winning" sprinkler system saved the building & equipment.
The stock was often not so lucky, but was usually saleable in some form or the other.

I have been a unbeliever in "award winning designs", ever since, many years ago, I ventured into a tool store to buy  a "centre punch".
They had a nice one, made in Britain, & recipient of the "Duke of Ediburgh's Export Award".
It was a thing of beauty, complete with a Teflon finger grip, a fancy blister pack, & so on.

The first time I used it, the tip broke off & flew across the room.

I still needed a centre punch, so I went to the local hardware store, where they had a box of Australian made ones on the shelf.
No fancy pack, no brand name, no award, no Teflon, no Duke, but it lasted me for years "rattling around" in the bottom of my toolbox.

More "on topic", even if a fault was not caused by a machine fault, life & packaging has its own little traps.

Even further in antiquity, I worked in a Electronics wholesaler.
The Electrical Department, two benches away, sold, along, with all the multiple delights suggested by their name, Eveready Lantern Batteries (the squarish ones with two spring contacts on top.).
They would from time to time get warranty returns of these batteries, test them, & dump the faulty ones in the "bin."

This place was a bit Dickensian, & the bin, shared by the "Radio' & Electrical Departments, consisted of a large "tea chest", into which all the detritus of daily work was deposited, including used wrapping paper & plastic, as well as the aforesaid "lantern batteries.

By pure luck, I noticed a small wisp of smoke coming from the bin.
Much panic, as we tipped it out, stomping upon the burning material, & finding two supposedly "dead" lantern batteries in an +ve to -ve, -ve to +ve "embrace", getting hotter & hotter by the second!

Such batteries are still stocked at many department stores.
Some have a substantial plastic "cap" covering each spring, but others rely on a much thinner & more brittle piece of plastic covering both springs.
I have seen that style, with the "piece of plastic" either missing, or cracked, revealing either or both spring contacts.

[jjmatthew95]

NOTE: This message has been deleted by the forum moderator Simon for being against the forum rules and/or at the discretion of the moderator as being in the best interests of the forum community and the nature of the thread.
If you believe this to be in error, please contact the moderator involved.
An optional additional explanation is:

[Stray Electron]

Looks like s few hundred lines of FORTRAN ought to cover it.

  Or one line in C.

[rstofer]

I think the Amazon warehouses are more sophisticated.  There is no physical constraint on the robot's path so they have some form of localization.  The robots don't do pick-and-pack but they do present the objects to the person doing that task.

[rstofer]

Why does anyone buy from Ocado? They seem to be the most expensive way to get your groceries.

It's amazing how a little pandemic will change your shopping habits.  We ALWAYS use curb side delivery.  The idea of mingling with other shoppers traversing the aisles is out of the question.   I suspect this form of shopping will become the new standard and automated pick and pack is a logical extension.

[Bud]

What would prevent them to pack and ship to you rotten potatoes and other poor quality food.  I want to pick my own, thank you. Are you going to sue them for unfresh Roman lattice ?

[mikeselectricstuff]

This one is vary neat, and potentially very cheap as the only infrastructure is rails on the shelves:

[coppice]

Why does anyone buy from Ocado? They seem to be the most expensive way to get your groceries.

It's amazing how a little pandemic will change your shopping habits.  We ALWAYS use curb side delivery.  The idea of mingling with other shoppers traversing the aisles is out of the question.   I suspect this form of shopping will become the new standard and automated pick and pack is a logical extension.

Every major supermarket chain in the UK offers a similar home delivery service to Ocado, but you get the food at the same price as in their stores. You miss out on special offers, cheap bakery stuff near the end of the day, etc, but that is all. If you are flexible about delivery times you don't need to pay extra for it. If you want a specific delivery time you do.

[Monkeh]

Why does anyone buy from Ocado? They seem to be the most expensive way to get your groceries.

It's amazing how a little pandemic will change your shopping habits.  We ALWAYS use curb side delivery.  The idea of mingling with other shoppers traversing the aisles is out of the question.   I suspect this form of shopping will become the new standard and automated pick and pack is a logical extension.

Every major supermarket chain in the UK offers a similar home delivery service to Ocado, but you get the food at the same price as in their stores. You miss out on special offers, cheap bakery stuff near the end of the day, etc, but that is all. If you are flexible about delivery times you don't need to pay extra for it. If you want a specific delivery time you do.

In the last year you're lucky to find a slot and pay what you have to.

[rstofer]

What would prevent them to pack and ship to you rotten potatoes and other poor quality food.  I want to pick my own, thank you. Are you going to sue them for unfresh Roman lattice ?

We don't buy a lot of fresh vegetables.  Who would buy something that other people have been snotting over all day long?  There are sick people all over the place!

If the vegetables are bagged then sure, we'll order some.  We have been doing this since last March and things have worked out fine.  Some items are in limited supply but we seem to get about 90% of what we order.  Not bad considering that grocery stores had to come up to speed with Internet ordering without much notice.

Paper products (paper towels, toilet paper) have been a problem.  I got a jump on the shortage and bought a few hundred rolls of TP and we have changed to folded paper towels instead of roll.  No big deal.

We try very hard to avoid actually shopping inside any store.  UPS comes every day with something from somewhere.  We buy literally everything online, delivered.