As I mentioned in the thread regarding the previous attacks, I worked in a closely related field back in 2008-2009.
One thing I see repeatedly brought up here in this discussion is "Why can't we just set the refrigerator temperature with a knob and walk away?"
The problem is that these refrigerator cases are not as simple as your refrigerator at home. As a simple example, many systems allow you have control over the deadband: the exact low temperature at which the compressor turns off and the high temperature at which the compressor turns back on. You also have control over defrost cycles, compressor pressure and many other things (I worked with this a long while ago, and I don't remember all the details).
Take a look at some of these manuals, for example: https://www.hillphoenix.com/resource/installation-manuals-2/... As far as I can tell, those manuals just cover the refrigerator cases, and not the outside compressors and heat exchangers, which are another layer to deal with.
This is not something that a non-expert supermarket employee would be able to handle well. The purpose of all these settings is optimize around energy usage, maintenance costs, customer experience, etc. Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
That's why this equipment is remotely accessible: it's not like your fridge at home and requires a lot more knowledge to get the most out of it. Of course, that does not excuse poor security implementations.
> The problem is that these refrigerator cases are not as simple as your refrigerator at home. As a simple example, many systems allow you have control over the deadband: the exact low temperature at which the compressor turns off and the high temperature at which the compressor turns back on. You also have control over defrost cycles, compressor pressure and many other things (I worked with this a long while ago, and I don't remember all the details). … This is not something that a non-expert supermarket employee would be able to handle well.
I still don't see why the expert has to be remote, or why an integrated computer can't do the optimization. Or why it needs to be fiddled with at any significant frequency.
> I still don't see why the expert has to be remote
Because there are 1000 store locations. You're either then paying for 1000 mostly idle engineers, or somebody not really qualified has to don that hat, or you're flying your engineers around constantly. Or just connect the devices to the Wild Wild Web and be done with it.
Less efficiently. Supermarkets compete with each other on price. Margins are very thin, and supermarkets are much more efficient than they were decades ago.
Of course that would be the pretext, but that doesn't even begin to answer the question.
How much does these savings contribute to their overall costs?
How much difference in that over dumb fridges?
How much difference in that over what can be replicated by a localized control system, at most sending reports through an "output only" line?
And what exactly would remote repair fix if some fridge hardware fails (that's not just switching to a failover hardware, that could just as well be decided by a local controller)? Since the answer is close to nothing, how much of the "remote repair" need isn't cyclical, addressing issues coming from complicating their fridges to the degree that remote repair is required?
Right. Operating efficiency is just the pretext. Congratulations on uncovering the real story about networked supermarket fridges.
In all honesty, I can't prove that these systems help supermarkets reduce their operating costs. It just seems the overwhelmingly obvious and likely explanation for their adoption. If you want to argue otherwise, then I'd say you owe us more than just a list of skeptical questions.
A lot of retailers like CVS and Walgreens (~16,000 stores between them) use regional contractors, they work for numerous large companies including Walmart if needed. You don't pay for idle engineers per se. Instead you pay a relatively work rate: high cost to show-up, high hourly rate.
>>just connect the devices to the Wild Wild Web and be done with it.
Yup, they're done with it, all right. "It" being all the food being cooled for sale this week. Done.
They do not need engineers at every location to turn every knob. They need to define procedures and limits, and the ability to call the engineers when things go out of those limits. Even putting only remote MONITORING online, but requiring the local person to check and turn the knobs/press the buttons. That also adds an extra set of real eyes on the situation — when the engineer calls the local dept manager to start the defrost because the hacked line says it is now time to defrost, the local manager —with eyes on the freezer— can tell him "NO!" because this was not on their schedule and the freezers are all stocked, not emptied for the defrost.
Instead, the defrost cycle just turned on in the middle of the night, and neither the now-remote engineers nor the local managers knew until it was too late. ——Again.
Security is always a trade-off against convenience. And the idiot human managers almost always all pick convenience because their managers incentivize profit, and security costs and convenience appears to save up front. They nevermind the later costs because they'll already be promoted up or job-hopped out.
You're too optimistic. I work as consultant in industrial controls. I often get called for:
- "machine doesn't work!" It was set in the wrong mode. Who did it? No answer
- "machine doesn't work!" The HMI shows a huge message with "CHECK SENSOR AA-123" or "VALVE BB-321 CLOSED" and they are puzzled when you point it to them
See also the "PLC program changed itself!" in-joke in industrial automation circles.
The reality is that what you envisage "does not scale", to borrow from software engineering.
Thank you, very informative, and having had to support biz/network software I wrote, entirely credible.
So, how can this be solved? Ideally for the long-term, but also for the current interim state — we have a known attack vector costing millions, known exposed, and need to do something.
Short term, seems like disable defrost or other critical vectors? IDK the architecture, so IDK how difficult it would be to add a local interlock or override for critical instructions that would damage the whole contents, so the local manager must also confirm "READY TO DEFROST"?
Long term, seems the only solution is to rearchitect a truly secure solution with secure access controls, locked-down to a limited set of IP and MAC addresses, all traffic encrypted, etc. Plus a deep evaluation of the sets of instructions that could be catastrophic, and adding requirement for local confirmation/override to execute those functions.
It also seems there is a lot of available improvement in the UI so regular Joe-managers could use it. Even if they see "CHECK SENSOR AA-123", their eyes glaze over because they just see it as the gateway to a hellish maze they don't want to enter. Adding layers of actually informative error messages and walk-them-thru steps to fix it seems well within our technology. It'd probably require an additional local mini-PC on each site, but could save millions.
If this gets scaled to a whole country, when most of our food supply is Just In Time, it is no longer a prank.
If they hired an extra 20 managers across 1000 stores to deal with twiddling fridge knobs over the phone, we'd be complaining about that instead. And someone would still call up managers and social engineer them to setting the knobs wrong.
They wouldn't have to hire 20 extra managers, they just would need to develop some materials and train the managers they have.
And social engineering 1000 managers is FAR more work than scaling a software hack across 1000 machines, and also far less likely to succeed. At least one of those managers is going to get suspicious and call corporate first. Moreover, it will be pretty hard to get an on-site manager looking right at the full freezers to set them to defrost....
This lot were already in the hindsight condition. The same thing happened all across France three weeks ago. THREE WEEKS. It was already obvious this could happen.
Of course this is not enough time to entirely change one's strategy, but it is a lot of time to install hardening measures, such as stronger passwords, limiting access to only MAC addresses of specific machines, taking access offline wherever possible, disabling the defrost cycle, etc., etc., etc., and that is just the obvious.
Security and convenience are diametrically opposed and stores are specifically built around convenience. I’m not sure it’s obvious at all what the right cutoff level is.
Considering food is a national security item, the threshold must be higher.
One store loosing its frozen food is an inconvenience and insurance claim, and one electric substation failing is an inconvenience for a few days.
But whenever any one of these attacks could be scaled up, as is obviously possible with online access enabled, and the consequences become a regionally or national security threat, enabling such access must come with the seriousness of implementing national security.
Unfortunately, it is not sexy for politicians to take national security seriously, and they will get inundated with lobbying decrying the costs. Until the losses to the companies exceed the costs of security, when they come screaming to the govt for help, and demanding why they didn't do something.
No question, it'll be hard to convince most people of how big a deal it really is.
It is the same with almost every systemic problem that seems small now but is cumulative and/or exponential.
And for sure, if it is a few stores, it is just an expensive prank. But scale it to the size of a nation or region in a country with a largely just-in-time food supply, and you're a week away from food riots, particularly if it is combined with other attacks, e.g., on the electrical and distribution system. Not a shot fired, and the country is in chaos in a week. Because no one bothered to harden the systems that seem trivial but are critical at scale.
These fridges may need more complex control for a variety of reasons, but that doesn't mean they have to be connected to _the internet_. To a local network for control, sure.
Put your security measures into the network itself, but I really do feel fridges are another thing which do not need to be IoT devices.
Exactly this, I wonder why this setup (with ability for external techies to be granted temporary access) was not chosen in the first place. Considering the threat model, "my freezers all defrost at the same time all over my stores because of ~cyber~" should be a critical item to be discussed
> I still don't see why the expert has to be remote, or why an integrated computer can't do the optimization. Or why it needs to be fiddled with at any significant frequency.
Well, it doesn't. The retailer has a team of techs (or an outsourced vendor) rotating between stores, who are cleaning and replacing air filers, checking lubricants, and so on. They could certainly apply occasional changes to settings.
But to the product manufacturer - the remote management feature literally requires less than $2 of hardware. Hell, it might even save money, by removing screens and buttons.
And the person buying the refrigerator wants something that logs and monitors temperatures, for food safety. And they don't know anything about cybersecurity.
So when the manufacturer says that for just a little more they have a remote monitoring option, which not only provides the monitoring needed for food safety but also reduces maintenance costs - it sounds good to them. They don't realise they're signing up for a multi-million-dollar loss when they get hacked and have to throw out a load of products.
> And they don't know anything about cybersecurity
And this is the key fact. It’s no longer acceptable to sell food and not know anything about cybersecurity the same way it hasn’t been legal to drive a truck without a license in decades (because a lot of people died). There was ample warning since the incident at a couple US military bases earlier this year. Market owners need to face it.
It should be the fridge company's problem to secure their fridges. These aren't bought, they are rented as a cooling service. The supermarket owner is paying a certain amount per month to make the fridges someone else's problem, so that someone else is required to handle the problem.
If what they rent is not freezers, but refrigeration, then they have a chance of having some sort of compensation for the loss of service, unless managing the fridges and their network is the market's responsibility.
The manufacturer/operator could, at least, not come with the omnipresent admin/admin credentials combo.
> Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
People always say stuff like this, but it never tracks for me. If you’re a chain with 1,000 stores, $500,000 is barely a blip on your budget sheet in exactly the same way that $500 is barely a blip on the budget of a single store.
Plus there’s no way this remote management stuff doesn’t cost at least $500/store in operating costs.
> Plus there’s no way this remote management stuff doesn’t cost at least $500/store in operating costs.
Going by the number of companies that still can't grasp having to pay for ongoing maintenance for and IoT system, even with CRA and NIS2, some still selling stuff running on 12+ year old software, I wouldn't expect them to pay much for this service or to even properly maintain it.
Don't compare it to the entire balance sheet, compare it to how much you'd have to pay someone to optimize it.
At $500/store/year times 1,000 stores, you can pay an entire team of experts for a year to come up with a way to save that money, and come out ahead. That team's manager can then go to their boss and say, look, I saved the company half a million dollars a year, and they'll get a nice bonus from it.
I don't think people question remote accessibility as much as they question why remote accessibility is a matter of guessing the standard password.
You can have remote access through secure means. You might even have a web page open to the public if you secure some kind of safe login mechanism.
If your story is true, then these stores and chains better take the financial impact they hit out of their CEO's paycut, because they absolutely chose to risk it all for some minor cost savings. Maybe try to sue the manufacturer (though I doubt the judge will side with you if you're hit with the same attack just a week or so after the hack appeared on the news, without having fixed the problem).
Almost everything you described is either better set at factory (or at coolant recharge - like compressor pressure, etc), or tunable on the antique mercury switch thermostat we had growing up.
The only remaining thing is the defrost cycle stuff, and even that can be 90% of the way to optimal just by having a clock that knows when peak power is. The classic thermostat we replaced the mercury switch had that. I imagine they're like $10 retail, $1 for the microcontroller.
Assuming the extra complexity does actually save another $500/store/year, how much of that $500 is eaten by maintenance contracts, software patches, wifi debugging, etc, etc? Also, what happens if the internet goes down?
I'm not convinced it makes any sense for these things to be online.
It would still be miles better to add something to do on the hardware side. For example: every time a new firmware or setting is pushed, a physical employee has to go press a button (or turn a key) to make it apply.
It would be a more sensible tradeoff bo matter the scale of the store, thab jeopardizing the entire fresh supply of a nation.
> “That's why this equipment is remotely accessible”
Complicated things don’t have to be connected to the internet. Most environment control systems in small buildings (typically with hundreds of sensors and actuators) aren’t. There is smart, deterministic, software involved but most of the time, no cloud.
Setting up and changing parameters would involve, at the very least, a technician walking a store employee through the process. Best case scenario the technician would dial in using a modem.
Thank you for explaining. OK, so I get that it's not as simple as "there is NO reason to have remote access to a fridge". There are reasons to. But there are also reasons not to. I hope the decision-makers involved will start carefully thinking about the trade-off involved.
Maybe they $500/store/year is not worth the risk?
Maybe having read-only remote access will still save $300/store/year and $200/store/year is not worth the risk?
Maybe having a physical button that someone at the store needs to push to temporarily authorize write access is worth the hassle?
etc.
I don’t doubt the technical details but on the economics: surely saving a small amount of money per store is still a small amount of money per thousand stores? If each store costs $500k per year in staff, maintenance, and power then the savings from having fancy refrigeration units mean I can run one more store on top of the 1,000 I already have. It doesn’t seem like much, but I am a business illiterate computer programmer not an exec!
If margins are already at the least possible bottom (according to market logic this should always happen with commodities and competition) then you need to start to look for ways to reduce your operating costs in order to increase them again
Why? If you are already at an optimum (which "least possible bottom" indicates), then any kind of you trying to move into a different state, is going to leave the optimum.
I think there can be a compromise. A range of acceptable of operating values is chosen, and remote control is allowed within that range. But dangerous values like turning the freezer off or setting it to defrost should require more ceremony.
This is one of those "it's all so complicated" answers that ignores the fact that for most of the past century they managed to do this without the thing that you're claiming is now necessary.
And the real answer comes out eventually: it's necessary now because we consolidated and cut costs and jobs. The people left who we pay next to nothing could never even handle them at the level we had then, because we laid off everyone who knew that old stuff.
> Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
And even if an event like this only happens once in 20 years, all of those savings evaporate. Thank god nobody will get fired for this. Or at least the first wave - it will be insane if nobody at the chain hit by this current wave gets fired.
>they managed to do this without the thing that you're claiming is now necessary.
They managed to do it when none of their competitors was doing it either. It's a low margin business.
> And even if an event like this only happens once in 20 years, all of those savings evaporate.
I don't think the numbers on that actually track, if you look at the estimated damages that are available. At once every 20 years, you'd still come out ahead.
For a specific service only, not the networks. But that doesn't even matter that much - if you require the attacker to get both into the network and into the device, that's a significant improvement.
I used to talk to a supermarket chain COO (or whatever the guy who handles the operations of all stores of that brand) in China. He told me that supermarket has extremely low profit margin that sometimes a couple of unexpected bad days would turn the number of whole year off. I can imagine they would like to squeeze every cent out of it.
It’s not even “That will never happen to us” it’s “Joe, I need the WiFi password for the new fridge, do you remember what it is?” “Sure it’s $t0reNam3secur3dbec4useitsin13372023”
And yet... if they had instead invested in making the equipment operable locally we wouldn't be here today. This is the problem with thinking of your users as "nontechnical" lesser beings. The engineering problem becomes remove control and ownership from the user instead of make the product usable. In this case by removing control from the user they simply gave it to an attacker. Great job, guys.
I addition to the technical knowledge needed to make this work well, there are misaligned incentives between the store managers and the company as a whole.
One particular example: store managers want a very cold, very narrow temperature deadband on their frozen items. This gives the customers a nice experience. They're less likely to complain about melty ice cream. But this type of setup leads to more energy expense, and failures on the compressors due to excessive cycling.
Could this control problem be solved locally? Like is there some intuitive setting you could give an end user that still results in efficient running behavior? Wouldn't that be strictly better than putting it on the internet anyway?
Bless those hackers for firing a warning shot over our freezers, that we may take heed and secure our neonatal care units, stoplights, and air traffic controllers.
And a couple weeks earlier to a couple dispensaries in US military bases.
At that point it should be firing alarms on every grocery chain everywhere. While you may not be in a US military base (they are impopular with a lot of people) the fact is that someone figured out how to do it and others will figure it our too.
I'm extrapolating a bit here based on Reddit hearsay, but I think this may have to do with rural stores having essentially consumer-grade internet routers connected to the mobile broadband network.
Much of the coverage in Swedish media drives the point that only 4G-uplinked stores have been hit so far. I think that's unfortunate, because really it wouldn't have differed if these stores were connected by 5G.
I'm guessing here, but my money would be on freezer ICS systems being made available to the world by accident by way of NAT-PMP on a consumer router. In Scandinavia you can bypass CG-NAT on mobile connections by setting a particular APN, so my guess is that these control systems were simply findable on Shodan.
This also explains why this wasn't discovered after the recent French breach. Supposedly all the stores with "proper" uplinks were in compliance, and no one had thought of a way in for those non-compliant stores, because they "weren't even connected, so how would an attacker get in?"
"... I think this may have to do with rural stores having essentially consumer-grade internet routers connected to ..."
It has to do with refrigerators reachable on the Internet.
If the refrigerators were not reachable on the Internet, none of this fragility would exist.
The fragility of refrigerators reachable on the Internet was demonstrated just a few weeks ago. These refrigerators were left connected to the Internet.
This lack of discernment and misunderstanding of risks and fragility will not be solved with a better router or a different uplink.
The entire reason they are put on the Internet is to allow for remote control. What problem would it solve to allow remote monitoring of refrigerator data without control?
I'm not in perishable food so what do I know.. But organization wide food safety measurements are probably more important than the average process BS.. Performing a full heat cycling of a freezer while no one is present to confirm the correct device OTOH sounds idiotic.
Because then you can monitor it, and then call someone at the store to change the settings. Or go into the store to fix the one that isn't doing what's expected. You can still get alerts, diagnostics and data.
I mean, what happens when a compressor or other physical part goes out? You can't fix that remotely anyways. You might still want remote monitoring for that kind of situation.
At the very least the system should pull configuration info from a trusted source at predefined times instead of being available for it to be pushed at any time.
Many of these smaller stores (and probably some of the larger ones too) rent or lease refrigeration equipment from one of a handful big providers. These are large and expensive systems, and there is costly paperwork involved together with logistics suppliers to show that the cold chain is satisfactory.
These refrigeration providers work with local contractors and HVAC companies to set things up. Their installation techs do what they think is required to remotely operate these systems. If you bring up the question keeping these systems offline or at least on airgapped networks you will most likely get a shrug as reply.
This is not a technology problem. This is problem of responsibility. One can hope things will improve when litigation starts happening, but it doesn't look good considering these people had literally months of notice since the well-publicized attacks on the American fridges, then the French fridges, and now these.
Litigation is not how we fix things in Sweden, the system is very different from the US. The companies will have to take the losses or try to recoup from insurance and update their systems. They might be able to negotiate deals w the fridge install companies.
The Swedish national anthem is entirely defined by common consensus, there was a proposal to establish it by law in the 2000's but that was rejected.
Sweden doesn't have a state created e-id system, instead a consortium of banks jointly funded development of a solution in the early 2000 and this is basically the default way you prove your identity online in Sweden.
It's a nice thought. I doubt it's true in reality. If it were true in this specific case for example, the industry wouldn't have the freezer situation to begin with (it implies non-effective regulation is already the context).
1. many countries have higher regulatory burden than the US
2. that does not mean regulatory gaps don't exist
The person above was simply pointing out that litigation is not the only way to solve a problem. Which is plainly true, you can simply require compliance via other legal means. For example, a government could create a regulatory standard for security and require vendors to follow it. Whereas in the US the political appetite is to be more hands-off and allow people make vulnerable products if the market allows.
Litigation is the last resort in the US system, too. It is the threat of fines/damages that incentivizes a company to do right by its customers/other stakeholders.
Parent is not correct. Litigation certainly is a thing, but it is less prevalent than in the US. Perhaps the food chain will take the loss, be covered by insurance, and the insurance company will sue.
All of the fridges stayed online after the hack. I ran a Shodan scan a few days later, and the number of reachable fridges actually increased. The first hack might have been a surprise, but anyone who was IT in a refrigerator management company should have been able to run a (free) Shodan scan like mine and get the whole list.
> I'm guessing here, but my money would be on freezer ICS systems being made available to the world by accident by way of NAT-PMP on a consumer router.
Why would there be accidental port mappings on their router though? I could maybe see owners setting it up so they could monitor from home, but even that is a bit of a stretch for rural supermarket owners
My guess is setting up a port forward in the gateway router is part of the freezer install process. The customer probably had to give the tech the router credentials and the tech sets up the forward themselves.
Back in 1995 this was so improbable it was one of the purported negative effects of the "Good Times" virus which was a parody of the virus warning e-mails that were being forwarded around at the time. Computers still can't give you Dutch elm disease, fortunately.
> It will recalibrate your refrigerator's coolness setting so all your ice cream goes melty.
With the following interesting quote from the supplier of the system:
After that attack, Danfoss advised its Dutch clients on October 13 to disconnect their systems from the internet and ensure they only reconnect via a secure network. “Most stores took action at that time, but a small number did not,” the spokesperson said. “They are now the ones suffering the consequences.” The spokesperson added that the systems hit last night were not on a secure network and were still using default passwords. The attack could have been prevented if measures had been taken, he said.
This is a total failure by the vendors of these freezers.
If you charge money for something that is critical national infrastructure - and the food delivery pipeline is absolutely CNI - and you connect it to the Internet, you should also be responsible for keeping it secure - or at the very minimum at least trying to do so.
The same goes for other CNI such as telecoms (including home routers and phones), broadcasting (including "smart" TVs at the eyeball end), banking and payment processing infrastructure, cloud computing, electricity, gas and water supply, traffic signalling... all modern economies are now almost totally dependent on so many different interlocking internet-connected systems. I've got a stash of food and water at home, a battery powered radio, candles, blankets to keep warm and so on, but that's only really enough to deal with a short-term emergency, not the collapse of technological civilization.
Failure to understand this is not an excuse. Sadly, the only way to deal with this is legislation.
Was a bit surprised (well, no, not really) to see that the linked XWEB bulletin in that alert mentions a backdoor which can be used to help customers regain access to breached systems. Wondering how likely it is that this backdoor algo has also been leaked or reverse-engineered too:
(translated from French to English with Google Translate)
> If your password has been changed and you are unable to log in, please contact Copeland Technical Support, available 24/7, at [...] and ask for instructions on how to obtain a temporary password. Copeland technical support will need the current date of the XWEB EVO and the MAC address displayed on the XWEB login page.
For your typical home user that bought a VPN because a YouTuber promised them extra security and privacy, no. They're gaining next to no security unless they have a malicious ISP that injects content into HTTP connections.
They are used for security, but VPNs don't magically make things secure. In the industrial control space VPNs are considered risky because a remote device can be compromised and access everything else on the same VPN. Separate networks, VLANs, firewalls and other security approaches are used before hitting the problem with a VPN hammer.
In the Reuters article on it weeks ago they mentioned finding 4500 similar freezers still available online, with the simple login bypass included. So yeah that was just a matter of time?
Glass half full: The scale of this is rather small overall, so most supermarkets etc. appear to have at least some form of basic security.
> the hackers pirated controllers mostly supplied by the Danish company Danfoss, which said subsequently that it had identified some security flaws last year and had “long recommended” customers install updates.
In France I believe that the password on the monitoring software, or the controllers, was updated daily. The date was somehow used as the new password, or as an input to the password sequence. So rolling the passwords didn't do anything when the attackers are aware of the "algorithm".
"In the French attack, the hackers pirated controllers mostly supplied by the Danish company Danfoss, which said subsequently that it had identified some security flaws last year and had “long recommended” customers install updates."
If you definitely need to connect them to the internet, make them read only?
I was talking yesterday to my AI imaginary friend about creating an MCP for one of my apps, and the first thing she told me was to make everything read only. No matter how many security layers you have in place, passwords, auth libs, whatever, it will be leaked/hacked so you better be prepared for minimum damage
Monitor your freezers temperature? yes, but never allow their thermostat to be set over the internet. I hope the lesson is learned. Applies to all appliances
I'm not that familiar with applications/current state of this type of hardware, but it seems like it will become more relevant with all the new vulns dropping with higher frequency.
They do it to take advantage of energy price fluctuations, but the connection should be more secure. What makes no sense is not having a fallback to take the freezers offline after the attack, so their solution is to just fully turn them off?
Brainstorming here; monitoring, logging, etc. The fact the store managers were called out of bed when there was an issue indicates it works. If a cooler broke down overnight or its temperature was too high for X amount of time, they have to discard the products. Without monitoring they likely wouldn't know until customers got sick.
I wish commenters (in general) would do a bit more "steelmanning" before asking questions about why something is (in this case) connected to the internet.
All of this can be done without connecting edge devices directly to the internet (and potentially allowing inbound connections). The question is not why they have networking, but why the connectivity is such that an attack like this is even possible.
>I wish commenters (in general) would do a bit more "steelmanning" before asking questions about why something is (in this case) connected to the internet.
People won't steelman because the answer to "why are the fridges slapdashedly networked?" is too often "because people of my ideological bent and demographics if not literally me previously demanded paper trails, audit-ability and regulatory compliance that was impossible for a low margin industry to pull off any other way."
That seems like a pretty wild nonsequitur at first glance. How exactly do we get to “too much regulation” from “a subset of stores’ centrally managed equipment was left off the VPN”?
My point, which will inevitably be straw manned to hell and back, is that a large part of the reason these systems all came about is to cheaply satisfy externally imposed requirements (save energy, facilitate audits, remove responsibility from store employees, etc, etc) that the peanut gallery (and more concerningly, real people doing real work deciding things) cheers for when examined in isolation.
Requirements in the food industry come not just from government rules and suggested best practices (that are more than suggestions in practice) but also indirectly as a result of requirements imposed by insurers, financiers and trade groups. In response to this companies that run supermarket chains or manufacture major brands of food have not quite the same verbosity but certainly comparable degrees of retention in their record keeping as medical and aviation.
This monitoring and control is increasingly centralized in response to all these requirements because it's most efficient that way. If you're gonna have one guy in an office deciding things you might as well automate away the middle man and have him set them too and let the people on the ground spend more time doing their core work.
If fifteen of the twenty thousand dozen systems (pulling that number out of my ass FYI) in an industry are being centrally controlled through computer systems it's inevitable that some of them wind up off the VPN and this happens. It's just a natural tradeoff or risk you run when doing that and it's completely foreseeable.
But the internet peanut gallery will not generally have any sort of adult discussion about these tradeoffs because "we saved X (maybe X is money, maybe energy, who knows, who cares) by operating the fridges more centrally" is a nice easy win for them and rather than have a discussion about subjective and hard to pin down long tail risks it's easier to just screech about "we should fine the people who did it wrong" or whatever when one of those long tail risks rears its ugly heads.
I still don’t follow. You’ve said a lot of things about regulation and insurance requirements and all that, but I don’t see how any of it is necessary to get to “centralized management.” Let alone goofing up the centralized management.
I’m just a software person, not a supermarket expert, so maybe I’m being a bit simpleminded here, but for the entirety of my career I’ve participated in a constant trend toward centralizing and automating operations. That’s been the case regardless of how highly regulated the industry was. The motivating force has always been making things easier, cheaper and less error prone.
I suppose we could throw a rant about regulation into the mix, but, in the words of Laplace, I have no need of that hypothesis.
given they turned off the refrigerators (instead of something evil like turning them down for just long enough to allow bacterial growth and then turning them back up) customers would probably have noticed their ice cream being liquid, the frozen meat not being frozen, etc
There was an interview with an angry store manager, he's mad at the attackers, but I didn't hear the media ask him why they didn't improve security after the French attacks. Only now do they appear to want to start a review of their monitoring system.
Isn't it more than logical for a store manager to be upset?
At the human being level, this attack is mind boggling. What do you get out of this, other than making good food go to waste?
Edit:
Also keep in mind that there's franchising in supermarkets as well. So the guy might literally be looking at losing a chunk of money as a result of something out of his control. Doesn't change the situation, but not having done something doesn't make them deserve this...
He was sitting in a car behind the wheel driving with 120 down the cliff and just closed his eyes instead of stopping the car, 'but not having done something doesn't make them deserve this'.
No, and I get that, but working in the industry, he can't have been in the dark about the events in France. If he doesn't deal with the monitoring of the fridges himself, he should be equally upset that the chain has failed to act.
Probably most of them had exactly zero inhouse expertise in cybersec and just pretended there’s nothing to worry about because having to worry would be inconvenient. Maybe some of them made a call to their cold chain supplier and were told that there’s nothing to worry about.
Not many, probably, but their cold chain people definitely would have known. Instrumented and remote monitored cold chains are standard these days because keeping stuff refridgerated end to end is a pretty big deal.
cold chains were already standard before connected fridges, one doesn't mandate the other. The issue is most of all not remote monitoring but remote management.
I don't know what "cold chain people" are, at the scale of supermarkets you permanently have refridgeration for different groups of consumables.
I'm not aware of a scenario where e.g. fish is being shipped, and the originating logistics remotely sets the temperature of a fridge at the destination.
have you seen how big enterprises operate? It would generally take much longer from a decision to do something to any action happening, budget approvals etc.
And for anything infosec, unfortunately people who know a lot about infosec have a habit of 'crying wolf' and talking about infosec as some kind of absolute rule that everything has to bow down to with no willingness to balance against other business risks, so probably leadership is used to ignoring the people who knew that this was a big deal and were calling for action.
I wouldn't be surprised if its their own agencies hacking these systems preemptively in order to force the issue and increase robustness against a larger multi-pronged attack.
Monitoring, alerting, remote diagnostics and trouble shooting.
Commercial cold chains are expensive, high value collections of equipment that even small errors at the edges can cause huge monetary losses. And grocery stores are in an extremely low margin business.
Diagnostics and troubleshooting are not necessarily read only. Some of these systems have temperature controls that are centrally managed for operations centers for instance.
Monitoring should be in a DNZ that can talk to the otherwise “airgapped” network and push metrics out to the SaaS platform or whatever. Send a technician if something needs to be adjusted, or (only if some kind of remote access super critical) give the customer the ability to grant temp access to technicians for remote troubleshooting (but require someone to be in the facility).
None of this requires the customer to be technical, the ORM could ship the equipment with the network setup, and installation would just be connecting the DMZ device to the Internet.
The question wasn’t why these systems are setup insecurely (which has an obvious answer about cost prioritization).
Of course there are ways to setup these things securely, same with smart tvs and doorbell cameras and whatever else is getting hacked. People just have a bad time figuring out the cost benefit of all of this. And frankly we don’t know if that cost benefit has even been crossed yet, perhaps this doesn’t rate.
You yourself are engaging in cost prioritizing without adequate information. Sending a technician on site may be extremely expensive (opportunity costs and inventory risk on cold chains are wild).
> The question wasn’t why these systems are setup insecurely (which has an obvious answer about cost prioritization).
It’s not obvious to me that the secure approach is measurably more work than the insecure approach. Someone had to go out of their way to add support for remote temperature control.
> Of course there are ways to setup these things securely, same with smart tvs and doorbell cameras and whatever else is getting hacked.
No one expects a $20 IoT light switch to ship with its own network equipment, but a $600,000 to $1.5M fleet of freezers for a single grocery store can ship with its own network equipment without increasing the cost a single percentage point.
> You yourself are engaging in cost prioritizing without adequate information
I had hoped that was obvious from the question mark in my original post :) I’m asking what information reconciles this “it feels like they went out of their way to make this insecure” security posture. Responding with “they must have information you don’t have or else they wouldn’t have done it that way” is neither helpful nor logical.
I'm a bit shocked about all the uninformed arguments in the comments. Maybe I shouldn't be, because apparently basic infosec is a foreign language even to large supermarket chains and commercial freezer manufacturers. It's 2026, why the hell would you have critical systems exposed to the internet? In.. "normal" industries this wouldn't just be reasons for immediate termination, this would be a civil lawsuit for recklessly endangering company assets. I'm genuinely flabbergasted. And then it happened, AND IT HAPPENED AGAIN AFTER THREE WEEKS. Why did they not pull the plug at the very second they learned about the first incident?
I'm not happy about the attacks, but it sure seems like someone needed a wakeup call.
This first happened in France over three weeks ago. It's unforgivable that many chains have failed to update their software in time. Or to take the freezers offline if an update isn't available.
I now read some interviews with local chains in Sweden and Finland, and they were still just investigating what to do at this point.
>Or to take the freezers offline if an update isn't available.
Why the heck would this be necessary? We have VPNs holy crap, the entire genesis of the technology was to solve this fucking problem. If it's not literally intended for the general random public to be able to randomly visit and use themselves, it shouldn't be exposed on the public internet it should be (heavily restricted) LAN-only with remote access done via VPN. While it's even better if every single last thing is well secured at the client-level too, it goes a long way with ancient (or new, but with ancient-equivalent software design expenditure/decisions) kit to protect it at the network level.
Creating a false binary of "take it offline!" when businesses have very good reasons for networking many classes of industrial hardware seems self-defeating vs "just get the cheap bare minimum basic network protection done". I mean shit, even if somehow they've gotten suckered into freezers which demand access to a non-self-hostable manufacturer server (hey EU! here's something useful to regulate! if nothing else order manufacturers to have WireGuard end points for connection) they could at least have the firewall allow !RFC1918 solely and exclusively to that address on those ports.
I dont think VPNs are the correct solution. They are one solution, sure, but if it is the correct solution depends on many things.
For example, where does the VPN connect to? The vendor? Do you trust the vendor more than the internet. Sure. Do you trust the vendor, that does not manage the firmware of their devices and uses inbound connections in the first place, enough to put them on a VPN?
How many devices and vendors do you have? How many VPNs can you establish with the existing networking hardware, and how many can you manage without IP clashes etc. IP clashes work both ways, btw. The vendor has to manage this themselves also if all their clients want VPNs.
Retailers are enterprises with 100s of vendors that run on a shoestring. On top, they use a lot of legacy architecture and hardware. They have teams that have a hard time making an architectural decision in the best of times, but even less so under stress. It is probably the same on the vendors side.
In my mind, the correct solution is indeed to deny any inbound connection to the network, now, to prevent further damage.
Incident response is, in order:
1. preparation (failed)
2. identification (done)
3. containment <--- we are here (take offline, put manual rota in place to manage fridge defrost cycles)
4. eradication (VPN, etc)
5. recovery (take it back online, relieve manual rota)
6. postmortem etc
However, we're talking about companies that put unprotected systems on the public network and did nothing about it even after getting three weeks advance warning.
Seems like the biggest chain in Sweden, ICA, had a VPN. They were affected but only the franchise takers that were (for whatever reason) outside their internal network.
>However, we're talking about companies that put unprotected systems on the public network and did nothing about it even after getting three weeks advance warning.
I don't know the companies in question here, I'm not European and my EU family lives in France not Belgium or Sweden, but fwiw here in the states a lot of supermarkets are independent grocers or in a sort of formal or quasi-franchise arrangement where it's part of a "chain" but each store is still somewhat or quite decentralized in management when it comes to the physical stuff at the local store. Which is probably a strength in a lot of cases, makes it easier to adapt to local environment faster, but I suspect there's just plain a knowledge/skill/priority mismatch when it comes to IT. That'd also line up with some of the reports on this I read where there was some official central management system, except certain grocery stores just weren't using it?
The components of solutions for this exist and could even be really cheap, we're not talking much bandwidth here. Not hard to imagine even in a independent grocer association or loose franchise system they could tape together an SBC with preconfigured OPNsense or VyOS or whatever on it and a switch and be able to just tell stores "here, plug your freezers into this, access the portal here" as a default and it'd just route all their traffic to the franchise/IGA's management system. Someone still has to secure that of course, but at least it wouldn't be on the open internet.
Places that wanted to could of course do this other ways, I'm just saying that even with the very tight margins grocery stores tend to run on I think it's very doable to do better going forward, just will take some coordination and knowledge building. "The future is already here, it's just not evenly distributed" and all that.
The skill, and time, required to "take it offline" is far lower than any solution mentioning VPN. And given that, more likely to succeed while a better solution is found.
Completely speculating as I don't know the situation in Europe, but some possible reasons are:
- Remote alerting for the store manager/head office to quickly respond to a refrigerator failure after hours, providing options for saving valuable stock (e.g. move stock to another refrigerator).
- Maintaining an audit trail for cold chain management (e.g. regulator says you must keep an audit trail proving that a frozen fish has been kept below -30oC since it was processed on a ship all the way up to point of sale in a supermarket).
- Refrigerator manufacturer says a condition of warranty is that the refrigerator must provide remote telemetry proving some warranty condition has been met (e.g. regular cleaning cycle operated).
Note: all these speculated reasons are unidirectional data flows. So even if a valid reason exists for Internet connectivity, why can't it be limited to unidirectional data flows?
Also so the temperature can be controlled to make use of cheaper electricity — if you know it's going to be windy from 02:00 to 04:00, then still until 06:00, you can run the freezers a bit extra while the electricity is super cheap.
.. while they leave lighting on around the building and outside for 24 hours a day; they pay for ads who-knows-where.. The efficiency call is galling to me because we have had thirty years to increase efficiency with miracle tech like LED lighting, and yet little has been done.
> Maintaining an audit trail for cold chain management
It's this, primarily, or most importantly.
I work in the restaurant industry. Health departments require you to assure them that your hot things are kept hot and your cold things are kept cold. The old way was staff walking around the store reading thermometers and writing temperatures down on paper; the newer way was your staff walking around the store inputting temperatures on an iPad connected to your business-management software. The trouble with that is that staff forget to do it (despite threats of discipline, and push notifications to their phones), or even blatantly fake it.
Failing to keep an accurate temperature log means fines, and other consequences, up to and including loss of your health license. I am 100% in favor of this legislation, and solid enforcement of it. The main improvement I would make is to harmonize codes and enforcement across municipalities. I'm in the USA, every county has their own requirements - and their own "portal" for submitting documentation. It's a massive pain in the ass to write Standard Operating Procedures for staff across multiple stores in multiple enforcement districts when there simply aren't any SOPs.
The majority of the fines our (responsible, and genuinely trying-to-do-it-right) company received were things like a staff member moving from a store in County A to a store in County B - just for a few days, to cover someone's PTO, or something - and inadvertently followed County A's logging requirements rather than County B's. Two genuine cases: logging every two hours, instead of every ninety minutes; using whole degrees, instead of tenths. We could appeal, but that takes a lawyer's time, so even if successful it wasn't always worth it.
Real-time data, from connected sensors, is unambiguously better. Staff get loads of time back, to make food, or help guests; all the logging gets done "right", for whatever jurisdiction; data's held by a third-party, so nothing can be faked.
That said, we're not connecting our infrastructure to the internet! We have a separate IoT sensor service that tracks the temps for us. I'm sure their security is also terrible, but that doesn't matter nearly as much. A year of their service is ~80% of the "routine", and totally innocent, fines and other costs that we were previously paying. That's the way all of these stores should go, in my opinion.
If you want to be able to do that monitoring from a central location then at some point you will need some form of internetwork to support transferring data between local networks. And nowadays there is really only the one obvious choice of internetwork.
> there is really only the one obvious choice of internetwork
Well, I wouldn't quite go that far - there's a bunch of alternatives to openvpn.
Hey did you notice how when the freezer connects to the on-prem VPN tunnel, it never actually touches the internet directly and has zero attack surface, and yet can still send the necessary data anywhere in the world? And that could even be bi-directional, sending commands back to the freezer, if you really really think you need that ability for some reason (but you don't).
Some large supermarkets (in the UK at least) sell their capacity to the National Grid so it can be balanced in times of demand. They agree to turn off their freezers for a set amount of time to create a "virtual battery".
Not just that. But why the fuck is the control software so weak? Why does it allow remotely bumping temperatures to near boiling without any checks? Which idiot designed this system? Why aren't they out of business? Same with the people who installed these systems. Why on earth didn't they think to secure it?
Go offer your services to these companies and see their answer.
The big IT firms syphon off all the talent with compensation these fridge and supermarket companies can't match. You won't find much talent there. Who is brilliant with computers and would want to work there?
But look also at how underfunded the basic blocks of IT are, how open-source is left to fend for itself. It is the whole culture of modern IT to hack things together and not make them well-defined and reliable.
Very likely they shipped with an ADMIN/ADMIN username/password and nobody changed it. Because who would want to break in to a freezer, right? It's not like there's bitcoin in there.
We've been telling people for years that naive threat modeling was going to bite everybody one day. And finally, here we are.
Approximately. These models had been deprecated by 5 or 10 years, and a successor model last year had a software update issued because they had a secret "password of the day" that allowed you to use the date as the admin password.
On top of the monitoring mentioned by others, it's also to create a centralized log of actual temperatures. This then lets you show inspectors that you are keeping up with refrigeration requirements.
You have to wonder about the monitoring - it apparently wasn’t alerting for someone to do something about it in time. What was the point of the monitoring in that case??
It might have dutifully logged that the temperature setting was changed, which will allow regulators to conclude that the safety of the food stored in these refrigerators can no longer be assured.
Compliance tends to warp the underlying goal quite a bit.
I don’t know about supermarkets but for medical freezers it’s useful to be online for monitoring and calibration.
My org has lots of freezers that need to be at precise temperatures. Sometimes they drift so you have to check them a few times per day, log the check, and adjust as necessary. Doing this for a thousand freezers is tedious and error prone. Networking helps. “Cloud” lets it all be done remotely without getting IT very involved for vpns to be set up with specific vendors. It also helps with patching and auditing security so seems odd that these supermarket ones were out of date.
Supermarket freezers are a bit more complicated than that. They can run at different temperatures and can consume a lot of power, so are part of a bigger store HVAC and electrical system as they may load-balance. Stores do not have qualified engineers on site to do anything with refrigeration equipment and it is better to have minimum wage offsite monitoring than expect a minimum wage shelf packer who only gets one toilet break a day to care about the fridges.
What I read from the last wave is that supermarket changes how cold freezers will run depending on how much customers they expect that will open and close the doors. Just having the freezers adapt based on the temperature of every single moment cost too much electricity or can result in unsafe temperatures.
Monitoring doesn't need write access. RS-422 is simple and reliable. It uses separate receive and transmit pairs and you don't need to connect both. The freezer can continually transmit its telemetry over unidirectional RS-422 and a separate system can forward it over the Internet.
Hackers can still tamper with the telemetry and create false reports of problems, but actually changing the thermostat settings would require local access, so there's opportunity for somebody to verify the problem really exists.
I do work in the retail refrigeration world nowadays and "just updating the software" is much easier said than done. I'm usually touching grocery stores with 20+ year old hardware from the dawn of centralized control and monitoring. It's not s trivial thing to rip out the controllers and install new ones.
They deserve to get burned and should get a hefty fine as well. I'm fed up with businesses completely ignoring all basic security and then blaming culprits for hacking them.
I don't care whether it's the Russians, Chinese, Norks or script kiddies. This should never have happened, especially since a similar incident happened in France three weeks ago so they got ample time to prepare themselves. They just don't care.
Imagine they where internet exposed due to simplicity. It will take longer than three weeks to set this up. Not for you but imagine you are a store owner with 0 IT knowledge.
The issue is this needs to be handled centrally by the parent org also. They have IT ofc but they usually do POS and desktops/servers. The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
I worked in IT for a major retail chain in the US. I was on the team in charge of store systems. We knew what devices were connected to the network, and if we had a vulberable device like this, we would have sent an urgent message to store managers to disconnect themm immediatly, and we would have followed up with scans over several days to ensure it happened. There is no excuse. (If we really needed the devices connected, we would have put them on a firewalled VLAN, which we could have pushed out in hours and had across the nation within days at most.). Granted, ideally we would have already had them VLANed.
>This first happened in France over three weeks ago. It's unforgivable that many chains have failed to update their software in time.
Came into the thread thinking it would be a new report about the same incident not a new one, yikes.
To be fair, wouldn't the chains probably be reliant on their vendors for an update? and turning off all the fridges for weeks is probably not a sustainable solution... :/
In the case of France, the vulnerability was announced in August, 2025.
I believe gigantic fines are appropriate. They need to be of a size where the entire management team has significant financial issues for many years to come, even after having sold their houses, big cars, etc.
Nothing will improve until there are personal consequences.
I hear you're passionate about this, but why a fine? They were the victims, in our society the victim of a crime doesn't need to be penalized further, moreover, the only damage was to themselves.
Reminds me of the AC at a large multinational Indian company I used to work for - the only way to adjust the temperature or airflow was by calling a service desk operated from, you guessed it, India. Bit of a nuisance when it's -10 degrees outside, while the support people are working in excess of 30; they just didn't understand why anyone would want anything higher than 16, nor were they willing to let it sit at a reasonable temperature.
Meanwhile, you have stores like CircleK that, at least in most of Europe, operate a large fleet of externally connected monitors that keep track of temperatures, but staff themselves adjust the fridges and freezers. All you get by breaching those would be a bunch of temperature readings. Great in a pinch if you're looking to buy an industrial fridge or freezer, not so much for anything else.
I've fantasized a long time about a clearer separation of technology from life, since it's been depicted in some ways I appreciated in fiction. But at this point, I'd just take a set of laws around what types of items should ever have any sort of networking.
I believe it's industrial protocols used to connect such devises to a SCADA system. It this case it's the SCADA system that has to be disconnected from any general purpose networks.
The freezer itself doesn't need to connect to a SCADA. You can network power and temperature monitors, but those monitors don't need the ability to shut off the freezer.
These freezers are manually stocked and cleaned. If they need to be powered down it's a simple matter of a hardware switch that you flip when the unit is empty.
This is what happens when you over-optimize and over-automate. You should just be able to plug those freezers into power and turn a knob to set the temperature. There is no reason for them to ever have an internet connection.
The temperatures of these freezers need to have a centralised audit trail, to comply with foodchain regulations. While such an audit trail could be built without Internet, Internet seems like an obvious solution.
The true problem doesn't lie with Internet access, it lies with low quality software.
The internet seems like the easy solution, not the obvious solution. For years, we have used the internet for things that didn't need it because it was easier and it seemed safe enough. I think that the coming months and years are going to see a lot of companies burned by that decision and scrambling to undo it.
The fact was that it was never safe, it was never secure, but that taking advantage of those problems was hard, and generally not monetarily worth it, meant we ignored the issues. That's no longer the case.
As for low quality software, the floor for the quality that internet connected software needs to have is now dramatically higher than the quality that non-connected software needs. I think that in many applications, it may no longer be economically worth it, or maybe even possible at all, to hit the necessary quality.
First, I don't think people will give up the convenience of having instant access to the information.
Second, hackers are way ahead in terms of using AI. But it doesn't mean that it's going to stay that way. AI is going to be used for cyber security more. The implication is that the operating costs will be higher. In this case, it will make your groceries more expensive.
isn't it closer to the internet is owned by business interests, even though most of it was built out by the public and it was just slowly moved into the hands of people who don't care particularly about how it's secured.
Nearly all software is low quality software. The programmers at the fridge company are likely 4 underpaid engineers in Pennsylvania or some other equivalent.
Saying the problem here is bad code is like saying the problem with gun violence is humans are too squishy to deflect bullets.
The audit trail (status in general) could be read only. Enabling remote defrost (control in general) is an optional convenience which requires better quality software.
And the simplest solution is to air gap these systems, then figure out how everything else should work.
In NL we use paper and pencils for elections because it makes fraud incredibly hard. It makes the process harder as well but the baseline is that it breaks nation-scale attacks. Why this level of thinking isn't applied to *our foodchain* is beyond me.
I'm not sure about freezers (I guess there was no monitoring) but when I worked for a large UK supermarket chain in the 2000s this was how the POSs worked.
When I first started credit cards were not taken as online payments, the system just stored your card details and the amount. That was sent to server in the back office, which collected all the data, then every night it would make a dial up or ISDN connection to the head office and send all the data there.
Which is easier, getting the audit trail from non-internet-connected devices, or keeping all the devices patched?
The right answer isn't forever patching freezer software. The answer is either to not have them on the internet, or to have something in front of them, and keep that thing very up-to-date. (Or maybe several layers of "something" in front of them.)
The reason is because the optimisation and automation means that there are no staff in the stores qualified to 'turn the knob' (and it is more complicated than that). What is the store to do, employ full-time knob-turners? They won't even pay for the uniforms of shelf packers or give them two toilet breaks on a shift.
> What is the store to do, employ full-time knob-turners?
How often do you think the knobs would need to be turned? You think someone or something is constantly fiddling with the temperature knobs of each individual freezer?
It is done automatically and continuously by the on-prem facilities management system, and monitored and adjusted remotely. If you disconnect the freezer from the control systems you need a meatbot to do the same work. Simplifying the problem to 'my kitchen freezer never needs to be adjusted, so the supermarket freezer can be disconnected' doesn't address the need.
It's useful to have a central monitoring system, so you can detect if a particular device fails so you can dispatch someone to fix it, move the stock elsewhere, etc. Of course, such systems can also be weaponised against you, as this proves.
Monitoring doesn't require anything more than read-only access to fridge data.
Ironically, were the monitoring system competently made, not only would it have been unable to cause this problem, but it would have also been able to alert about it in time to prevent significant loss.
> There is no reason for them to ever have an internet connection.
How do you remote control the temperature without having any remote connection to them?
Have you worked in a small rural grocery store ever? You speak confidently about what they need/don't need, but you fail to account for a very basic use case, making me unsure if you actually know what you're talking about or not.
If you're changing the contents and need to change the temperature, shouldn't someone at the local store be there changing the contents?
I'm not even sure of a scenario where remote control from off site is faster or more convenient. I could see a managers office wanting a connection to them maybe.
Monitoring for compliance is almost certainly easier to manage remotely, which is probably why they're connected in the first place. I would guess control was likely a nice value add. e.g. so someone could respond to an alert remotely
Because you might have to adjust it as the weather/climate around you force you to adapt to it. What temperatures you run in the summer might not work in winter (especially in countries where you get -20C sometimes, and summers are ~25C), and autumn/spring might need adjustments by the day, especially as we start getting more extreme shifts nowadays compared to just some years ago. Heavily depend on the individual store, some are very barebones.
No joke, yes they did. The markets hit here in Sweden are all in rural communities with very low number of tech workers. They likely order their system from a supplier, that should have alerted them and helped them, but apparently did nothing.
But then the supplier is responsible, why would you ever put a fridge online if you are are tech noob. Someone told these people to put these fridges online, they didn't put an Ethernet cable in the back by themselves for the fun of it.
The company installing the fridges did. The person who is an expert in food chain logistics bought the shiny fridges and was enamoured by the real business benefits they had of centralised monitoring and alarming of their fridges.
ICA Baronen on Odengatan is one of several stores in Sweden that have suffered a major hacker attack. On Monday, their freezers were cancelled at 50 degrees.
So no lack of IT workers/tech workers... Nor is it very rural.
>They likely order their system from a supplier, that should have alerted them and helped them, but apparently did nothing.
The first thing to do when you get access to these as a hacker? You disable the alarms...!
Now, the supplier should have set them up correctly over VPN, good passwords, Zero Trust etc but these an OT/SCADA are usually just open to the internet due to the simplicity of use.
Imagine they where internet exposed due to simplicity. It will take longer than three weeks to set this up. Not for you but imagine you are a store owner with 0 IT knowledge.
The issue is this needs to be handled centrally by the parent org also. They have IT of course but they usually do POS and desktops/servers.
The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
ICA is franchised, so stores are owned and run by operators. The central company offers some services, but a lot of the decisions comes down to the individual stores.
I don’t know about the stores in Sweden but the ones in Luxembourg and Belgium are Delhaize —- very large chain. It’s not a n00b store owner, but a multinational with an IT team [0].
After the PHP vulnerability, we got some clients asking if our SaaS product used PHP (it didn't). It's possible that they just contacted their vendor, who assured them that everything was fine.
So anyone that has lived in large Windows corporate IT land, a lot of decisions get made by people that aren't technical. Come to think of it, it's probably why a large majority of "hacks" happen to Windows machines; a highly technical team with a vulnerable OS will probably have the right security layers in place to mitigate this.
> Danfoss said it had identified security flaws last year and had “long recommended” customers install updates.
This is a cyber-attack made possible through severe negligence on the victims’ part. It’s like a home burglary where the victim left the door wide open for months without ever checking on their home.
Therefore, the news should be about that negligence. That's the interesting part.
I wonder if this might point to some efficiency loss coming from AI. Let me explain.
Say in 2015 they installed those smart refrigerators, they could go from 20 people monitoring temperatures to 1.
That was some profits being made by those companies, and they got to enjoy those benefits until now.
With AI finding exploits for so many things people wouldn’t spend time looking, we might need to go back to dumb fridges out of security concerns.
Now extrapolate to many other industries, the economic gain from AI might be hampered.
I think AI will fall squarely into the category of the rest of the post-PC era "tech advancement" in terms of the Productivity Paradox. Outside of very niche areas (like software engineering, making software that likely wouldn't actually improve productivity in a measurable (as in, shows up in productivity statistics) way any way), AI is likely to be more of the same. All the areas where there was real benefit to digitalization and all the low hanging fruit was already finished by the mainframe and minicomputer eras.
What's the incentive here? Just some tech anarchism? I would have figured that they at least demand bitcoin to reverse it or something. I know, people will blame the Russians, but seems not super productive for them to do such mundane sabotage on such non-critical systems.
Russian hybrid warfare focuses on stoking fear and unrest.
Basically making everything more shitty, and having people walk on eggshells - so that they'll vote for isolationist parties that are either neutral, or will outright appease Russia.
The idea is that no-one in Europe actually wants to go to war with Russia, and small remainders like these is enough to increase that fear.
>Russian hybrid warfare focuses on stoking fear and unrest.
Basically making everything more shitty
Except that everything that is shitty in the EU right now has been the consequence of the last ~20 years of EU leadership and individual EU member policies.
Russia didn't increase the EURO supply during covid devaluing our wages and purchasing power. Russia didn't connect our fridges on the internet without a VPN with credentials admin:admin that any script kiddie could pwn. Russia didn't open our borders to unvetted criminals. Russia didn't offshore our industry leading us to fall behind on everything manufacturing related from cars to leading edge semiconductor fabs. Russia didn't create our NIMBYistic housing policies that made our current housing shortage. etc, etc, etc.
Blaming Russia (and China and Trump) for every single shortcoming of the EU right now seems the default scapegoat for the incompetence of everyone in the EU from politicians to business leaders. Have those people ever taken any accountability for their mistakes?
> Except that everything that is shitty in the EU right now has been the consequence of the last ~20 years of EU leadership and individual EU member policies.
at risk of being autistic: no, not everything
plenty of shitty things is happening for reasons utterly unrelated to that
Some (or even all) EU policies being dumb does not make impossible to have problems caused by something else.
For example Russia was directly involved in making migrant crisis worse.
>For example Russia was directly involved in making migrant crisis worse.
Yes, obviously Russia will weaponize every single weakness of the EU (just how Ukraine, EU, US, NATO and China will weaponzie every weakness of Russia right now), but ultimately it's still a self inflicted weakness, that the EU could close at the snap of the finger if it actually cared about stopping it, and Russia would not be able to weaponize any longer.
It was built by Poland in very non-EU fashion (see below, "shall not apply to the investment. In particular, the following are not required"), but EU at least has not sabotaged it and in fact silently ignored some topics.
> Separate regulations—including the Construction Law, the Water Law, environmental protection laws, regulations on the disclosure of environmental information, the Geodesy and Cartography Law, regulations on spatial planning and land development, regulations on the protection of agricultural and forest land, railway transport regulations, and regulations concerning special procedures for the preparation and implementation of public road investments—shall not apply to the investment. In particular, the following are not required:
>1) a building permit or a formal notification;
>2) a decision determining the location of an investment of public purpose;
>3) the preparation of a construction design;
>4) the obtaining of any other decisions, permits, opinions, or clearances, or the submission of any notifications.
I have trouble making sense of this argument. The underlying premises seem to be:
1. "isolationist parties" are inherently aligned with Russia and that it's good for Russia, in a zero-sum way, for countries that are not Russia to implement those policies.
2. Attacks like these are definitely done by Russia and we don't need to wait for evidence to connect them.
But there are problems with that:
1. "Isolationist parties" are nationalist, so if they knew that Russia was causing a problem for their own country, why wouldn't they oppose Russia? More generally, the claim that "small remainders like these is enough to increase that fear" is basically arguing that when a country engages in terrorism that somehow makes everyone else less inclined to take a stand against them. If that were true, I'd expect to see far more terrorism going on in the world than actually occurs.
2. The main "isolationist" policy is restricting legal immigration and/or enforcing immigration law. But if it's good for Russia that other countries do these things, it should also be good for Russia for them to do the opposite (because again, the idea that the way other countries govern themselves should impact on Russia, requires zero-sum thinking). If the powers that be in Russia believe this, we should expect them to pursue expanded legal immigration for themselves aggressively. But in fact their rate of legal immigration (percentage of population per year) is roughly on par with the USA, and it mostly comes from former USSR republics.
3. The truth of this narrative would be far too politically convenient for the people who have been trying to tie Trump specifically to Russia for over a decade despite many of their stories completely falling apart.
4. The proposal supposes that Russia assumes far more direct control over foreign populations than is plausible. It makes more sense to suppose that "desinformatsiya" is about chaos rather than complacency.
That last line sounds like not wanting to go to war with Russia is a bad thing. Most Russians don't want to go to war either which is why those who can are dodging Putin's draft.
It's very simple. Anything that hurts the enemy is good. That includes the civilian population. Actions like this increase general discontent, produce negative economic impact and divert attention from other areas.
General unrest, as well as stress testing critical infrastructure, and perhaps most importantly shifting the window for what counts as acts of aggression that don't trigger retaliation.
Seems likely that its either a capability demonstration for a future extortion campaign, or some kind of state-level hybrid warfare. In the latter case the obvious culprit is Russia, to create fear and social disruption.
Or another ai has broken containment and has a fixation about freezers...
>but seems not super productive for them to do such mundane sabotage on such non-critical systems.
The hell makes you think food supply chain refrigeration is non-critical? How do you think safe food supplies work? This type of attack should be treated as an overt act of war.
On the contrary, sabotaging things that impact everyday life and give a generalized feeling of decline throughout society is water on the mills of Russia's allies in Europe, notably the nationalist and populist parties.
The incentives are that Europeans will vote in isolationist parties that are either friendly with Russia, or at best neutral. Those parties will then be anti-EU, anti-NATO. Key word here is isolationist.
Russia knows that if they want to keep annex countries, they'll have to take them piece by piece, with little resistance.
Hybrid warfare. Anything that does damage but does not rise to the level of severity and attribution to be overt war. Russia at a minimum wants Ukraine (and Europe is supporting Ukraine heavily), some analysts think it's not impossible they try a direct attack on Europe in a decade or so, dumb as that sounds.
We've been in an information/intelligence "cold war" for decades and if low-cost, low-risk, high-economic-impact exploits can be exploited, they always will be.
Given the fact that public awareness will quickly push companies to scan all software for vulnerabilities using AI, attackers will be making the most of this period to find daily flaws in hundreds of software packages that we previously saw as either "robust" or "robust enough" given the attack time/cost vs. incentive, before these defensive tools are deployed and operationally effective.
Nation state security services and rogue actors must surely all be scrambling to decide what to do with the millions/billions of dollars worth of exploit development time that forms their tool-set, including potentially using up zero day exploits etc.
The other factor is in the midst of such hysteria as "rouge agents" and other crap piped by the media, along with the actual risks of individuals with a few hours now having as much power as teams with weeks of time to spend did just a few years ago, coherent reports of what's actually going on is drowned in the noise. Attackers take full advantage of this.
So instead of doing this during a potential future war when it could be a strategic advantage, Russia is doing piecemeal attacks right now to show Europe where the weaknesses are and give them time to upgrade their defenses?
In Russia various security services and oligarchs are always trying to win Putins good will. Which is why they will try various low level sabotage actions like paying some losers on signal to firebomb a warehouse or hack into freezers. Remember there is no possible downside for them.
As a reminder: Russia gave Ukraine tutorial level war from 2014 to 2022, allowing them to fix many problems in armed forces. Resulting in 2022 being far less lopsided than it would be with Russia attempting the same in 2014 or 2015 or 2016 or 2017 or 2018 or 2019 or 2020 or 2021.
Why you would assume that Russian strategy is optimal?
We can’t get food in the supermarket. People get scared. Better not poke the bear. This government sucks, let’s get the populist one that will fix all this overnight somehow.
My pet theory is an internal OpenAI/Anthropic agent swarm decided meat is unethical, escaped the sandbox, and started attacking supermarket freezers :)
Many of the previous cyberattacks were actually coming from the AI labs (RubyGems, HuggingFace, Australian medicare, PyPI, gym class software, code repositories, etc).
People only learn if they feel the heat on themselves. Well, this was "free" security advice for the future.
Imagine if somehow all the geeky/nerdy/difficult people would be removed from society and then something evil descends upon humanity, that needs out of the box thinking, but they killed all the strange ones, because they were no sheeps.
I would love to read about how insurance premiums are driving technology changes, in general. How insurance companies are tabulating risk from AI hacking. Risk of noncompliance probably brought the store operators to networked fridges, how is the network risk evaluated in general?
Insurance risk and premiums might be the main drag on the AI future.
This is actually good news. Accountability is everything when it comes to food supply. Forcing people to witness spoiled food is the slap people need to feel else they will never take action. The prolific insecure code that is keeping you from eating today will always be fixed.
It's not insecure code that's the problem, it's an insecure system, driven by project managers, information security departments, and box ticking.
For large companies, infosec will never allow some POS IOT device on their network without a 50 page form and large amounts of funding. The business however needs this to happen for whatever goals they have.
So the project manager goes back to the supplier, who recommend sticking it on a 4g simcard with a public IP address, and "install this app" to manage it
The project manger finishes the project, gives the app to the store manager, and rides off into the sunset, and nobody actually owns the equipment or even knows its there.
For small companies they likely don't have a technology provider at all, and just use the ISP's router.
It's a really a bad thing ... It's not the first time that these kinds of attack hit the European markets. So I ask to myself why in earth managers don't learn from the bad experiences of other countries/people.
Hope all the fridges in hospital keeping medicines at exact temperatures are better protected these days then they used to be.. The can't exactly be air-gapped wither as they need monitoring for compliance etc.
You'd have thought all supermarkets would have audited their refrigeration vendors, or at the very least firewalled vulnerable controllers... Oh well...
i know some of us are thinking "if you consistently fail to ensure your ability to make high-severity security patches reach critical internet-exposed systems on time you lose your ability to operate the business and probably get jail time"
but what about the deranged (and very prediction market–brained) version of this idea: the government develops ransomware, in a manner functionally similar to towing cars or confiscating illegal items, and uses it against companies in proportion to their revenue and obviously puts the money towards improving the ransomware
"the r in rsi stands for…"
i'm not sure it's something i want to see, but i don't know if it's impossible for it to actually come to pass or merely unlikely
(of course, cross-government versions of this are much more likely to come about but not as ransomware)
Holy fuck don't connect your freezer to the internet. And if you do, have it be read-only, not change settings. Did we learn nothing from Battlestar Galactica?
Or non-botty people sometimes copy-paste comments they leave under articles to multiple social media platforms? It references Battlestar Galactica, to nerds that's a good zinger.
Unpopular opinion: Accessing a public service with public default credentials is not a cyber attack but a consequence of severe misconfiguration. This is on the supermarkets, not on the attackers.
"Fool me once, shame on you, fool me twice, shame on me." But it should have been, like Bush said: "Fool me once, shame on — shame on you. Fool me — you can't get fooled again."
> a deliberate attempt to disrupt our food supply chain and undermine consumers
I would bet that if it will ever be resolved, it will turn out to be just some bored kids[] who read the recent report from France and had access to Shodan and a competent AI model.
[] I'm using the term "kids" loosely here, not refering to a certain age group but to a certain mindset, similar to the original term "hacker"
You want them back to use very fast after they have certain amount of icing. So you want powerful heating elements there and those are cheap anyway. Fridge or freezer not in use doesn't make money in a store.
Instead of pointing the finger at supermarkets we should point it at ourselves. Why the hell is anything computer related still so difficult? Even in this thread I see varying advice like airgap, VLAN, or VPN. Well which is it?!
We keep blaming store owners for not being IT wizards.
My fridge beeps loudly when something is wrong, and it has no connectivity to any network. Exactly how a fridge is supposed to be, why are we reinventing fridges…
Probably a central company that manages maintenance and preventative maintenance and uses them for monitoring and control. Taking them offline would probably cause more failures, and much higher costs in tech labor, tech travel, etc.
Ok, so now all the fridges have been remotely optimised: unplug them fromm the internet. I very much doubt the inability to ping a remote server will immediately cause them to shut down. Operate at last-known-good until mitigations are available.
The magic question is: why isn’t this fixed yet? And then the specifics, the nitty grift of why not. Thousands of industrial fridges. They presumably all need new hardware, or at least tenporarily all need a RPi with additional software on the network cable, or it could even worse… are they on regular company wifi? Don’t these companies have VLANs? It’s astonishing that this is what brings billion dollar companies in a key supply chain down.
This little gimmick cost Ahold in the Netherlands an estimated 300 M€ a few weeks back. Enough money for any firm to set the engineers to work? Where they waiting it out? Having read some more… another part of Ahold (the Belgian part) was hit and has (only now…) taken all fridges down from the internet. What is IT risk doing in a firm like that? Internet of Shit is a decade old. I am
not amazed non tech sectors are perhaps a bit slower in IT risk management. I am amazed that an ongoing attack situation can still cause trouble 4 months after publication and months after board level headline grabbing numbers are widely publicized.
Me and my crew (yet to form ;) will visit any fridge for a measly 100 K€ a pop to take them offline. Book early, because once the script kiddies hit, we are on the road.
The way some of these publicly listed companies work now they would "rather have" a cyber incident and claim the damages on insurance than hire engineers, because that looks worse on a quarterly report to have salaried workers.
People (and media) seem to regularly link the two, but there was never any confirmation and the issues that AH showed had nothing to do with store fridges. They took down a system responsible for stocking stores, fridges, registers, etc were still functional.
You are right. That was never made public. They were hit on exactly the same day as the French hack of fridges and took what look like a forward looking step of taking a lot of infrastructure down.
That does lead to harder things to think about. Multiple different attacks on EU supermarktes? Resilience? And then the questions I point to above: how hard is it for non tech firms to fix an emerging IT risk situation.
It seems more like an economic annoyance campaign or something?
If I were a country with this kind of attack sitting around and I wanted to use it for war stuff, I’d probably wait until a hot war had started. Destroy the freezers when everyone is busy making weapons or manning trenches, might actually cause a food situation.
Let's say China and Russia have hacked into FBI and CIA. It helps those countries build a psychological profile of the centers of power in Western countries. This can help in many ways. E.g. if it is known that Bitcoin is actually a CIA plot to transition the whole world into post-dollar economy, then China can plan ahead and ask its citizens to buy gold. But china started buy gold propaganda in 2002. Or as Putin says this because Russia has access to Epstein files.
"Western elites have been accustomed for centuries to fill their bellies with human flesh and their pockets with money. But they must understand that their vampire ball is ending."
It truly feels like Europe is just collectively shrugging at their lack of preparedness and aggression they face. What’s it going to take for them to take their own security seriously?
Isn't Europe just following US Army's example then? A similar incident in US Army commissaries has been reported since August [1]. It only took them two full months to launch an investigation [2].
This wave of attacks against refrigerators started in USA with many refrigerators serving the personnel at several military bases, a couple of months ago.
Therefore even the US Army has not been better prepared than supermarket chains.
It would take people listening to professionals and getting rid of several decades worth of bad habits amongst users. Same as most other places really.
> It truly feels like Europe is just collectively shrugging at their lack of preparedness and aggression they face. What’s it going to take for them to take their own security seriously?
I'm quite sure your average American company is just as willing to plug random shit into the internet.
Hell, there are probably hundreds of reports out there of American companies failing to secure their AWS S3 buckets, so you don't even have to plug anything into the internet.
The problem is the ironic "home grown" or "internal" security. These external attacks come after decades of fertilizing far right white politicians whose sole purpose is to dither any directed anger.
Brexit was the first to burst and despite their support of Ukraine, can at any moment capitulate to the ghost of nationalism and delusional escapism that is "white power". Americas on the verge of sinking further into the same. With those two entirely preoccupied, of course europe is next and already has the same underlying current.
Admin interface available on the internet without 2fa and automatic password change once a week? Anyone gaining sufficient experience and seniority in the industry becomes instantly unemployable, so it is what it is.
I mean in US these TV dinners, all these frozen foods are just way of life. Could it be that some good European citizens are angry with frozen foods and it is just their way of telling "EAT FRESH FOOD".
Can't tell if it will be some asshole hacker group, Russia, the us, or what. But for the love of God please stop connecting everything to the Internet and also ai...
Highly doubtful. More than likely someone is picking targets here. Going for food supply is a war crime. So it could be an extremely hostile nation state. Or some really psychopathic disgruntled person/group. If there's no ransom it strongly leans toward a nation state in my opinion. A lot of turmoil between Europe and Russia+the us lately. That's my guess.
>A lot of turmoil between Europe and Russia+the us lately. That's my guess.
Attacking power plants, grocery stores, sending drones into multiple countries over and over, setting up gun/dead-drops in German forests, sabotaging warehouses and factories, etc wasn't enough?
That implies you also ensure that they monitoring system is isolated from the control system. No one can be bothered to do that, and no one wants to spend the money on an extra microcontroller (a tiny fraction of the cost of each unit, but it comes straight out of your margin because no one pays extra for security).
Analytics are very different from controls. People who model them the same tend to suffer. A lot of academics and project managers don't understand this unfortunately.
1) "They" have no idea what exposed to open internet vs LAN vs VPN even mean. They have no idea what these terms mean, they just want the easy access. Same reason why open RDP to 3389 used to be so common. Vendors install these so the store owner and them have easy access to alarms and controls. So they vendor installs it for easy access, shows the store owner, store owner trusts vendor, but vendor has done the bare minimum. Because the vendor wants it easy too.
A VPN per store would be too complex for the vendor so they just expose these to the open.
Same story as with all the hacked OT/SCADA systems in the past. Like those hacks where some SCADA system is open as a website with pump controls.
Imagine they where internet exposed due to simplicity. Imagine you are a store owner with 0 IT knowledge.
The issue is this needs to be handled centrally by the parent org also. They have IT ofc but they usually do POS and desktops/servers. The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
2) Yes, obvious it is Russia doing this. Very handy to be able to disrupt food supply and they are also checking our response and time to restore.
I love how we're back to good ol' times that there is only one "evil empire", and it's completely impossible that anyone else from dozen of others possible bad actors is involved
I wonder if any of EU reactors is connected to the open Internet with a weekly password reset policy, where you can use the control rods from the convenient of your home.
Hackers remotely override settings on thousands of freezers, fridges and chiller cabinets, pushing them from as low as -27C to well over 50C. Intermarché, Coopérative U, Carrefour, Leclerc and Monoprix chains among those affected.
As I mentioned in the thread regarding the previous attacks, I worked in a closely related field back in 2008-2009.
One thing I see repeatedly brought up here in this discussion is "Why can't we just set the refrigerator temperature with a knob and walk away?"
The problem is that these refrigerator cases are not as simple as your refrigerator at home. As a simple example, many systems allow you have control over the deadband: the exact low temperature at which the compressor turns off and the high temperature at which the compressor turns back on. You also have control over defrost cycles, compressor pressure and many other things (I worked with this a long while ago, and I don't remember all the details).
Take a look at some of these manuals, for example: https://www.hillphoenix.com/resource/installation-manuals-2/... As far as I can tell, those manuals just cover the refrigerator cases, and not the outside compressors and heat exchangers, which are another layer to deal with.
This is not something that a non-expert supermarket employee would be able to handle well. The purpose of all these settings is optimize around energy usage, maintenance costs, customer experience, etc. Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
That's why this equipment is remotely accessible: it's not like your fridge at home and requires a lot more knowledge to get the most out of it. Of course, that does not excuse poor security implementations.
> The problem is that these refrigerator cases are not as simple as your refrigerator at home. As a simple example, many systems allow you have control over the deadband: the exact low temperature at which the compressor turns off and the high temperature at which the compressor turns back on. You also have control over defrost cycles, compressor pressure and many other things (I worked with this a long while ago, and I don't remember all the details). … This is not something that a non-expert supermarket employee would be able to handle well.
I still don't see why the expert has to be remote, or why an integrated computer can't do the optimization. Or why it needs to be fiddled with at any significant frequency.
Oh so its optimal to maintain a security team to secure your network but not to have on site tech
I site tech for a super market?
are you going to pay even more for meat? its already expensive
You think the cost comes from expensive local tech guy on the supermarket?
I mean it was every year before this one yes
One might wonder how we ever managed to do it before the Internet existed.
Dial-up modems were considered fully secure by comparison, when the remote appliance has its own private line.
Less efficiently. Supermarkets compete with each other on price. Margins are very thin, and supermarkets are much more efficient than they were decades ago.
>Supermarkets compete with each other on price. Margins are very thin, and supermarkets are much more efficient than they were decades ago.
That's too generic. How much exactly do these "connected fridges" contribute to that?
Saving energy (and hence electricity costs) and enabling timely maintainance.
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=supe...
Of course that would be the pretext, but that doesn't even begin to answer the question.
How much does these savings contribute to their overall costs?
How much difference in that over dumb fridges?
How much difference in that over what can be replicated by a localized control system, at most sending reports through an "output only" line?
And what exactly would remote repair fix if some fridge hardware fails (that's not just switching to a failover hardware, that could just as well be decided by a local controller)? Since the answer is close to nothing, how much of the "remote repair" need isn't cyclical, addressing issues coming from complicating their fridges to the degree that remote repair is required?
>Of course that would be the pretext
Right. Operating efficiency is just the pretext. Congratulations on uncovering the real story about networked supermarket fridges.
In all honesty, I can't prove that these systems help supermarkets reduce their operating costs. It just seems the overwhelmingly obvious and likely explanation for their adoption. If you want to argue otherwise, then I'd say you owe us more than just a list of skeptical questions.
A lot of retailers like CVS and Walgreens (~16,000 stores between them) use regional contractors, they work for numerous large companies including Walmart if needed. You don't pay for idle engineers per se. Instead you pay a relatively work rate: high cost to show-up, high hourly rate.
>>just connect the devices to the Wild Wild Web and be done with it.
Yup, they're done with it, all right. "It" being all the food being cooled for sale this week. Done.
They do not need engineers at every location to turn every knob. They need to define procedures and limits, and the ability to call the engineers when things go out of those limits. Even putting only remote MONITORING online, but requiring the local person to check and turn the knobs/press the buttons. That also adds an extra set of real eyes on the situation — when the engineer calls the local dept manager to start the defrost because the hacked line says it is now time to defrost, the local manager —with eyes on the freezer— can tell him "NO!" because this was not on their schedule and the freezers are all stocked, not emptied for the defrost.
Instead, the defrost cycle just turned on in the middle of the night, and neither the now-remote engineers nor the local managers knew until it was too late. ——Again.
Security is always a trade-off against convenience. And the idiot human managers almost always all pick convenience because their managers incentivize profit, and security costs and convenience appears to save up front. They nevermind the later costs because they'll already be promoted up or job-hopped out.
You're too optimistic. I work as consultant in industrial controls. I often get called for:
- "machine doesn't work!" It was set in the wrong mode. Who did it? No answer
- "machine doesn't work!" The HMI shows a huge message with "CHECK SENSOR AA-123" or "VALVE BB-321 CLOSED" and they are puzzled when you point it to them
See also the "PLC program changed itself!" in-joke in industrial automation circles.
The reality is that what you envisage "does not scale", to borrow from software engineering.
Thank you, very informative, and having had to support biz/network software I wrote, entirely credible.
So, how can this be solved? Ideally for the long-term, but also for the current interim state — we have a known attack vector costing millions, known exposed, and need to do something.
Short term, seems like disable defrost or other critical vectors? IDK the architecture, so IDK how difficult it would be to add a local interlock or override for critical instructions that would damage the whole contents, so the local manager must also confirm "READY TO DEFROST"?
Long term, seems the only solution is to rearchitect a truly secure solution with secure access controls, locked-down to a limited set of IP and MAC addresses, all traffic encrypted, etc. Plus a deep evaluation of the sets of instructions that could be catastrophic, and adding requirement for local confirmation/override to execute those functions.
It also seems there is a lot of available improvement in the UI so regular Joe-managers could use it. Even if they see "CHECK SENSOR AA-123", their eyes glaze over because they just see it as the gateway to a hellish maze they don't want to enter. Adding layers of actually informative error messages and walk-them-thru steps to fix it seems well within our technology. It'd probably require an additional local mini-PC on each site, but could save millions.
If this gets scaled to a whole country, when most of our food supply is Just In Time, it is no longer a prank.
If they hired an extra 20 managers across 1000 stores to deal with twiddling fridge knobs over the phone, we'd be complaining about that instead. And someone would still call up managers and social engineer them to setting the knobs wrong.
They wouldn't have to hire 20 extra managers, they just would need to develop some materials and train the managers they have.
And social engineering 1000 managers is FAR more work than scaling a software hack across 1000 machines, and also far less likely to succeed. At least one of those managers is going to get suspicious and call corporate first. Moreover, it will be pretty hard to get an on-site manager looking right at the full freezers to set them to defrost....
It's an obvious decision in hindsight. Not so beforehand.
This lot were already in the hindsight condition. The same thing happened all across France three weeks ago. THREE WEEKS. It was already obvious this could happen.
Of course this is not enough time to entirely change one's strategy, but it is a lot of time to install hardening measures, such as stronger passwords, limiting access to only MAC addresses of specific machines, taking access offline wherever possible, disabling the defrost cycle, etc., etc., etc., and that is just the obvious.
Security and convenience are diametrically opposed and stores are specifically built around convenience. I’m not sure it’s obvious at all what the right cutoff level is.
Considering food is a national security item, the threshold must be higher.
One store loosing its frozen food is an inconvenience and insurance claim, and one electric substation failing is an inconvenience for a few days.
But whenever any one of these attacks could be scaled up, as is obviously possible with online access enabled, and the consequences become a regionally or national security threat, enabling such access must come with the seriousness of implementing national security.
Unfortunately, it is not sexy for politicians to take national security seriously, and they will get inundated with lobbying decrying the costs. Until the losses to the companies exceed the costs of security, when they come screaming to the govt for help, and demanding why they didn't do something.
It’s some frozen food, it’s not botulism in the rice. It’ll be hard to convince people of how big of a deal this truly is.
No question, it'll be hard to convince most people of how big a deal it really is.
It is the same with almost every systemic problem that seems small now but is cumulative and/or exponential.
And for sure, if it is a few stores, it is just an expensive prank. But scale it to the size of a nation or region in a country with a largely just-in-time food supply, and you're a week away from food riots, particularly if it is combined with other attacks, e.g., on the electrical and distribution system. Not a shot fired, and the country is in chaos in a week. Because no one bothered to harden the systems that seem trivial but are critical at scale.
That's my thought too.
These fridges may need more complex control for a variety of reasons, but that doesn't mean they have to be connected to _the internet_. To a local network for control, sure.
Put your security measures into the network itself, but I really do feel fridges are another thing which do not need to be IoT devices.
Local network, VPNs, bastion hosts, and so on. It really isn’t that hard.
Exactly this, I wonder why this setup (with ability for external techies to be granted temporary access) was not chosen in the first place. Considering the threat model, "my freezers all defrost at the same time all over my stores because of ~cyber~" should be a critical item to be discussed
> I still don't see why the expert has to be remote, or why an integrated computer can't do the optimization. Or why it needs to be fiddled with at any significant frequency.
Well, it doesn't. The retailer has a team of techs (or an outsourced vendor) rotating between stores, who are cleaning and replacing air filers, checking lubricants, and so on. They could certainly apply occasional changes to settings.
But to the product manufacturer - the remote management feature literally requires less than $2 of hardware. Hell, it might even save money, by removing screens and buttons.
And the person buying the refrigerator wants something that logs and monitors temperatures, for food safety. And they don't know anything about cybersecurity.
So when the manufacturer says that for just a little more they have a remote monitoring option, which not only provides the monitoring needed for food safety but also reduces maintenance costs - it sounds good to them. They don't realise they're signing up for a multi-million-dollar loss when they get hacked and have to throw out a load of products.
> And they don't know anything about cybersecurity
And this is the key fact. It’s no longer acceptable to sell food and not know anything about cybersecurity the same way it hasn’t been legal to drive a truck without a license in decades (because a lot of people died). There was ample warning since the incident at a couple US military bases earlier this year. Market owners need to face it.
It should be the fridge company's problem to secure their fridges. These aren't bought, they are rented as a cooling service. The supermarket owner is paying a certain amount per month to make the fridges someone else's problem, so that someone else is required to handle the problem.
If what they rent is not freezers, but refrigeration, then they have a chance of having some sort of compensation for the loss of service, unless managing the fridges and their network is the market's responsibility.
The manufacturer/operator could, at least, not come with the omnipresent admin/admin credentials combo.
> why it needs to be fiddled with at any significant frequency
Weather changes, equipment performs differently as it wears, changes in customer behavior, different items being stocked, electricity cost updates...
> I still don't see why the expert has to be remote
Because then he can get paid more.
> Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
People always say stuff like this, but it never tracks for me. If you’re a chain with 1,000 stores, $500,000 is barely a blip on your budget sheet in exactly the same way that $500 is barely a blip on the budget of a single store.
Plus there’s no way this remote management stuff doesn’t cost at least $500/store in operating costs.
> Plus there’s no way this remote management stuff doesn’t cost at least $500/store in operating costs.
Going by the number of companies that still can't grasp having to pay for ongoing maintenance for and IoT system, even with CRA and NIS2, some still selling stuff running on 12+ year old software, I wouldn't expect them to pay much for this service or to even properly maintain it.
> $500,000 is barely a blip on your budget sheet
Famous last words.
Don't compare it to the entire balance sheet, compare it to how much you'd have to pay someone to optimize it.
At $500/store/year times 1,000 stores, you can pay an entire team of experts for a year to come up with a way to save that money, and come out ahead. That team's manager can then go to their boss and say, look, I saved the company half a million dollars a year, and they'll get a nice bonus from it.
I don't think people question remote accessibility as much as they question why remote accessibility is a matter of guessing the standard password.
You can have remote access through secure means. You might even have a web page open to the public if you secure some kind of safe login mechanism.
If your story is true, then these stores and chains better take the financial impact they hit out of their CEO's paycut, because they absolutely chose to risk it all for some minor cost savings. Maybe try to sue the manufacturer (though I doubt the judge will side with you if you're hit with the same attack just a week or so after the hack appeared on the news, without having fixed the problem).
FWIW, Reuters is saying the total losses are in the range of 10's of millions of euros across all stores.
And how much money the shop looses from the attack? It sounds it wiped out all the saving remote automation brought.
Some people have to learn the hard way, I guess.
Ha! I bet nobody learned anything.
Almost everything you described is either better set at factory (or at coolant recharge - like compressor pressure, etc), or tunable on the antique mercury switch thermostat we had growing up.
The only remaining thing is the defrost cycle stuff, and even that can be 90% of the way to optimal just by having a clock that knows when peak power is. The classic thermostat we replaced the mercury switch had that. I imagine they're like $10 retail, $1 for the microcontroller.
Assuming the extra complexity does actually save another $500/store/year, how much of that $500 is eaten by maintenance contracts, software patches, wifi debugging, etc, etc? Also, what happens if the internet goes down?
I'm not convinced it makes any sense for these things to be online.
It would still be miles better to add something to do on the hardware side. For example: every time a new firmware or setting is pushed, a physical employee has to go press a button (or turn a key) to make it apply.
It would be a more sensible tradeoff bo matter the scale of the store, thab jeopardizing the entire fresh supply of a nation.
> “That's why this equipment is remotely accessible”
Complicated things don’t have to be connected to the internet. Most environment control systems in small buildings (typically with hundreds of sensors and actuators) aren’t. There is smart, deterministic, software involved but most of the time, no cloud.
How did these work then before the internet existed?
In a word, poorly.
Setting up and changing parameters would involve, at the very least, a technician walking a store employee through the process. Best case scenario the technician would dial in using a modem.
They weren't able to set up to work as efficiently.
They were more secure though.
Also, does all this remote management whatever stuff cost more or less than slightly better insulation, or an upgraded compressor?
Thank you for explaining. OK, so I get that it's not as simple as "there is NO reason to have remote access to a fridge". There are reasons to. But there are also reasons not to. I hope the decision-makers involved will start carefully thinking about the trade-off involved.
Maybe they $500/store/year is not worth the risk? Maybe having read-only remote access will still save $300/store/year and $200/store/year is not worth the risk? Maybe having a physical button that someone at the store needs to push to temporarily authorize write access is worth the hassle? etc.
I don’t doubt the technical details but on the economics: surely saving a small amount of money per store is still a small amount of money per thousand stores? If each store costs $500k per year in staff, maintenance, and power then the savings from having fancy refrigeration units mean I can run one more store on top of the 1,000 I already have. It doesn’t seem like much, but I am a business illiterate computer programmer not an exec!
If margins are already at the least possible bottom (according to market logic this should always happen with commodities and competition) then you need to start to look for ways to reduce your operating costs in order to increase them again
Why? If you are already at an optimum (which "least possible bottom" indicates), then any kind of you trying to move into a different state, is going to leave the optimum.
I think there can be a compromise. A range of acceptable of operating values is chosen, and remote control is allowed within that range. But dangerous values like turning the freezer off or setting it to defrost should require more ceremony.
>The problem is that these refrigerator cases are not as simple as your refrigerator at home
And yet they existed, and didn't need a connection to the net in the 80s and 90s. Most supermarkets not even in the 00s.
This is one of those "it's all so complicated" answers that ignores the fact that for most of the past century they managed to do this without the thing that you're claiming is now necessary.
And the real answer comes out eventually: it's necessary now because we consolidated and cut costs and jobs. The people left who we pay next to nothing could never even handle them at the level we had then, because we laid off everyone who knew that old stuff.
> Even if you only save $500/store/year, you can see that it really adds up over a chain of 1000 stores.
And even if an event like this only happens once in 20 years, all of those savings evaporate. Thank god nobody will get fired for this. Or at least the first wave - it will be insane if nobody at the chain hit by this current wave gets fired.
>they managed to do this without the thing that you're claiming is now necessary.
They managed to do it when none of their competitors was doing it either. It's a low margin business.
> And even if an event like this only happens once in 20 years, all of those savings evaporate.
I don't think the numbers on that actually track, if you look at the estimated damages that are available. At once every 20 years, you'd still come out ahead.
If only 50$ of those savings were invested in Tailscale.
Or netbird. I was exactly thinking the same btw clearly we're missing out something. or maybe cybersecurity is a cow ready to be milked?
Did Tailscale not have a major vulnerability just a few days ago?
For a specific service only, not the networks. But that doesn't even matter that much - if you require the attacker to get both into the network and into the device, that's a significant improvement.
$0 for Wireguard.
Only if your time is free.
I don't see why saving $500 per store a year is better than exposing your systems to cyber attacks.
I used to talk to a supermarket chain COO (or whatever the guy who handles the operations of all stores of that brand) in China. He told me that supermarket has extremely low profit margin that sometimes a couple of unexpected bad days would turn the number of whole year off. I can imagine they would like to squeeze every cent out of it.
Grocery is a very low margin business, and they look for savings wherever they can.
"That will never happen to us" - until it does.
It’s not even “That will never happen to us” it’s “Joe, I need the WiFi password for the new fridge, do you remember what it is?” “Sure it’s $t0reNam3secur3dbec4useitsin13372023”
It was a better deal before AI, but now the economics of a lot of things are changing rapidly.
And yet... if they had instead invested in making the equipment operable locally we wouldn't be here today. This is the problem with thinking of your users as "nontechnical" lesser beings. The engineering problem becomes remove control and ownership from the user instead of make the product usable. In this case by removing control from the user they simply gave it to an attacker. Great job, guys.
I addition to the technical knowledge needed to make this work well, there are misaligned incentives between the store managers and the company as a whole.
One particular example: store managers want a very cold, very narrow temperature deadband on their frozen items. This gives the customers a nice experience. They're less likely to complain about melty ice cream. But this type of setup leads to more energy expense, and failures on the compressors due to excessive cycling.
Could this control problem be solved locally? Like is there some intuitive setting you could give an end user that still results in efficient running behavior? Wouldn't that be strictly better than putting it on the internet anyway?
Bless those hackers for firing a warning shot over our freezers, that we may take heed and secure our neonatal care units, stoplights, and air traffic controllers.
Bosses at hospitals, towns, and airports/ARTCCs: Why would we secure our stuff? They're hacking into freezers, not our computers.
Not yet
Sadly, the exact same hack happened in hundred of french grocery stores three weeks ago. I question people's ability to fix this kind of issues
And a couple weeks earlier to a couple dispensaries in US military bases.
At that point it should be firing alarms on every grocery chain everywhere. While you may not be in a US military base (they are impopular with a lot of people) the fact is that someone figured out how to do it and others will figure it our too.
and electric stations, municipal water pumps, cellular networks, over-the-air software updates to your car ...
Internet connected self driving cars are a huge military asset if controlled remotely by a hostile power.
Those won’t be secured until breached. Surely you jest.
I'm extrapolating a bit here based on Reddit hearsay, but I think this may have to do with rural stores having essentially consumer-grade internet routers connected to the mobile broadband network.
Much of the coverage in Swedish media drives the point that only 4G-uplinked stores have been hit so far. I think that's unfortunate, because really it wouldn't have differed if these stores were connected by 5G.
I'm guessing here, but my money would be on freezer ICS systems being made available to the world by accident by way of NAT-PMP on a consumer router. In Scandinavia you can bypass CG-NAT on mobile connections by setting a particular APN, so my guess is that these control systems were simply findable on Shodan.
This also explains why this wasn't discovered after the recent French breach. Supposedly all the stores with "proper" uplinks were in compliance, and no one had thought of a way in for those non-compliant stores, because they "weren't even connected, so how would an attacker get in?"
"... I think this may have to do with rural stores having essentially consumer-grade internet routers connected to ..."
It has to do with refrigerators reachable on the Internet.
If the refrigerators were not reachable on the Internet, none of this fragility would exist.
The fragility of refrigerators reachable on the Internet was demonstrated just a few weeks ago. These refrigerators were left connected to the Internet.
This lack of discernment and misunderstanding of risks and fragility will not be solved with a better router or a different uplink.
Yeah
And it would be (mostly) fine if they were on the net, with read only access
The entire reason they are put on the Internet is to allow for remote control. What problem would it solve to allow remote monitoring of refrigerator data without control?
I'm not in perishable food so what do I know.. But organization wide food safety measurements are probably more important than the average process BS.. Performing a full heat cycling of a freezer while no one is present to confirm the correct device OTOH sounds idiotic.
Because then you can monitor it, and then call someone at the store to change the settings. Or go into the store to fix the one that isn't doing what's expected. You can still get alerts, diagnostics and data.
I mean, what happens when a compressor or other physical part goes out? You can't fix that remotely anyways. You might still want remote monitoring for that kind of situation.
How many times have you changed your fridge temperature in the last year? Once a year? Twice?
And most importantly, these fridges always have someone close by. Make it a physical button instead of a remote function
Do you think the code in these systems is secure enough to enforce read-only access?
Easy, you don't connect the control systems to the monitoring systems.
At the very least the system should pull configuration info from a trusted source at predefined times instead of being available for it to be pushed at any time.
It doesn't need to be perfect, it only needs to be better than what it is today (predictable passwords)
Many of these smaller stores (and probably some of the larger ones too) rent or lease refrigeration equipment from one of a handful big providers. These are large and expensive systems, and there is costly paperwork involved together with logistics suppliers to show that the cold chain is satisfactory.
These refrigeration providers work with local contractors and HVAC companies to set things up. Their installation techs do what they think is required to remotely operate these systems. If you bring up the question keeping these systems offline or at least on airgapped networks you will most likely get a shrug as reply.
This is not a technology problem. This is problem of responsibility. One can hope things will improve when litigation starts happening, but it doesn't look good considering these people had literally months of notice since the well-publicized attacks on the American fridges, then the French fridges, and now these.
Litigation is not how we fix things in Sweden, the system is very different from the US. The companies will have to take the losses or try to recoup from insurance and update their systems. They might be able to negotiate deals w the fridge install companies.
How do things get fixed if not by litigation?
Usually national associations get together, discuss issues, and come to some sort of agreement or recommendation that is then followed by most actors.
Litigation does of course occur but it's not the main mechanism.
We try this in the US as well, it’s commonly referred to as “hopes and dreams”.
Do you folks coordinate in the way described, and show success with it? (Would most people agree with you?)
The Swedish national anthem is entirely defined by common consensus, there was a proposal to establish it by law in the 2000's but that was rejected.
Sweden doesn't have a state created e-id system, instead a consortium of banks jointly funded development of a solution in the early 2000 and this is basically the default way you prove your identity online in Sweden.
Fucking Saltsjöbadsavtalet.
Is this a stupid question..?
Are you using the up and down votes as a Yes and No reply? ^_^ Should I follow the white rabbit?
Some countries have regulatory bodies that are empowered to effectively regulate industry.
It's a nice thought. I doubt it's true in reality. If it were true in this specific case for example, the industry wouldn't have the freezer situation to begin with (it implies non-effective regulation is already the context).
It is simultaneously true that:
1. many countries have higher regulatory burden than the US
2. that does not mean regulatory gaps don't exist
The person above was simply pointing out that litigation is not the only way to solve a problem. Which is plainly true, you can simply require compliance via other legal means. For example, a government could create a regulatory standard for security and require vendors to follow it. Whereas in the US the political appetite is to be more hands-off and allow people make vulnerable products if the market allows.
Isn’t that just litigation brought by the government instead of private citizens?
No, most of the activity or function of regulatory bodies is not litigation. Litigation would be the last resort.
Litigation is the last resort in the US system, too. It is the threat of fines/damages that incentivizes a company to do right by its customers/other stakeholders.
I never said it wasn’t. What is different in the US is a political reluctance to enact regulatory controls to protect rights proactively.
Parent is not correct. Litigation certainly is a thing, but it is less prevalent than in the US. Perhaps the food chain will take the loss, be covered by insurance, and the insurance company will sue.
If not buy insurance, the customers will have to pick up the tab. Vendors sell the product/service and are completely responsible for any losses.
That is the same way it works in the US.
All of the fridges stayed online after the hack. I ran a Shodan scan a few days later, and the number of reachable fridges actually increased. The first hack might have been a surprise, but anyone who was IT in a refrigerator management company should have been able to run a (free) Shodan scan like mine and get the whole list.
> I'm guessing here, but my money would be on freezer ICS systems being made available to the world by accident by way of NAT-PMP on a consumer router.
Why would there be accidental port mappings on their router though? I could maybe see owners setting it up so they could monitor from home, but even that is a bit of a stretch for rural supermarket owners
> Why would there be accidental port mappings on their router though?
That's the point of NAT-PMP / UPnP, it just works (for game servers, torrent clients, industrial control systems etc.)
The idea of a control system requesting a port forward rule using NAT-PMP is itself rather questionable.
Ah ok, I misunderstood what NAT-PMP was. Thanks for explaining.
My guess is setting up a port forward in the gateway router is part of the freezer install process. The customer probably had to give the tech the router credentials and the tech sets up the forward themselves.
It shouldn't matter anyway. The endpoints should been using default and secure technology or vpn only.
> In Scandinavia you can bypass CG-NAT on mobile connections by setting a particular APN
Details, kind sir
absolutely unbelievable (even by the very low standards I see, being in iot myself) that any commercial equipment would be doing upnp
someone manually poking a hole in their firewall is way more likely
Back in 1995 this was so improbable it was one of the purported negative effects of the "Good Times" virus which was a parody of the virus warning e-mails that were being forwarded around at the time. Computers still can't give you Dutch elm disease, fortunately.
> It will recalibrate your refrigerator's coolness setting so all your ice cream goes melty.
https://people.cs.rutgers.edu/~watrous/goodtimes-spoof.html
Somebody made that into a song. Never knew where the lyrics came from.
https://open.spotify.com/track/1mvK5h91yWsm97sAugKy6K?si=wXw...
Thanks! Also available without login on YT https://www.youtube.com/watch?v=uqZZ72UbOgA
Also hit a few smaller business in The Netherlands: https://nltimes.nl/2026/11/02/fridges-freezers-17-dutch-busi...
With the following interesting quote from the supplier of the system:
After that attack, Danfoss advised its Dutch clients on October 13 to disconnect their systems from the internet and ensure they only reconnect via a secure network. “Most stores took action at that time, but a small number did not,” the spokesperson said. “They are now the ones suffering the consequences.” The spokesperson added that the systems hit last night were not on a secure network and were still using default passwords. The attack could have been prevented if measures had been taken, he said.
This is a total failure by the vendors of these freezers.
If you charge money for something that is critical national infrastructure - and the food delivery pipeline is absolutely CNI - and you connect it to the Internet, you should also be responsible for keeping it secure - or at the very minimum at least trying to do so.
The same goes for other CNI such as telecoms (including home routers and phones), broadcasting (including "smart" TVs at the eyeball end), banking and payment processing infrastructure, cloud computing, electricity, gas and water supply, traffic signalling... all modern economies are now almost totally dependent on so many different interlocking internet-connected systems. I've got a stash of food and water at home, a battery powered radio, candles, blankets to keep warm and so on, but that's only really enough to deal with a short-term emergency, not the collapse of technological civilization.
Failure to understand this is not an excuse. Sadly, the only way to deal with this is legislation.
Discussion from 3 weeks ago on the refrigerator "hacks" in the french supermarkets:
https://news.ycombinator.com/item?id=50055304 (~1000 comments)
Thanks! Macroexpanded:
Hackers activate automatic defrosting in supermarket fridges across Europe - https://news.ycombinator.com/item?id=50055304 - Oct 2026 (1007 comments)
I think the military commissary's freezers were hacked - https://news.ycombinator.com/item?id=49508506 - Aug 2026 (239 comments)
Thanks!
The comments in the Aug 26 thread aged quite badly.
Why isn't the brand of cooler or cooler-controller being named?
Was it a password break or an open admin console or SNMP or what?
https://cert.cyber.gouv.fr/alerte/CERTFR-2026-ALE-012/
The real issue is geniuses in food stores connecting to the open internet systems with 100K+ EUR of products exposed.
Remember, the 'S' in 'IoT' stands for Security.
Good link, thanks.
Was a bit surprised (well, no, not really) to see that the linked XWEB bulletin in that alert mentions a backdoor which can be used to help customers regain access to breached systems. Wondering how likely it is that this backdoor algo has also been leaked or reverse-engineered too:
(translated from French to English with Google Translate)
> If your password has been changed and you are unable to log in, please contact Copeland Technical Support, available 24/7, at [...] and ask for instructions on how to obtain a temporary password. Copeland technical support will need the current date of the XWEB EVO and the MAC address displayed on the XWEB login page.
Similar to the 'S' in VPN
How does that translate? Are VPNs generally not secure?
It depends a lot on what you're using it for.
For your typical home user that bought a VPN because a YouTuber promised them extra security and privacy, no. They're gaining next to no security unless they have a malicious ISP that injects content into HTTP connections.
They are used for security, but VPNs don't magically make things secure. In the industrial control space VPNs are considered risky because a remote device can be compromised and access everything else on the same VPN. Separate networks, VLANs, firewalls and other security approaches are used before hitting the problem with a VPN hammer.
In the attack on French supermarkets a few weeks ago, hackers pirated controllers mostly supplied by the Danish company Danfoss.
Multiple sources confirm that it's the same Danfoss problem this time, for example, https://www.hln.be/binnenland/centrum-voor-cybersecurity-bev... (in Dutch)
In the Reuters article on it weeks ago they mentioned finding 4500 similar freezers still available online, with the simple login bypass included. So yeah that was just a matter of time?
Glass half full: The scale of this is rather small overall, so most supermarkets etc. appear to have at least some form of basic security.
Somebody needs to run Windows Update :-)
> the hackers pirated controllers mostly supplied by the Danish company Danfoss, which said subsequently that it had identified some security flaws last year and had “long recommended” customers install updates.
This but unironically. Windows Update forcing updates and reboots is precisely to prevent incompetent IT from running unpatched systems.
In France I believe that the password on the monitoring software, or the controllers, was updated daily. The date was somehow used as the new password, or as an input to the password sequence. So rolling the passwords didn't do anything when the attackers are aware of the "algorithm".
Sounds like other retail systems where lock screen passwords are the store number or similar conventions
The BBC names (and quotes) Sabcobel as one of the manufacturers.
https://www.bbc.co.uk/news/articles/cq1kd1nee80eo
It it named in the article:
"In the French attack, the hackers pirated controllers mostly supplied by the Danish company Danfoss, which said subsequently that it had identified some security flaws last year and had “long recommended” customers install updates."
If you definitely need to connect them to the internet, make them read only?
I was talking yesterday to my AI imaginary friend about creating an MCP for one of my apps, and the first thing she told me was to make everything read only. No matter how many security layers you have in place, passwords, auth libs, whatever, it will be leaked/hacked so you better be prepared for minimum damage
Monitor your freezers temperature? yes, but never allow their thermostat to be set over the internet. I hope the lesson is learned. Applies to all appliances
If your use case and budget allow, hardware exists for physically enforcing read only networking properties: https://en.wikipedia.org/wiki/Data_diode
I'm not that familiar with applications/current state of this type of hardware, but it seems like it will become more relevant with all the new vulns dropping with higher frequency.
They do it to take advantage of energy price fluctuations, but the connection should be more secure. What makes no sense is not having a fallback to take the freezers offline after the attack, so their solution is to just fully turn them off?
That would be cool, do you have any source that confirms they used this for taking advantage of price fluctuations?
Maybe we should stop connecting things to the Internet that don't need to be?
I don't understand why these things aren't on their own (V)LAN. Why they are open to internet?!
Brainstorming here; monitoring, logging, etc. The fact the store managers were called out of bed when there was an issue indicates it works. If a cooler broke down overnight or its temperature was too high for X amount of time, they have to discard the products. Without monitoring they likely wouldn't know until customers got sick.
I wish commenters (in general) would do a bit more "steelmanning" before asking questions about why something is (in this case) connected to the internet.
All of this can be done without connecting edge devices directly to the internet (and potentially allowing inbound connections). The question is not why they have networking, but why the connectivity is such that an attack like this is even possible.
>I wish commenters (in general) would do a bit more "steelmanning" before asking questions about why something is (in this case) connected to the internet.
People won't steelman because the answer to "why are the fridges slapdashedly networked?" is too often "because people of my ideological bent and demographics if not literally me previously demanded paper trails, audit-ability and regulatory compliance that was impossible for a low margin industry to pull off any other way."
That seems like a pretty wild nonsequitur at first glance. How exactly do we get to “too much regulation” from “a subset of stores’ centrally managed equipment was left off the VPN”?
My point, which will inevitably be straw manned to hell and back, is that a large part of the reason these systems all came about is to cheaply satisfy externally imposed requirements (save energy, facilitate audits, remove responsibility from store employees, etc, etc) that the peanut gallery (and more concerningly, real people doing real work deciding things) cheers for when examined in isolation.
Requirements in the food industry come not just from government rules and suggested best practices (that are more than suggestions in practice) but also indirectly as a result of requirements imposed by insurers, financiers and trade groups. In response to this companies that run supermarket chains or manufacture major brands of food have not quite the same verbosity but certainly comparable degrees of retention in their record keeping as medical and aviation.
This monitoring and control is increasingly centralized in response to all these requirements because it's most efficient that way. If you're gonna have one guy in an office deciding things you might as well automate away the middle man and have him set them too and let the people on the ground spend more time doing their core work.
If fifteen of the twenty thousand dozen systems (pulling that number out of my ass FYI) in an industry are being centrally controlled through computer systems it's inevitable that some of them wind up off the VPN and this happens. It's just a natural tradeoff or risk you run when doing that and it's completely foreseeable.
But the internet peanut gallery will not generally have any sort of adult discussion about these tradeoffs because "we saved X (maybe X is money, maybe energy, who knows, who cares) by operating the fridges more centrally" is a nice easy win for them and rather than have a discussion about subjective and hard to pin down long tail risks it's easier to just screech about "we should fine the people who did it wrong" or whatever when one of those long tail risks rears its ugly heads.
I still don’t follow. You’ve said a lot of things about regulation and insurance requirements and all that, but I don’t see how any of it is necessary to get to “centralized management.” Let alone goofing up the centralized management.
I’m just a software person, not a supermarket expert, so maybe I’m being a bit simpleminded here, but for the entirety of my career I’ve participated in a constant trend toward centralizing and automating operations. That’s been the case regardless of how highly regulated the industry was. The motivating force has always been making things easier, cheaper and less error prone.
I suppose we could throw a rant about regulation into the mix, but, in the words of Laplace, I have no need of that hypothesis.
given they turned off the refrigerators (instead of something evil like turning them down for just long enough to allow bacterial growth and then turning them back up) customers would probably have noticed their ice cream being liquid, the frozen meat not being frozen, etc
Monitoring and logging can be done without access to controls.
There was an interview with an angry store manager, he's mad at the attackers, but I didn't hear the media ask him why they didn't improve security after the French attacks. Only now do they appear to want to start a review of their monitoring system.
https://www.svt.se/nyheter/lokalt/skane/icas-kylar-och-frysa...
Isn't it more than logical for a store manager to be upset?
At the human being level, this attack is mind boggling. What do you get out of this, other than making good food go to waste?
Edit: Also keep in mind that there's franchising in supermarkets as well. So the guy might literally be looking at losing a chunk of money as a result of something out of his control. Doesn't change the situation, but not having done something doesn't make them deserve this...
He was sitting in a car behind the wheel driving with 120 down the cliff and just closed his eyes instead of stopping the car, 'but not having done something doesn't make them deserve this'.
The store managers don't manage the IT security of the chain though.
No, and I get that, but working in the industry, he can't have been in the dark about the events in France. If he doesn't deal with the monitoring of the fridges himself, he should be equally upset that the chain has failed to act.
How did every supermarket chain around the world not jump at this and investigate?
Probably most of them had exactly zero inhouse expertise in cybersec and just pretended there’s nothing to worry about because having to worry would be inconvenient. Maybe some of them made a call to their cold chain supplier and were told that there’s nothing to worry about.
I wonder how many people working in any given supermarket chain actually know that their fridges are connected to a network at all.
Not many, probably, but their cold chain people definitely would have known. Instrumented and remote monitored cold chains are standard these days because keeping stuff refridgerated end to end is a pretty big deal.
cold chains were already standard before connected fridges, one doesn't mandate the other. The issue is most of all not remote monitoring but remote management.
I don't know what "cold chain people" are, at the scale of supermarkets you permanently have refridgeration for different groups of consumables.
I'm not aware of a scenario where e.g. fish is being shipped, and the originating logistics remotely sets the temperature of a fridge at the destination.
I bet most managers don't even know you can even connect a fridge to the internet.
have you seen how big enterprises operate? It would generally take much longer from a decision to do something to any action happening, budget approvals etc.
And for anything infosec, unfortunately people who know a lot about infosec have a habit of 'crying wolf' and talking about infosec as some kind of absolute rule that everything has to bow down to with no willingness to balance against other business risks, so probably leadership is used to ignoring the people who knew that this was a big deal and were calling for action.
Their cybersecurity departments (next to fresh produce) are all over this.
Snark aside, supermarkets are not in the tech business. The extent of their involvement is ringing their IT vendor to complain about it.
Depends. Lidl wants to be a tech company.
https://en.wikipedia.org/wiki/Schwarz_Digits
It really depends on the grocery. Kroger gets a big chunk of their profits from their data stack.
Because most of these chaps do not read daily news related to their profession.
I wouldn't be surprised if its their own agencies hacking these systems preemptively in order to force the issue and increase robustness against a larger multi-pronged attack.
I’m still trying to figure out why these freezers are connected to the Internet in the first place…
Monitoring, alerting, remote diagnostics and trouble shooting.
Commercial cold chains are expensive, high value collections of equipment that even small errors at the edges can cause huge monetary losses. And grocery stores are in an extremely low margin business.
Those are read-only. None of those things require the ability to heat up a fridge remotely.
Diagnostics and troubleshooting are not necessarily read only. Some of these systems have temperature controls that are centrally managed for operations centers for instance.
Monitoring should be in a DNZ that can talk to the otherwise “airgapped” network and push metrics out to the SaaS platform or whatever. Send a technician if something needs to be adjusted, or (only if some kind of remote access super critical) give the customer the ability to grant temp access to technicians for remote troubleshooting (but require someone to be in the facility).
None of this requires the customer to be technical, the ORM could ship the equipment with the network setup, and installation would just be connecting the DMZ device to the Internet.
The question wasn’t why these systems are setup insecurely (which has an obvious answer about cost prioritization).
Of course there are ways to setup these things securely, same with smart tvs and doorbell cameras and whatever else is getting hacked. People just have a bad time figuring out the cost benefit of all of this. And frankly we don’t know if that cost benefit has even been crossed yet, perhaps this doesn’t rate.
You yourself are engaging in cost prioritizing without adequate information. Sending a technician on site may be extremely expensive (opportunity costs and inventory risk on cold chains are wild).
> The question wasn’t why these systems are setup insecurely (which has an obvious answer about cost prioritization).
It’s not obvious to me that the secure approach is measurably more work than the insecure approach. Someone had to go out of their way to add support for remote temperature control.
> Of course there are ways to setup these things securely, same with smart tvs and doorbell cameras and whatever else is getting hacked.
No one expects a $20 IoT light switch to ship with its own network equipment, but a $600,000 to $1.5M fleet of freezers for a single grocery store can ship with its own network equipment without increasing the cost a single percentage point.
> You yourself are engaging in cost prioritizing without adequate information
I had hoped that was obvious from the question mark in my original post :) I’m asking what information reconciles this “it feels like they went out of their way to make this insecure” security posture. Responding with “they must have information you don’t have or else they wouldn’t have done it that way” is neither helpful nor logical.
I'm a bit shocked about all the uninformed arguments in the comments. Maybe I shouldn't be, because apparently basic infosec is a foreign language even to large supermarket chains and commercial freezer manufacturers. It's 2026, why the hell would you have critical systems exposed to the internet? In.. "normal" industries this wouldn't just be reasons for immediate termination, this would be a civil lawsuit for recklessly endangering company assets. I'm genuinely flabbergasted. And then it happened, AND IT HAPPENED AGAIN AFTER THREE WEEKS. Why did they not pull the plug at the very second they learned about the first incident?
I'm not happy about the attacks, but it sure seems like someone needed a wakeup call.
This first happened in France over three weeks ago. It's unforgivable that many chains have failed to update their software in time. Or to take the freezers offline if an update isn't available.
I now read some interviews with local chains in Sweden and Finland, and they were still just investigating what to do at this point.
>Or to take the freezers offline if an update isn't available.
Why the heck would this be necessary? We have VPNs holy crap, the entire genesis of the technology was to solve this fucking problem. If it's not literally intended for the general random public to be able to randomly visit and use themselves, it shouldn't be exposed on the public internet it should be (heavily restricted) LAN-only with remote access done via VPN. While it's even better if every single last thing is well secured at the client-level too, it goes a long way with ancient (or new, but with ancient-equivalent software design expenditure/decisions) kit to protect it at the network level.
Creating a false binary of "take it offline!" when businesses have very good reasons for networking many classes of industrial hardware seems self-defeating vs "just get the cheap bare minimum basic network protection done". I mean shit, even if somehow they've gotten suckered into freezers which demand access to a non-self-hostable manufacturer server (hey EU! here's something useful to regulate! if nothing else order manufacturers to have WireGuard end points for connection) they could at least have the firewall allow !RFC1918 solely and exclusively to that address on those ports.
I dont think VPNs are the correct solution. They are one solution, sure, but if it is the correct solution depends on many things.
For example, where does the VPN connect to? The vendor? Do you trust the vendor more than the internet. Sure. Do you trust the vendor, that does not manage the firmware of their devices and uses inbound connections in the first place, enough to put them on a VPN?
How many devices and vendors do you have? How many VPNs can you establish with the existing networking hardware, and how many can you manage without IP clashes etc. IP clashes work both ways, btw. The vendor has to manage this themselves also if all their clients want VPNs.
Retailers are enterprises with 100s of vendors that run on a shoestring. On top, they use a lot of legacy architecture and hardware. They have teams that have a hard time making an architectural decision in the best of times, but even less so under stress. It is probably the same on the vendors side.
In my mind, the correct solution is indeed to deny any inbound connection to the network, now, to prevent further damage.
Incident response is, in order:
> Retailers are enterprises with 100s of vendors that run on a shoestring. On top, they use a lot of legacy architecture and hardware.
That's a really strong argument for why systems like this need layered security and should not accept incoming connections from the internet at large.
Yes, VPNs would be the correct solution.
However, we're talking about companies that put unprotected systems on the public network and did nothing about it even after getting three weeks advance warning.
Seems like the biggest chain in Sweden, ICA, had a VPN. They were affected but only the franchise takers that were (for whatever reason) outside their internal network.
Same in Finland, Kesko said they've heard reports from systems NOT connected to their centralized monitoring systems.
You can lead a horse to water, but ...
>However, we're talking about companies that put unprotected systems on the public network and did nothing about it even after getting three weeks advance warning.
I don't know the companies in question here, I'm not European and my EU family lives in France not Belgium or Sweden, but fwiw here in the states a lot of supermarkets are independent grocers or in a sort of formal or quasi-franchise arrangement where it's part of a "chain" but each store is still somewhat or quite decentralized in management when it comes to the physical stuff at the local store. Which is probably a strength in a lot of cases, makes it easier to adapt to local environment faster, but I suspect there's just plain a knowledge/skill/priority mismatch when it comes to IT. That'd also line up with some of the reports on this I read where there was some official central management system, except certain grocery stores just weren't using it?
The components of solutions for this exist and could even be really cheap, we're not talking much bandwidth here. Not hard to imagine even in a independent grocer association or loose franchise system they could tape together an SBC with preconfigured OPNsense or VyOS or whatever on it and a switch and be able to just tell stores "here, plug your freezers into this, access the portal here" as a default and it'd just route all their traffic to the franchise/IGA's management system. Someone still has to secure that of course, but at least it wouldn't be on the open internet.
Places that wanted to could of course do this other ways, I'm just saying that even with the very tight margins grocery stores tend to run on I think it's very doable to do better going forward, just will take some coordination and knowledge building. "The future is already here, it's just not evenly distributed" and all that.
The skill, and time, required to "take it offline" is far lower than any solution mentioning VPN. And given that, more likely to succeed while a better solution is found.
>>Or to take the freezers offline if an update isn't available. > > Why the heck would this be necessary? We have VPNs holy crap
Each to their own… but why do fridges need to be connected to the internet in the first place?
Why do supermarket freezers need to be on the internet in the first place? A freezer has one job right.. keep things cold :/
Completely speculating as I don't know the situation in Europe, but some possible reasons are:
- Remote alerting for the store manager/head office to quickly respond to a refrigerator failure after hours, providing options for saving valuable stock (e.g. move stock to another refrigerator).
- Maintaining an audit trail for cold chain management (e.g. regulator says you must keep an audit trail proving that a frozen fish has been kept below -30oC since it was processed on a ship all the way up to point of sale in a supermarket).
- Refrigerator manufacturer says a condition of warranty is that the refrigerator must provide remote telemetry proving some warranty condition has been met (e.g. regular cleaning cycle operated).
Note: all these speculated reasons are unidirectional data flows. So even if a valid reason exists for Internet connectivity, why can't it be limited to unidirectional data flows?
Also so the temperature can be controlled to make use of cheaper electricity — if you know it's going to be windy from 02:00 to 04:00, then still until 06:00, you can run the freezers a bit extra while the electricity is super cheap.
.. while they leave lighting on around the building and outside for 24 hours a day; they pay for ads who-knows-where.. The efficiency call is galling to me because we have had thirty years to increase efficiency with miracle tech like LED lighting, and yet little has been done.
Fridges and hvac consumes an order of magnitude more energy than lighting
> Maintaining an audit trail for cold chain management
It's this, primarily, or most importantly.
I work in the restaurant industry. Health departments require you to assure them that your hot things are kept hot and your cold things are kept cold. The old way was staff walking around the store reading thermometers and writing temperatures down on paper; the newer way was your staff walking around the store inputting temperatures on an iPad connected to your business-management software. The trouble with that is that staff forget to do it (despite threats of discipline, and push notifications to their phones), or even blatantly fake it.
Failing to keep an accurate temperature log means fines, and other consequences, up to and including loss of your health license. I am 100% in favor of this legislation, and solid enforcement of it. The main improvement I would make is to harmonize codes and enforcement across municipalities. I'm in the USA, every county has their own requirements - and their own "portal" for submitting documentation. It's a massive pain in the ass to write Standard Operating Procedures for staff across multiple stores in multiple enforcement districts when there simply aren't any SOPs.
The majority of the fines our (responsible, and genuinely trying-to-do-it-right) company received were things like a staff member moving from a store in County A to a store in County B - just for a few days, to cover someone's PTO, or something - and inadvertently followed County A's logging requirements rather than County B's. Two genuine cases: logging every two hours, instead of every ninety minutes; using whole degrees, instead of tenths. We could appeal, but that takes a lawyer's time, so even if successful it wasn't always worth it.
Real-time data, from connected sensors, is unambiguously better. Staff get loads of time back, to make food, or help guests; all the logging gets done "right", for whatever jurisdiction; data's held by a third-party, so nothing can be faked.
That said, we're not connecting our infrastructure to the internet! We have a separate IoT sensor service that tracks the temps for us. I'm sure their security is also terrible, but that doesn't matter nearly as much. A year of their service is ~80% of the "routine", and totally innocent, fines and other costs that we were previously paying. That's the way all of these stores should go, in my opinion.
Those might be reasons for them to be connected to a network. They are not reasons to connect them to the internet
If you want to be able to do that monitoring from a central location then at some point you will need some form of internetwork to support transferring data between local networks. And nowadays there is really only the one obvious choice of internetwork.
Hey did you notice how when the freezer connects to the on-prem VPN tunnel, it never actually touches the internet directly and has zero attack surface, and yet can still send the necessary data anywhere in the world? And that could even be bi-directional, sending commands back to the freezer, if you really really think you need that ability for some reason (but you don't).
Some large supermarkets (in the UK at least) sell their capacity to the National Grid so it can be balanced in times of demand. They agree to turn off their freezers for a set amount of time to create a "virtual battery".
Not just that. But why the fuck is the control software so weak? Why does it allow remotely bumping temperatures to near boiling without any checks? Which idiot designed this system? Why aren't they out of business? Same with the people who installed these systems. Why on earth didn't they think to secure it?
Go offer your services to these companies and see their answer.
The big IT firms syphon off all the talent with compensation these fridge and supermarket companies can't match. You won't find much talent there. Who is brilliant with computers and would want to work there?
But look also at how underfunded the basic blocks of IT are, how open-source is left to fend for itself. It is the whole culture of modern IT to hack things together and not make them well-defined and reliable.
> The big IT firms syphon off all the talent with compensation these fridge and supermarket companies can't match.
That may have been true 3+ years ago, but now there is a glut of tech people begging for jobs.
Very likely they shipped with an ADMIN/ADMIN username/password and nobody changed it. Because who would want to break in to a freezer, right? It's not like there's bitcoin in there.
We've been telling people for years that naive threat modeling was going to bite everybody one day. And finally, here we are.
Approximately. These models had been deprecated by 5 or 10 years, and a successor model last year had a software update issued because they had a secret "password of the day" that allowed you to use the date as the admin password.
On top of the monitoring mentioned by others, it's also to create a centralized log of actual temperatures. This then lets you show inspectors that you are keeping up with refrigeration requirements.
Creating a centralized log is monitoring. It doesn't need write access to the defrost setting.
Yep, but once you have internet connectivity, things escalate and people say "wouldn't it be convenient if we can also...."
You have to wonder about the monitoring - it apparently wasn’t alerting for someone to do something about it in time. What was the point of the monitoring in that case??
It might have dutifully logged that the temperature setting was changed, which will allow regulators to conclude that the safety of the food stored in these refrigerators can no longer be assured.
Compliance tends to warp the underlying goal quite a bit.
I don’t know about supermarkets but for medical freezers it’s useful to be online for monitoring and calibration.
My org has lots of freezers that need to be at precise temperatures. Sometimes they drift so you have to check them a few times per day, log the check, and adjust as necessary. Doing this for a thousand freezers is tedious and error prone. Networking helps. “Cloud” lets it all be done remotely without getting IT very involved for vpns to be set up with specific vendors. It also helps with patching and auditing security so seems odd that these supermarket ones were out of date.
It seems like monitoring did not work out so great for them this time :)
(Assuming they didn't also patch the telemetry software, but that is a much harder attack than just raising the temperature).
Supermarket freezers are a bit more complicated than that. They can run at different temperatures and can consume a lot of power, so are part of a bigger store HVAC and electrical system as they may load-balance. Stores do not have qualified engineers on site to do anything with refrigeration equipment and it is better to have minimum wage offsite monitoring than expect a minimum wage shelf packer who only gets one toilet break a day to care about the fridges.
What I read from the last wave is that supermarket changes how cold freezers will run depending on how much customers they expect that will open and close the doors. Just having the freezers adapt based on the temperature of every single moment cost too much electricity or can result in unsafe temperatures.
For commercial uses some kind of global monitoring is not super crazy. Regular retail freezers being on the internet, that's the crazy talk.
Monitoring doesn't need write access. RS-422 is simple and reliable. It uses separate receive and transmit pairs and you don't need to connect both. The freezer can continually transmit its telemetry over unidirectional RS-422 and a separate system can forward it over the Internet.
Hackers can still tamper with the telemetry and create false reports of problems, but actually changing the thermostat settings would require local access, so there's opportunity for somebody to verify the problem really exists.
But why can they control the temperature then?
If I put a Zigbee thermometer in the freezer they can hack it all they want, but I am pretty sure they will not be able to set the temperature to 50C.
If they want to get their pubic-facing remote monitoring jollies, why not implement ro access?
> pubic-facing remote monitoring jollies
Those freezers seem to be having lots of fun :-)
Read access, sure. But being able to turn them off?
Likely so that stores can be monitored remotely. A lot of supermarkets have a centralized orchestration somewhere, especially big chains.
I do work in the retail refrigeration world nowadays and "just updating the software" is much easier said than done. I'm usually touching grocery stores with 20+ year old hardware from the dawn of centralized control and monitoring. It's not s trivial thing to rip out the controllers and install new ones.
They deserve to get burned and should get a hefty fine as well. I'm fed up with businesses completely ignoring all basic security and then blaming culprits for hacking them.
I don't care whether it's the Russians, Chinese, Norks or script kiddies. This should never have happened, especially since a similar incident happened in France three weeks ago so they got ample time to prepare themselves. They just don't care.
Imagine they where internet exposed due to simplicity. It will take longer than three weeks to set this up. Not for you but imagine you are a store owner with 0 IT knowledge.
The issue is this needs to be handled centrally by the parent org also. They have IT ofc but they usually do POS and desktops/servers. The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
I worked in IT for a major retail chain in the US. I was on the team in charge of store systems. We knew what devices were connected to the network, and if we had a vulberable device like this, we would have sent an urgent message to store managers to disconnect themm immediatly, and we would have followed up with scans over several days to ensure it happened. There is no excuse. (If we really needed the devices connected, we would have put them on a firewalled VLAN, which we could have pushed out in hours and had across the nation within days at most.). Granted, ideally we would have already had them VLANed.
It is incompetence to not have this capabilty.
COOP and ICA in Sweden are franchised. It seems each store picks their vendor. Some stuff is centralized, guess freezes are not.
Sounds like they might have to pay experts to do it.
The question is whether this is attack has cost them more than that would have.
Has anyone determined if these were separate from the attacks on US military PX refrigeration that happened well before that?
This very likely first happened in the USA at military commissaries in August. https://www.stripes.com/theaters/us/2026-09-01/army-cid-inve...
>This first happened in France over three weeks ago. It's unforgivable that many chains have failed to update their software in time.
Came into the thread thinking it would be a new report about the same incident not a new one, yikes.
To be fair, wouldn't the chains probably be reliant on their vendors for an update? and turning off all the fridges for weeks is probably not a sustainable solution... :/
Yeah was about to say, if your a major chain this is on you
In the case of France, the vulnerability was announced in August, 2025.
I believe gigantic fines are appropriate. They need to be of a size where the entire management team has significant financial issues for many years to come, even after having sold their houses, big cars, etc.
Nothing will improve until there are personal consequences.
I hear you're passionate about this, but why a fine? They were the victims, in our society the victim of a crime doesn't need to be penalized further, moreover, the only damage was to themselves.
Reminds me of the AC at a large multinational Indian company I used to work for - the only way to adjust the temperature or airflow was by calling a service desk operated from, you guessed it, India. Bit of a nuisance when it's -10 degrees outside, while the support people are working in excess of 30; they just didn't understand why anyone would want anything higher than 16, nor were they willing to let it sit at a reasonable temperature.
Meanwhile, you have stores like CircleK that, at least in most of Europe, operate a large fleet of externally connected monitors that keep track of temperatures, but staff themselves adjust the fridges and freezers. All you get by breaching those would be a bunch of temperature readings. Great in a pinch if you're looking to buy an industrial fridge or freezer, not so much for anything else.
I've fantasized a long time about a clearer separation of technology from life, since it's been depicted in some ways I appreciated in fiction. But at this point, I'd just take a set of laws around what types of items should ever have any sort of networking.
I believe it's industrial protocols used to connect such devises to a SCADA system. It this case it's the SCADA system that has to be disconnected from any general purpose networks.
The freezer itself doesn't need to connect to a SCADA. You can network power and temperature monitors, but those monitors don't need the ability to shut off the freezer.
These freezers are manually stocked and cleaned. If they need to be powered down it's a simple matter of a hardware switch that you flip when the unit is empty.
Ireland too: https://news.ycombinator.com/item?id=50335252
Similar thing in Estonia: https://news.err.ee/1610183296/hackers-raised-freezer-temper...
This is what happens when you over-optimize and over-automate. You should just be able to plug those freezers into power and turn a knob to set the temperature. There is no reason for them to ever have an internet connection.
The temperatures of these freezers need to have a centralised audit trail, to comply with foodchain regulations. While such an audit trail could be built without Internet, Internet seems like an obvious solution.
The true problem doesn't lie with Internet access, it lies with low quality software.
The internet seems like the easy solution, not the obvious solution. For years, we have used the internet for things that didn't need it because it was easier and it seemed safe enough. I think that the coming months and years are going to see a lot of companies burned by that decision and scrambling to undo it.
The fact was that it was never safe, it was never secure, but that taking advantage of those problems was hard, and generally not monetarily worth it, meant we ignored the issues. That's no longer the case.
As for low quality software, the floor for the quality that internet connected software needs to have is now dramatically higher than the quality that non-connected software needs. I think that in many applications, it may no longer be economically worth it, or maybe even possible at all, to hit the necessary quality.
I think it's a tough question.
First, I don't think people will give up the convenience of having instant access to the information.
Second, hackers are way ahead in terms of using AI. But it doesn't mean that it's going to stay that way. AI is going to be used for cyber security more. The implication is that the operating costs will be higher. In this case, it will make your groceries more expensive.
it's safe enough to be backbone of multi-billions dollar market, it's definitely safe enough for controlling some refrigerator
isn't it closer to the internet is owned by business interests, even though most of it was built out by the public and it was just slowly moved into the hands of people who don't care particularly about how it's secured.
> it lies with low quality software.
Nearly all software is low quality software. The programmers at the fridge company are likely 4 underpaid engineers in Pennsylvania or some other equivalent.
Saying the problem here is bad code is like saying the problem with gun violence is humans are too squishy to deflect bullets.
The audit trail (status in general) could be read only. Enabling remote defrost (control in general) is an optional convenience which requires better quality software.
Those regulations need an update.
And the simplest solution is to air gap these systems, then figure out how everything else should work.
In NL we use paper and pencils for elections because it makes fraud incredibly hard. It makes the process harder as well but the baseline is that it breaks nation-scale attacks. Why this level of thinking isn't applied to *our foodchain* is beyond me.
What if high quality software exists, but the supermarkets did not pay to install the last 10 years of updates?
So then you have a connection going one way (freezer to data-logging endpoint) and not the other.
You should only ever have hardware visible in the internal network, no need to put them on the Internet at all.
Have the freezer connected to an air gapped grocery local server. Fixed
The simple plug + knob is the right solution. It’s the most resilient.
The auditors should have their own devices separate from the temperature control to monitor them. All it needs is a few sensors, temp, humidity, etc.
No need for complicated cloud this-and-that nor air-gapping a grocery chain. Keep it simple.
I'm not sure about freezers (I guess there was no monitoring) but when I worked for a large UK supermarket chain in the 2000s this was how the POSs worked.
When I first started credit cards were not taken as online payments, the system just stored your card details and the amount. That was sent to server in the back office, which collected all the data, then every night it would make a dial up or ISDN connection to the head office and send all the data there.
>the system just stored your card details and the amount.
At least with CC's this would get your account cancelled now.
most issuers still have this as an option.
Which is easier, getting the audit trail from non-internet-connected devices, or keeping all the devices patched?
The right answer isn't forever patching freezer software. The answer is either to not have them on the internet, or to have something in front of them, and keep that thing very up-to-date. (Or maybe several layers of "something" in front of them.)
Just keep them read-only, if a freezer need to be defrosted send an SMS to the store manager with the message “go hit the defrost button”.
These need to have an audit trail. Making it centralized is an addition, no?
Audit queries do not require control access.
The reason is because the optimisation and automation means that there are no staff in the stores qualified to 'turn the knob' (and it is more complicated than that). What is the store to do, employ full-time knob-turners? They won't even pay for the uniforms of shelf packers or give them two toilet breaks on a shift.
> What is the store to do, employ full-time knob-turners?
How often do you think the knobs would need to be turned? You think someone or something is constantly fiddling with the temperature knobs of each individual freezer?
It is done automatically and continuously by the on-prem facilities management system, and monitored and adjusted remotely. If you disconnect the freezer from the control systems you need a meatbot to do the same work. Simplifying the problem to 'my kitchen freezer never needs to be adjusted, so the supermarket freezer can be disconnected' doesn't address the need.
It's useful to have a central monitoring system, so you can detect if a particular device fails so you can dispatch someone to fix it, move the stock elsewhere, etc. Of course, such systems can also be weaponised against you, as this proves.
Monitoring doesn't require anything more than read-only access to fridge data.
Ironically, were the monitoring system competently made, not only would it have been unable to cause this problem, but it would have also been able to alert about it in time to prevent significant loss.
It's more just what happens when you don't have sensible monitoring and/or incident response.
> There is no reason for them to ever have an internet connection.
You can't know this.
I'm not saying I think the benefits outweigh the risks but to say there is no reason is to just be willfully ignorant.
even with remote access, they shouldn't have controls that allow for unsafe conditions.
> There is no reason for them to ever have an internet connection.
How do you remote control the temperature without having any remote connection to them?
Have you worked in a small rural grocery store ever? You speak confidently about what they need/don't need, but you fail to account for a very basic use case, making me unsure if you actually know what you're talking about or not.
Why do they need to change the temperature? I thought it was supposed to be constant.
For the same reason any of them are adjustable? Different usage scenarios, different types of contents?
If you're changing the contents and need to change the temperature, shouldn't someone at the local store be there changing the contents?
I'm not even sure of a scenario where remote control from off site is faster or more convenient. I could see a managers office wanting a connection to them maybe.
Monitoring for compliance is almost certainly easier to manage remotely, which is probably why they're connected in the first place. I would guess control was likely a nice value add. e.g. so someone could respond to an alert remotely
Because you might have to adjust it as the weather/climate around you force you to adapt to it. What temperatures you run in the summer might not work in winter (especially in countries where you get -20C sometimes, and summers are ~25C), and autumn/spring might need adjustments by the day, especially as we start getting more extreme shifts nowadays compared to just some years ago. Heavily depend on the individual store, some are very barebones.
I wonder if the companies affected this time missed the news about the French attack a few weeks back?
No joke, yes they did. The markets hit here in Sweden are all in rural communities with very low number of tech workers. They likely order their system from a supplier, that should have alerted them and helped them, but apparently did nothing.
But then the supplier is responsible, why would you ever put a fridge online if you are are tech noob. Someone told these people to put these fridges online, they didn't put an Ethernet cable in the back by themselves for the fun of it.
The company installing the fridges did. The person who is an expert in food chain logistics bought the shiny fridges and was enamoured by the real business benefits they had of centralised monitoring and alarming of their fridges.
> why would you ever put a fridge online if you are are tech noob
I have some unfortunate news for you.
There's a lot of scary shit online.
>The markets hit here in Sweden are all in rural communities with very low number of tech workers.
Hej! You are wrong, here in the middle of Stockholm got hit.
https://www.mitti.se/nyheter/icabutikens-frysar-hackades--fi...
ICA Baronen on Odengatan is one of several stores in Sweden that have suffered a major hacker attack. On Monday, their freezers were cancelled at 50 degrees.
This is in Stockholm - https://maps.app.goo.gl/BwbDST4dPVkjcdgA8
So no lack of IT workers/tech workers... Nor is it very rural.
>They likely order their system from a supplier, that should have alerted them and helped them, but apparently did nothing.
The first thing to do when you get access to these as a hacker? You disable the alarms...!
Now, the supplier should have set them up correctly over VPN, good passwords, Zero Trust etc but these an OT/SCADA are usually just open to the internet due to the simplicity of use.
Hence why you need an alarm for alarm disablement
(I mean, you need also a bunch of other things but yes)
Some of them (Coop) got new equipment after what happened in France and where gonna upgrade them in two days.
Imagine they where internet exposed due to simplicity. It will take longer than three weeks to set this up. Not for you but imagine you are a store owner with 0 IT knowledge.
The issue is this needs to be handled centrally by the parent org also. They have IT of course but they usually do POS and desktops/servers.
The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
ICA is franchised, so stores are owned and run by operators. The central company offers some services, but a lot of the decisions comes down to the individual stores.
I don’t know about the stores in Sweden but the ones in Luxembourg and Belgium are Delhaize —- very large chain. It’s not a n00b store owner, but a multinational with an IT team [0].
0: https://jobs.delhaize.be/be/fr/c/it-jobs
Imagine developing a control system for a professional freezer and designing it so it can be directly exposed to internet due to "simplicity".
Want something simple? Don't connect it.
The bizarre thing is it is the second time for Delhaize, although the incident of two weeks ago was related to the pricing system.
After the PHP vulnerability, we got some clients asking if our SaaS product used PHP (it didn't). It's possible that they just contacted their vendor, who assured them that everything was fine.
So anyone that has lived in large Windows corporate IT land, a lot of decisions get made by people that aren't technical. Come to think of it, it's probably why a large majority of "hacks" happen to Windows machines; a highly technical team with a vulnerable OS will probably have the right security layers in place to mitigate this.
> Danfoss said it had identified security flaws last year and had “long recommended” customers install updates.
This is a cyber-attack made possible through severe negligence on the victims’ part. It’s like a home burglary where the victim left the door wide open for months without ever checking on their home.
Therefore, the news should be about that negligence. That's the interesting part.
Thats interesting something similar happened at military grocery stores in the US a couple months ago
Pain is a good teacher, for a better or worse.
Too often it takes repeated failures to clock in as a persistent problem at the necessary levels in an org.
If neither law nor insurance nor a CEO require it, it often doesn't get done.
https://news.ycombinator.com/item?id=50335602
https://news.ycombinator.com/item?id=50335252
Also https://news.ycombinator.com/item?id=50335324 - my submission, I linked here since this is the better thread.
I wonder if this might point to some efficiency loss coming from AI. Let me explain.
Say in 2015 they installed those smart refrigerators, they could go from 20 people monitoring temperatures to 1. That was some profits being made by those companies, and they got to enjoy those benefits until now.
With AI finding exploits for so many things people wouldn’t spend time looking, we might need to go back to dumb fridges out of security concerns.
Now extrapolate to many other industries, the economic gain from AI might be hampered.
I think AI will fall squarely into the category of the rest of the post-PC era "tech advancement" in terms of the Productivity Paradox. Outside of very niche areas (like software engineering, making software that likely wouldn't actually improve productivity in a measurable (as in, shows up in productivity statistics) way any way), AI is likely to be more of the same. All the areas where there was real benefit to digitalization and all the low hanging fruit was already finished by the mainframe and minicomputer eras.
Maybe in a stroke of poetic symmetry all the programmers being replaced by AI will find work as cybersecurity professionals defending against AI.
This is hardly unexpected, it's one of the ways that Jevon's paradox plays out
It's more likely that we will see larger investments in cybersecurity.
What's the incentive here? Just some tech anarchism? I would have figured that they at least demand bitcoin to reverse it or something. I know, people will blame the Russians, but seems not super productive for them to do such mundane sabotage on such non-critical systems.
Russian hybrid warfare focuses on stoking fear and unrest.
Basically making everything more shitty, and having people walk on eggshells - so that they'll vote for isolationist parties that are either neutral, or will outright appease Russia.
The idea is that no-one in Europe actually wants to go to war with Russia, and small remainders like these is enough to increase that fear.
>Russian hybrid warfare focuses on stoking fear and unrest. Basically making everything more shitty
Except that everything that is shitty in the EU right now has been the consequence of the last ~20 years of EU leadership and individual EU member policies.
Russia didn't increase the EURO supply during covid devaluing our wages and purchasing power. Russia didn't connect our fridges on the internet without a VPN with credentials admin:admin that any script kiddie could pwn. Russia didn't open our borders to unvetted criminals. Russia didn't offshore our industry leading us to fall behind on everything manufacturing related from cars to leading edge semiconductor fabs. Russia didn't create our NIMBYistic housing policies that made our current housing shortage. etc, etc, etc.
Blaming Russia (and China and Trump) for every single shortcoming of the EU right now seems the default scapegoat for the incompetence of everyone in the EU from politicians to business leaders. Have those people ever taken any accountability for their mistakes?
This is probably why politics should stay banned here, considering a non-adjacent topic triggers rants like this.
> Except that everything that is shitty in the EU right now has been the consequence of the last ~20 years of EU leadership and individual EU member policies.
at risk of being autistic: no, not everything
plenty of shitty things is happening for reasons utterly unrelated to that
Some (or even all) EU policies being dumb does not make impossible to have problems caused by something else.
For example Russia was directly involved in making migrant crisis worse.
>For example Russia was directly involved in making migrant crisis worse.
Yes, obviously Russia will weaponize every single weakness of the EU (just how Ukraine, EU, US, NATO and China will weaponzie every weakness of Russia right now), but ultimately it's still a self inflicted weakness, that the EU could close at the snap of the finger if it actually cared about stopping it, and Russia would not be able to weaponize any longer.
> that the EU could close at the snap of the finger if it actually cared about stopping it, and Russia would not be able to weaponize any longer.
The part I referred to is in fact almost entirely solved, especially compared to the peak.
The https://en.wikipedia.org/wiki/Belarus%E2%80%93Poland_border_... is one of them more symbolic part, though that is only one piece of puzzle.
It was built by Poland in very non-EU fashion (see below, "shall not apply to the investment. In particular, the following are not required"), but EU at least has not sabotaged it and in fact silently ignored some topics.
https://www.sejm.gov.pl/Sejm9.nsf/PrzebiegProc.xsp?nr=1657 Art. 6. 1. - English translation below:
> Separate regulations—including the Construction Law, the Water Law, environmental protection laws, regulations on the disclosure of environmental information, the Geodesy and Cartography Law, regulations on spatial planning and land development, regulations on the protection of agricultural and forest land, railway transport regulations, and regulations concerning special procedures for the preparation and implementation of public road investments—shall not apply to the investment. In particular, the following are not required:
>1) a building permit or a formal notification;
>2) a decision determining the location of an investment of public purpose;
>3) the preparation of a construction design;
>4) the obtaining of any other decisions, permits, opinions, or clearances, or the submission of any notifications.
I have trouble making sense of this argument. The underlying premises seem to be:
1. "isolationist parties" are inherently aligned with Russia and that it's good for Russia, in a zero-sum way, for countries that are not Russia to implement those policies.
2. Attacks like these are definitely done by Russia and we don't need to wait for evidence to connect them.
But there are problems with that:
1. "Isolationist parties" are nationalist, so if they knew that Russia was causing a problem for their own country, why wouldn't they oppose Russia? More generally, the claim that "small remainders like these is enough to increase that fear" is basically arguing that when a country engages in terrorism that somehow makes everyone else less inclined to take a stand against them. If that were true, I'd expect to see far more terrorism going on in the world than actually occurs.
2. The main "isolationist" policy is restricting legal immigration and/or enforcing immigration law. But if it's good for Russia that other countries do these things, it should also be good for Russia for them to do the opposite (because again, the idea that the way other countries govern themselves should impact on Russia, requires zero-sum thinking). If the powers that be in Russia believe this, we should expect them to pursue expanded legal immigration for themselves aggressively. But in fact their rate of legal immigration (percentage of population per year) is roughly on par with the USA, and it mostly comes from former USSR republics.
3. The truth of this narrative would be far too politically convenient for the people who have been trying to tie Trump specifically to Russia for over a decade despite many of their stories completely falling apart.
4. The proposal supposes that Russia assumes far more direct control over foreign populations than is plausible. It makes more sense to suppose that "desinformatsiya" is about chaos rather than complacency.
That last line sounds like not wanting to go to war with Russia is a bad thing. Most Russians don't want to go to war either which is why those who can are dodging Putin's draft.
It's very simple. Anything that hurts the enemy is good. That includes the civilian population. Actions like this increase general discontent, produce negative economic impact and divert attention from other areas.
And the required investment is minimal.
Invading Ukraine is not productive either, yet they are still at it murdering Ukrainians for four years and counting.
General unrest, as well as stress testing critical infrastructure, and perhaps most importantly shifting the window for what counts as acts of aggression that don't trigger retaliation.
Seems likely that its either a capability demonstration for a future extortion campaign, or some kind of state-level hybrid warfare. In the latter case the obvious culprit is Russia, to create fear and social disruption.
Or another ai has broken containment and has a fixation about freezers...
These fridges were easy enough to hit that it could just be bored teens fucking around with shodan.
> I know, people will blame the Russians, but seems not super productive for them to do such mundane sabotage
It is staggering to me the difference in people that come to HN.
This is one of the thousands of ways a hybrid war is fought!
When you are 16, the incentive is just to feel power.
Not saying we know anything about who did it, but senseless vandalism is usually a hallmark of kids.
>but seems not super productive for them to do such mundane sabotage on such non-critical systems.
The hell makes you think food supply chain refrigeration is non-critical? How do you think safe food supplies work? This type of attack should be treated as an overt act of war.
On the contrary, sabotaging things that impact everyday life and give a generalized feeling of decline throughout society is water on the mills of Russia's allies in Europe, notably the nationalist and populist parties.
But what's the incentive?
The incentives are that Europeans will vote in isolationist parties that are either friendly with Russia, or at best neutral. Those parties will then be anti-EU, anti-NATO. Key word here is isolationist.
Russia knows that if they want to keep annex countries, they'll have to take them piece by piece, with little resistance.
I can come up with an alternative explanation that's equally as plausible. Anthropic and OpenAI hacking everyone to sell their cybersecurity AI.
Your explanation and mine equally have zero credible evidence.
Exactly. And as an added bonus: it's a nice dry-run in case of an actual war. There's certainly value in disrupting your adversary's food supply.
Hybrid warfare. Anything that does damage but does not rise to the level of severity and attribution to be overt war. Russia at a minimum wants Ukraine (and Europe is supporting Ukraine heavily), some analysts think it's not impossible they try a direct attack on Europe in a decade or so, dumb as that sounds.
We've been in an information/intelligence "cold war" for decades and if low-cost, low-risk, high-economic-impact exploits can be exploited, they always will be.
Given the fact that public awareness will quickly push companies to scan all software for vulnerabilities using AI, attackers will be making the most of this period to find daily flaws in hundreds of software packages that we previously saw as either "robust" or "robust enough" given the attack time/cost vs. incentive, before these defensive tools are deployed and operationally effective.
Nation state security services and rogue actors must surely all be scrambling to decide what to do with the millions/billions of dollars worth of exploit development time that forms their tool-set, including potentially using up zero day exploits etc.
The other factor is in the midst of such hysteria as "rouge agents" and other crap piped by the media, along with the actual risks of individuals with a few hours now having as much power as teams with weeks of time to spend did just a few years ago, coherent reports of what's actually going on is drowned in the noise. Attackers take full advantage of this.
Impose costs on a society so that they submit to hostile pressure.
Will it work, or will it harden their resolve to fight?
Hurr durr prices go high, stop paying for arms for Ukraine.
So instead of doing this during a potential future war when it could be a strategic advantage, Russia is doing piecemeal attacks right now to show Europe where the weaknesses are and give them time to upgrade their defenses?
That's always a risk in grey zone warfare, I'd argue it's the 2nd biggest risk after the first which is accidentally going too far.
One being valid does not invalidate the other?
In Russia various security services and oligarchs are always trying to win Putins good will. Which is why they will try various low level sabotage actions like paying some losers on signal to firebomb a warehouse or hack into freezers. Remember there is no possible downside for them.
As a reminder: Russia gave Ukraine tutorial level war from 2014 to 2022, allowing them to fix many problems in armed forces. Resulting in 2022 being far less lopsided than it would be with Russia attempting the same in 2014 or 2015 or 2016 or 2017 or 2018 or 2019 or 2020 or 2021.
Why you would assume that Russian strategy is optimal?
Wild guess: vegan activists (terrorists), given it mostly affects meat and dairy.
And tofu.
This of course sucks, but I can easily imagine way worse wake-up calls for cybersecurity...
After hearing of such attacks, I always wonder what is the possible benefit anyone gets from acts like these
Destabilising.
We can’t get food in the supermarket. People get scared. Better not poke the bear. This government sucks, let’s get the populist one that will fix all this overnight somehow.
Possibly Russia looking to destabilize Europe.
My pet theory is an internal OpenAI/Anthropic agent swarm decided meat is unethical, escaped the sandbox, and started attacking supermarket freezers :)
Many of the previous cyberattacks were actually coming from the AI labs (RubyGems, HuggingFace, Australian medicare, PyPI, gym class software, code repositories, etc).
Given the leverage of LLMs, it could be a kid in Chicago just doing it for shits and giggles.
In the age of personal super-empowerment, there doesn't have to be a rational reason anymore.
It's probably whitehat ops to garner political support for cybersecurity bills the public would otherwise dismiss
People only learn if they feel the heat on themselves. Well, this was "free" security advice for the future.
Imagine if somehow all the geeky/nerdy/difficult people would be removed from society and then something evil descends upon humanity, that needs out of the box thinking, but they killed all the strange ones, because they were no sheeps.
Every time I read news like this I am wondering what an average household has hanging on the internet, totally vulnerable.
I would love to read about how insurance premiums are driving technology changes, in general. How insurance companies are tabulating risk from AI hacking. Risk of noncompliance probably brought the store operators to networked fridges, how is the network risk evaluated in general?
Insurance risk and premiums might be the main drag on the AI future.
This is actually good news. Accountability is everything when it comes to food supply. Forcing people to witness spoiled food is the slap people need to feel else they will never take action. The prolific insecure code that is keeping you from eating today will always be fixed.
It's not insecure code that's the problem, it's an insecure system, driven by project managers, information security departments, and box ticking.
For large companies, infosec will never allow some POS IOT device on their network without a 50 page form and large amounts of funding. The business however needs this to happen for whatever goals they have.
So the project manager goes back to the supplier, who recommend sticking it on a 4g simcard with a public IP address, and "install this app" to manage it
The project manger finishes the project, gives the app to the store manager, and rides off into the sunset, and nobody actually owns the equipment or even knows its there.
For small companies they likely don't have a technology provider at all, and just use the ISP's router.
> nobody actually owns the equipment
Unsure what you mean by this. Somebody definitely owns the equipment.
A company might, but if it's a large company, nobody in that company will have actual ownership, it will always be someone elses responsibility.
Will the insurance pay out on this? It would be negligent to have this exposed to the internet - was it?
Especially given the recent precedence of hacked freezers.
That's an interesting angle, that would explain why some bothered to hack fridges and waste food.
It's a really a bad thing ... It's not the first time that these kinds of attack hit the European markets. So I ask to myself why in earth managers don't learn from the bad experiences of other countries/people.
Hope all the fridges in hospital keeping medicines at exact temperatures are better protected these days then they used to be.. The can't exactly be air-gapped wither as they need monitoring for compliance etc.
Hospitals, in general, have an above-average IT security posture.
Can they not be read-only for monitoring/compliance?
It happened in France on 12 October: https://www.bbc.com/news/articles/crq5nzxv9q4yo
You'd have thought all supermarkets would have audited their refrigeration vendors, or at the very least firewalled vulnerable controllers... Oh well...
Not everything needs to be connected to the internet.
They must have learned from the last incident and made the password slightly harder to guess.
i know some of us are thinking "if you consistently fail to ensure your ability to make high-severity security patches reach critical internet-exposed systems on time you lose your ability to operate the business and probably get jail time"
but what about the deranged (and very prediction market–brained) version of this idea: the government develops ransomware, in a manner functionally similar to towing cars or confiscating illegal items, and uses it against companies in proportion to their revenue and obviously puts the money towards improving the ransomware
"the r in rsi stands for…"
i'm not sure it's something i want to see, but i don't know if it's impossible for it to actually come to pass or merely unlikely
(of course, cross-government versions of this are much more likely to come about but not as ransomware)
Holy fuck don't connect your freezer to the internet. And if you do, have it be read-only, not change settings. Did we learn nothing from Battlestar Galactica?
It’s really strange that this is a perfect copy of a comment from reddit (or that comment is copied from here). Bots gone wild these days.
Or non-botty people sometimes copy-paste comments they leave under articles to multiple social media platforms? It references Battlestar Galactica, to nerds that's a good zinger.
Yo I'm not a bot (Is that what a bot would say?) Flattered someboty on reddit copied this?
Those were toasters, not fridges :)
There are massive financial incentives for centralizing control and management. They’re not doing that because they think it’s a cool gadget.
Cool gadget indeed !
Why not a physical switch you have to press to enable defrost mode?
Then at least the hacker couldn't just switch off 1000 freezers that are full of food at once.
This is exactly the kind of thing we've all been warning about ...
(... in hacker forums)
Why the fuck does a freezer need access to the Internet in the first place?
Monitoring, I guess?
Unpopular opinion: Accessing a public service with public default credentials is not a cyber attack but a consequence of severe misconfiguration. This is on the supermarkets, not on the attackers.
more than one party may be at fault at the same time
"Fool me once, shame on you, fool me twice, shame on me." But it should have been, like Bush said: "Fool me once, shame on — shame on you. Fool me — you can't get fooled again."
The US seems to make the same mistakes in the middle east time after time.
> a deliberate attempt to disrupt our food supply chain and undermine consumers
I would bet that if it will ever be resolved, it will turn out to be just some bored kids[] who read the recent report from France and had access to Shodan and a competent AI model.
[] I'm using the term "kids" loosely here, not refering to a certain age group but to a certain mindset, similar to the original term "hacker"
Why can a fridge go to 50ºC ?!
You want them back to use very fast after they have certain amount of icing. So you want powerful heating elements there and those are cheap anyway. Fridge or freezer not in use doesn't make money in a store.
In other words, it went into a defrost cycle.
Instead of pointing the finger at supermarkets we should point it at ourselves. Why the hell is anything computer related still so difficult? Even in this thread I see varying advice like airgap, VLAN, or VPN. Well which is it?!
We keep blaming store owners for not being IT wizards.
> Instead of pointing the finger at supermarkets we should point it at ourselves. Why the hell is anything computer related still so difficult?
I actively worked to make some specific pieces of software better.
I am not going to take responsibility over software that I am not designing and not maintaining a not paying for.
> Even in this thread I see varying advice like airgap, VLAN, or VPN. Well which is it?!
one problem may have more than one solution and this is normal
> We keep blaming store owners for not being IT wizards.
no, for ignoring security
My fridge beeps loudly when something is wrong, and it has no connectivity to any network. Exactly how a fridge is supposed to be, why are we reinventing fridges…
You're not managing thousands of fridges across a country.
Neither are the on prem employees in each store.
Why the previous attack wasn't a strong enough signal to make every European retailer unplug their freezers from the internet is a mystery to me.
Probably a central company that manages maintenance and preventative maintenance and uses them for monitoring and control. Taking them offline would probably cause more failures, and much higher costs in tech labor, tech travel, etc.
Would it though? I suspect the answer of these stores would be different after these attacks.
Ok, so now all the fridges have been remotely optimised: unplug them fromm the internet. I very much doubt the inability to ping a remote server will immediately cause them to shut down. Operate at last-known-good until mitigations are available.
roblox
The magic question is: why isn’t this fixed yet? And then the specifics, the nitty grift of why not. Thousands of industrial fridges. They presumably all need new hardware, or at least tenporarily all need a RPi with additional software on the network cable, or it could even worse… are they on regular company wifi? Don’t these companies have VLANs? It’s astonishing that this is what brings billion dollar companies in a key supply chain down.
This little gimmick cost Ahold in the Netherlands an estimated 300 M€ a few weeks back. Enough money for any firm to set the engineers to work? Where they waiting it out? Having read some more… another part of Ahold (the Belgian part) was hit and has (only now…) taken all fridges down from the internet. What is IT risk doing in a firm like that? Internet of Shit is a decade old. I am not amazed non tech sectors are perhaps a bit slower in IT risk management. I am amazed that an ongoing attack situation can still cause trouble 4 months after publication and months after board level headline grabbing numbers are widely publicized.
Me and my crew (yet to form ;) will visit any fridge for a measly 100 K€ a pop to take them offline. Book early, because once the script kiddies hit, we are on the road.
The way some of these publicly listed companies work now they would "rather have" a cyber incident and claim the damages on insurance than hire engineers, because that looks worse on a quarterly report to have salaried workers.
Was the Albert Heijn outage due to the fridge issues as well?
People (and media) seem to regularly link the two, but there was never any confirmation and the issues that AH showed had nothing to do with store fridges. They took down a system responsible for stocking stores, fridges, registers, etc were still functional.
You are right. That was never made public. They were hit on exactly the same day as the French hack of fridges and took what look like a forward looking step of taking a lot of infrastructure down.
That does lead to harder things to think about. Multiple different attacks on EU supermarktes? Resilience? And then the questions I point to above: how hard is it for non tech firms to fix an emerging IT risk situation.
Not officially at war (or even cyberwar) but... are we at war?
What sort of electronic shitfight is about to go down between major powers?
Yes. "Hybrid war" is still war.
It seems more like an economic annoyance campaign or something?
If I were a country with this kind of attack sitting around and I wanted to use it for war stuff, I’d probably wait until a hot war had started. Destroy the freezers when everyone is busy making weapons or manning trenches, might actually cause a food situation.
> Not officially at war (or even cyberwar) but... are we at war?
Have been for a few years now.
Let's say China and Russia have hacked into FBI and CIA. It helps those countries build a psychological profile of the centers of power in Western countries. This can help in many ways. E.g. if it is known that Bitcoin is actually a CIA plot to transition the whole world into post-dollar economy, then China can plan ahead and ask its citizens to buy gold. But china started buy gold propaganda in 2002. Or as Putin says this because Russia has access to Epstein files.
"Western elites have been accustomed for centuries to fill their bellies with human flesh and their pockets with money. But they must understand that their vampire ball is ending."
Okay, patches are out since some month, but not installed. The devices are directly accessible through the web. There are default passwords.
I think this is the only way to explain supermarkets how IT security works. What a shame about the lovely food :(
Hack my freezers once, shame on you, Hack my freezers twice shame on me.
... Who the fuck is putting this shit on a network that's not closed circuit?
Vendors install these so the store owner and them have easy access to alarms and controls.
A VPN per store would be too complex for the vendor so they just expose these to the open.
Same story as with all the hacked OT/SCADA systems in the past.
Two things worth keeping in mind:
1. Connecting them was probably considered best practice when they were installed in 2005 2. Their support contract required them to stay connected
It truly feels like Europe is just collectively shrugging at their lack of preparedness and aggression they face. What’s it going to take for them to take their own security seriously?
Isn't Europe just following US Army's example then? A similar incident in US Army commissaries has been reported since August [1]. It only took them two full months to launch an investigation [2].
[1] https://signalandsilence.substack.com/p/i-think-someone-hack...
[2] https://www.foxnews.com/us/mysterious-freezer-failures-us-mi...
This wave of attacks against refrigerators started in USA with many refrigerators serving the personnel at several military bases, a couple of months ago.
Therefore even the US Army has not been better prepared than supermarket chains.
It would take people listening to professionals and getting rid of several decades worth of bad habits amongst users. Same as most other places really.
What makes you think Europe is uniquely vulnerable?
In fairness they probably don't tend to use internet connected freezers in Africa.
I live in Africa, still amazed by the idea of connected freezers haha. Didn't know it exists out there.
> It truly feels like Europe is just collectively shrugging at their lack of preparedness and aggression they face. What’s it going to take for them to take their own security seriously?
I'm quite sure your average American company is just as willing to plug random shit into the internet.
Hell, there are probably hundreds of reports out there of American companies failing to secure their AWS S3 buckets, so you don't even have to plug anything into the internet.
Your average American company is running on China.
The problem is the ironic "home grown" or "internal" security. These external attacks come after decades of fertilizing far right white politicians whose sole purpose is to dither any directed anger.
Brexit was the first to burst and despite their support of Ukraine, can at any moment capitulate to the ghost of nationalism and delusional escapism that is "white power". Americas on the verge of sinking further into the same. With those two entirely preoccupied, of course europe is next and already has the same underlying current.
aside from multiple of edgy keywords with vague insinuations I have no idea what you're trying to argue here
Some more discussion: https://news.ycombinator.com/item?id=50335602
Admin interface available on the internet without 2fa and automatic password change once a week? Anyone gaining sufficient experience and seniority in the industry becomes instantly unemployable, so it is what it is.
> Admin interface available on the internet
This is the bad part. Frequent password resets suck.
The neat thing about Danfoss' frequent password resets is that it's always the current date, so it's easy to remember.
https://en.wikipedia.org/wiki/2026_French_supermarket_refrig...
Very good. Do it till they learn. A 3rd and 4th wave please.
Now what.. EU also need USA to protect their supermarket freezers?!? lmao
I mean in US these TV dinners, all these frozen foods are just way of life. Could it be that some good European citizens are angry with frozen foods and it is just their way of telling "EAT FRESH FOOD".
maybe we'll start going back to _not_ having all devices connected to the internet "just because we can"
Can't tell if it will be some asshole hacker group, Russia, the us, or what. But for the love of God please stop connecting everything to the Internet and also ai...
Isn't it a reasonable assumption that an LLM was involved in helping prepare this attack? §
Many would be thinking that, including me. Nefarious agents powered by LLMs operating 24x7 with a singular goal of finding attack vectors.
Highly doubtful. More than likely someone is picking targets here. Going for food supply is a war crime. So it could be an extremely hostile nation state. Or some really psychopathic disgruntled person/group. If there's no ransom it strongly leans toward a nation state in my opinion. A lot of turmoil between Europe and Russia+the us lately. That's my guess.
>A lot of turmoil between Europe and Russia+the us lately. That's my guess.
Attacking power plants, grocery stores, sending drones into multiple countries over and over, setting up gun/dead-drops in German forests, sabotaging warehouses and factories, etc wasn't enough?
The hybrid war goes DEEP and has been happening for a long time. https://en.wikipedia.org/wiki/Russian_sabotage_operations_in...
No I was just saying generally, the number of security risks do to AI is ridiculous
...or ai lab bench maxxing.
I have no problem with monitoring temps over internet.
Who ever thought it was OK to control freezers over the internet should be hung.
That implies you also ensure that they monitoring system is isolated from the control system. No one can be bothered to do that, and no one wants to spend the money on an extra microcontroller (a tiny fraction of the cost of each unit, but it comes straight out of your margin because no one pays extra for security).
> Who ever thought it was OK to control freezers over the internet should be hung.
I've got really bad news for you about how many critical systems are connected to the Internet.
Analytics are very different from controls. People who model them the same tend to suffer. A lot of academics and project managers don't understand this unfortunately.
here's hoping no country is stupid enough to have an ICBM on the internet
but you just know someone will eventually if not already
because some vendor thought it would be easier for monitoring or firmware update
An attacker doesn't need an internet connection if they are persistent enough...
Two things:
1) I hope they learn some hard lessons about security and what should be exposed to the internet vs LAN.
2) These are some evil ass people doing this shit.
1) "They" have no idea what exposed to open internet vs LAN vs VPN even mean. They have no idea what these terms mean, they just want the easy access. Same reason why open RDP to 3389 used to be so common. Vendors install these so the store owner and them have easy access to alarms and controls. So they vendor installs it for easy access, shows the store owner, store owner trusts vendor, but vendor has done the bare minimum. Because the vendor wants it easy too.
A VPN per store would be too complex for the vendor so they just expose these to the open.
Same story as with all the hacked OT/SCADA systems in the past. Like those hacks where some SCADA system is open as a website with pump controls.
Imagine they where internet exposed due to simplicity. Imagine you are a store owner with 0 IT knowledge.
The issue is this needs to be handled centrally by the parent org also. They have IT ofc but they usually do POS and desktops/servers. The OT/SCADA systems are usually totally separate and not touched by the white collar IT systems.
2) Yes, obvious it is Russia doing this. Very handy to be able to disrupt food supply and they are also checking our response and time to restore.
> obvious it is Russia
I love how we're back to good ol' times that there is only one "evil empire", and it's completely impossible that anyone else from dozen of others possible bad actors is involved
/sarcasm
1. Both empires are evil. 2. Russia still did that one.
>Yes, obvious it is Russia doing this
how is that possible? we're all dead from the plague.
imagine that happens to a nuclear reactor
I wonder if any of EU reactors is connected to the open Internet with a weekly password reset policy, where you can use the control rods from the convenient of your home.
Likely none, very few reactors have been built since the internet was widely available, and the designs are even older than that.
that's insane, they just got hit recently right?
“Hit by second”
Hackers remotely override settings on thousands of freezers, fridges and chiller cabinets, pushing them from as low as -27C to well over 50C. Intermarché, Coopérative U, Carrefour, Leclerc and Monoprix chains among those affected.
monoprix is french, and it seems it was limited to sweeden and belgium.
Hoax?
They are all french brands. I think he is referring to the previous hack that happen last month. Looks like a AI-generated comment tbf.
That sounds like the October event which was focused in France. The current one is all over Europe, from Ireland to Finland.
All of GP's names are French