Power cut in Spain and Portugal

[5U4GB]

While I am sure its more complicated then I understand it to be, i still think there is a gold standard for engineering design where it tries to agree with common human principles, like less being safer/easier to deal with, etc.

Russian design philosophy is to have equipment dumb, operators smart.  Compare a Russian and German samovar as an example :-).

[paulca]

As nuclear power plants cannot be shut down (*) nor really modulated, the huge share of solar energy weakens the whole grid.



(*) except for security/safety reasons, of course

Um, actually; traditionally nuclear power plants are shut down during major blackouts.   The cooling pumps and other important systems are run from grid power and when that's gone they have to shift to backup generators and shut down the reactors.  Once shut down nuclear power plants take the longest to start up.

Why are they not able to run pumps from their own electrical output?

I believe at least some of them can it is probably a matter of always having a backup for the backup and that they have to shutdown regardless to get the output low enough to handle no load

The problem of "loss of grid" is infamously tricky for nukes.  When the grid drops the load off the plant the generators MUST trip before the resistor banks overheat.  (Seconds)

If the generators are tripping the load will come off the turbines which will over RPM and destroy the building in a matter of minutes. 

So the steam has to be diverted and then vented.  There is only so far you can take this.

LONG ago in this process the reactor will have auto scrammed.  So it's thermal output will drop from maybe 1000MW down to 10MW nearly instantly.  It will then cool along a standard curve, rapidly at first, but all in all, while it drops to <1% output very quickly, even after several days it is still producing 100s of kW of heat ... that has to go somewhere or Fukishima.

At this stage, outside of the infamous "generator run down tests" (specifically designed to see if the plant could run on the power from the generator after it tripped) the plant is on battery power until the generators start.  The generators and batteries provide adequate power with margins to run the cooling required for the shutdown reactors and fuel pools.

I don't think there is any practical way to energise the generator at such a low level, get it up to RPM and into some kind of phase and even stable RPM.  These are 400MW+ generators.  They don't do 10kW to run the plant.  They just don't.  You could I suppose put a small turbine on the vent steam and generate some power there, but the reactor will most hopefully be "out of steam" within seconds if the safeties work.

They have battery rooms to run the plant for "hours", diesel generators to run the plant for days until fuel is needed.  ALL of these were broken, drowned, used up, spent and exhausted in Fukishima.

They can obviously run off external grid power which in 99% of routine shutdowns is exactly what they do.  Their main output transformers go dead, but they have a standard 3 phase incoming feed too.

3 mile island was a successful test of safety systems.  Fukashima was also a successful test of safety systems.  Chernobyl was a failed test of a rundown. Windscale was a British "YOLO, Have at it lads".  TMI everything worked and there were minimal impacts external to the plant. Fukashima, successfully proved that each of the systems put in place would have worked.  However it also proved that a complete and sustained "loss of grid", coupled with a generator room swamping... was not designed for and thus the plant became compromised.

I am pro-nuclear.  However when it come to Japan.  No.  Just No.  Not for you.  There is a checklist.  Japan should read it.

* Local Earthquakes?  TICK
* Local volcanoes?  TICK
* High population density near the coast?  TICK
* Tsunamis or freak waves  TICK.

These and many other "TICKS" for Japan (and other regions), should put nuclear power to the very bottom of the list.  IMHO.  For god sake it's the pacific ring of fire.  The most energetic part of earth crust, the most earth quake and tsunami prone part of the world.  It is not suitable for nuclear power, period.  They should be out right banned by international bodies.

[timeandfrequency]

Maryland (USA) based ' Whisker Labs' remotely monitored the Spanish grid around Madrid. See § 'A timeline'.
Minor fluctuations did begin to show up 3 hours before collapse.

[tggzzz]

The EU officials are denying cyber attak, but a denial from politicians is offten as good as an admmision of guilt.

You forgot to mention that MSSMM[sic] can't be trusted, and you should "do you're[sic] own research".

Save paranoid conspiracy theories for other forums, and wait until an explanation has been determined.

Besides, everybody knows it was the pixies (or gremlins) dancing on the HV lines.

White cockatoos in fire proof suits swinging on the lines.
It used to work for the open wire phone lines, near Wyndham Western Australia, but they didn't need the fireproof suits.

Fried squirrels dancing in on substation interruptors gave Cambridge UK a blackout >40 years ago. Thanks for reminding me

[JimboJack]

I wonder if they been using GPS to syncronise the equipment clocks wind farms, solar panels , battery banks, when you removed the large base load that provided the sync across the grid, you have to look else where........  or got remotely adjusted, simply IGBT transistors cannot take the surging or damping of a massive steam, spinning turbine control theory 1 on 1.

[David Hess]

...

Due to lack of "rotating reserve" (literal rotating generators with mass) this fault was insufficiently dampened, and the high percentage of PV on the grid then causes a cascade failure.

This sounds more plausible than "atmospheric phenomena".

Emulating newtons laws with inverters is difficult.
I suspect there will be amendments to EN 50549 due to this.

Emulating Newton's laws with inverters is trivial, at least on a small scale.  There are two cases:

1. The inverter provides a current into the grid proportional to the voltage.  A load draws current inversely proportional to its load resistance.  In this case the current is reversed, negative, so the inverter is synthesizing a negative resistance and connecting it to the grid to become a source instead of a load.

2. The inverter provides a current into the grid proportional to the voltage as above, but with phase lag or phase lead.  This requires phase locking a local frequency reference to the grid, and using the phase difference between the reference and the grid to adjust the current.  This is what a rotating source effectively does.  With a time reference, the inverter can now attempt to regulate grid frequency by adjusting the phase of its current compared to the line voltage.  With an external time reference, the inverter can attempt to regulate the grid phase.

The difficulty is one of scale.  A rotating source has enormous inertia and peak power capability which is either difficult to implement or uneconomical.  A 1000 watt grid-tie inverter with a 1000 watts of solar panels is not going to provide much if any overload or reserve capacity.  An inverter with a battery bank like the large Tesla systems could however, but at considerably extra cost.

[5U4GB]

I wonder if they been using GPS to syncronise the equipment clocks wind farms, solar panels , battery banks, when you removed the large base load that provided the sync across the grid, you have to look else where........  or got remotely adjusted, simply IGBT transistors cannot take the surging or damping of a massive steam, spinning turbine control theory 1 on 1.

That's what's used to synchronise PMUs, phasor measurement units, which are used for grid control.

Note that isn't as brittle as you'd think, unless you can jam GPS on a nationwide scale you can at most cause some local degradation, however having said that I'm also not party to the threat modelling for loss-of-GPS events.

[paulca]

The EU officials are denying cyber attak, but a denial from politicians is offten as good as an admmision of guilt.

You forgot to mention that MSSMM[sic] can't be trusted, and you should "do you're[sic] own research".

Save paranoid conspiracy theories for other forums, and wait until an explanation has been determined.

Besides, everybody knows it was the pixies (or gremlins) dancing on the HV lines.

White cockatoos in fire proof suits swinging on the lines.
It used to work for the open wire phone lines, near Wyndham Western Australia, but they didn't need the fireproof suits.

Fried squirrels dancing in on substation interruptors gave Cambridge UK a blackout >40 years ago. Thanks for reminding me

Came close to being one of them squirrels in Spain.  They do love their overhead lines they do.

The thing about HV lines, they are nearly impossible to see from the air. 

I found myself gliding (paraglider), "landing out", down a descending valley and first thought, "going to be a long glide to land (due to the descending terrain), but it's towards the actual field at least...", then... "Awww shit...  that's an HV pylon over there and... theres its sister over there.  That means the lines are straight across my glide.  It looks like there is an 80% chance I will land before them.....  damn it.  ABORT."

Thing is... there is no "ABORT" in a glider, not when you are at 50ft.  So that just become...  pick somewhere else.  Ideally somewhere that will hurt the least.... no not the trees... yes... there.... straight downwind into the side of a hill.  This will give me a definitive touch down point and prevent me getting close to the trees and the rotor winds/turbulence... or IN the trees.

So I avoided a power line landing / hang up, opted for a crash landing into a hill and got away with nothing but a bruised knee and slice out of the heel of my hand on a rock.

The WORST case isn't flying into them.  Usually this creates a big flash as you bring the phases together and you get 30 seconds to a minute to get out before they vapourise your glider with a current pulse to clear the line short.

The WORST case is landing and having the glider clip them while you are stood on the ground. 

We were advised if you get hung up in them.  If you are low enough to "leap" out, do so immediately.  You are already in the arc zone.

If you are too high up to jump out, do NOT attempt to lower yourself down.  STAY PUT.  You will be up there for hours, but do expect the lines re-energise and attempt to burn your glider if it's shorting the lines.  Do not panic, STAY PUT. 

When leaving the area do so in long bouncing strides, only 1 foot in contact at a time.  If the line does arc to earth the "step potential is still lethal over a single step.

They paragliding club there have an emergency number to call.  As soon as the hung up person is reported that line will be shutdown until crews arrive to assist.  However, that can take minutes.  Unless they de-energize the line it will "surge current" to clear any shorts which are normally caused by trees and can be vapourized easily. 

Don't be a tree.

I felt the 20% wasn't worth the risk.

[tggzzz]

The EU officials are denying cyber attak, but a denial from politicians is offten as good as an admmision of guilt.

You forgot to mention that MSSMM[sic] can't be trusted, and you should "do you're[sic] own research".

Save paranoid conspiracy theories for other forums, and wait until an explanation has been determined.

Besides, everybody knows it was the pixies (or gremlins) dancing on the HV lines.

White cockatoos in fire proof suits swinging on the lines.
It used to work for the open wire phone lines, near Wyndham Western Australia, but they didn't need the fireproof suits.

Fried squirrels dancing in on substation interruptors gave Cambridge UK a blackout >40 years ago. Thanks for reminding me

Came close to being one of them squirrels in Spain.  They do love their overhead lines they do.

The thing about HV lines, they are nearly impossible to see from the air. 

Low voltage lines are bad enough, e.g. spotting a farmhouse separated from the road, and then trying to spot the LV lines that will almost certainly be leading to it. Etc

As for "flash overs", here's a good 10min BBC TV segment about someone on his first trial lesson. The glider was hit by lightning and disintegrated; fortunately their parachutes saved them. Farnborough/AAIB examined the remnants very carefully, and the lessons learned have shaped how composite airliners are made.



In other news, it feel like the overhead line ground clearance is reducing. As far as I can determine, this new link from Hinkley Point C nuke is 400kV
https://www.google.com/maps/@51.3767387,-2.8387647,3a,75y,144.09h,87.89t/data=!3m7!1e1!3m5!1s9GV-K7Q11OwzYo-pdcioMw!2e0!6shttps:%2F%2Fstreetviewpixels-pa.googleapis.com%2Fv1%2Fthumbnail%3Fcb_client%3Dmaps_sv.tactile%26w%3D900%26h%3D600%26pitch%3D2.1133614422956697%26panoid%3D9GV-K7Q11OwzYo-pdcioMw%26yaw%3D144.08889292380593!7i16384!8i8192?entry=ttu&g_ep=EgoyMDI1MDQyNy4xIKXMDSoJLDEwMjExNDU1SAFQAw%3D%3D

Perhaps the pylons are closer together, so hot weather line sag is less of an issue.

[nctnico]

...

Due to lack of "rotating reserve" (literal rotating generators with mass) this fault was insufficiently dampened, and the high percentage of PV on the grid then causes a cascade failure.

This sounds more plausible than "atmospheric phenomena".

Emulating newtons laws with inverters is difficult.
I suspect there will be amendments to EN 50549 due to this.

Emulating Newton's laws with inverters is trivial, at least on a small scale.  There are two cases:

1. The inverter provides a current into the grid proportional to the voltage.  A load draws current inversely proportional to its load resistance.  In this case the current is reversed, negative, so the inverter is synthesizing a negative resistance and connecting it to the grid to become a source instead of a load.

2. The inverter provides a current into the grid proportional to the voltage as above, but with phase lag or phase lead.  This requires phase locking a local frequency reference to the grid, and using the phase difference between the reference and the grid to adjust the current.  This is what a rotating source effectively does.  With a time reference, the inverter can now attempt to regulate grid frequency by adjusting the phase of its current compared to the line voltage.  With an external time reference, the inverter can attempt to regulate the grid phase.

The difficulty is one of scale.  A rotating source has enormous inertia and peak power capability which is either difficult to implement or uneconomical.  A 1000 watt grid-tie inverter with a 1000 watts of solar panels is not going to provide much if any overload or reserve capacity.  An inverter with a battery bank like the large Tesla systems could however, but at considerably extra cost.

Agreed. Solar grid inverters can't do much in case real power is involved. A solar grid inverter can't supply more energy than it is already supplying (a typical solar grid inverter is always maxed out) and it can't absorb (much) real power either. A spinning mass inside a generator however has the ability to convert electric power to and from kinetic energy (accumulator function). Grid connected battery banks can do the same but not much of those have been installed yet.

[Zucca]

Can you PM me when the root-cause is identified?

Thanks.

[G7PSK]

The UK is re introducing 1920's technology to help stabilise the grid. Namely Syncronous condensers with massive flywheels.

https://www.quinbrook.com/news-insights/quinbrook-commissions-first-new-synchronous-condenser-for-the-uk-power-market/#:~:text=LONDON%20%E2%80%93%2015%20February%202022%20%E2%80%93%20Quinbrook,at%20Rassau%2C%20in%20Ebbw%20Vale%2C

These put real rotating mass back into the system.
Lister Drive, Liverpool:
ABB is installing two synchronous condensers at this location as part of a pathfinder project with National Grid and Statkraft, reports The Engineer. These condensers, coupled with flywheels, will boost inertia and provide grid stabilization services, according to ABB.
Rassau, Wales:
A project involving Welsh Power, Quinbrook, National Grid ESO, Siemens, and Western Power Distribution has installed a synchronous condenser and flywheel to enhance grid stability, reports Watt-Logic.
Grain Power Station, Kent:
Uniper has installed two synchronous condensers at this site, which are now supplying grid-stabilizing services, according to Uniper.
Pembroke Power Station, Wales:
MYTILINEOS is installing a synchronous condenser for RWE at this site, reports Metlen. This project supports RWE's decarbonization plan within the Pembroke Net Zero Centre, according to Metlen.
Keith, Scotland:
Statkraft announced plans to install a synchronous condenser at its stability project in Keith, reports the National Energy System Operator (NESO).
Purpose of Synchronous Condensers:
Grid Stabilization:
They provide inertia, short-circuit current, and voltage support, contributing to the stability of the grid, according to Siemens Energy.
Enabling Renewable Energy Integration:
They help facilitate the transition to a more renewable energy-based system by providing essential grid services that are often lacking in renewable energy sources, reports the National Energy System Operator (NESO).
Increasing Grid Inertia:
Synchronous condensers, especially when combined with flywheels, can significantly increase the inertia of the grid, which is crucial for maintaining stability during sudden changes in supply and demand, according to The Engineer.
Reactive Power Support:
They can provide or absorb reactive power, improving power factor and grid performance, according to Siemens Energy

[tv84]

20 years ago, a stork was (supposedly) responsible for an HV line incident which brought down the grid in southern Portugal. As of yesterday, this was monday's suspect lineup.

[G7PSK]

Wrong time of year for suspect no. 4

[valcher]

Let's end politics and look for those to blame within our own borders!

[themadhippy]

Wrong time of year for suspect no. 4

He could of been heading down to the balearics for a well earned break

[ejeffrey]

Um, actually; traditionally nuclear power plants are shut down during major blackouts.   The cooling pumps and other important systems are run from grid power and when that's gone they have to shift to backup generators and shut down the reactors.  Once shut down nuclear power plants take the longest to start up.

Why are they not able to run pumps from their own electrical output?

Some can, many cannot.

Under normal operation, the coolant pumps run from external grid power.  They need to be able to run when the reactor is shut down to provide cooling for the decay heat and also to cool the spent fuel pools.  They can be switched over to diesel backup generators in the case of a power outage.  The failure of these generators was a major element in the chain of failures that led to the Fukushima meltdown.

In the case of a blackout you have to disconnect the reactor from the grid.  You don't want to be trying to supply infinite power to a grid with no other supplies, and you don't want to over-power a small island.  So until you are able to do a controlled restart you want the reactor disconnected.  There are some reactors that can switch the cooling loop to internal power in this case.  The issue is that with no grid to power, the reactor has to be run at much less than normal power.  Some reactors can't do this, some can but it can still take a long time to ramp back up to full power, depending on how long it stays at the low power configuration.

Nuclear reactors probably could be designed and operated to be more proactive in grid stability, and to aid in cold-starts.  However, they are not well suited to it, so in most cases that hasn't been a worthwhile design criteria.  Generally it's easier to do with almost anything else.

[timeandfrequency]

The UK is re introducing 1920's technology to help stabilise the grid. Namely Syncronous condensers with massive flywheels.

https://www.quinbrook.com/news-insights/quinbrook-commissions-first-new-synchronous-condenser-for-the-uk-power-market/#:~:text=LONDON%20%E2%80%93%2015%20February%202022%20%E2%80%93%20Quinbrook,at%20Rassau%2C%20in%20Ebbw%20Vale%2C

These put real rotating mass back into the system.
[...]

It would be interesting to know how much production-consumption imbalance all these synchronous condenser could compensate, and for how much time (seconds, minutes ?).
Just to compare with the jolt that occured in Spain (15 GW production loss in 5 seconds).
The purpose of these synchronous condensers is probably not to compensate the total amount of loss, but just give additional time (let's say a couple of minutes) to organize shedding, which is actually the sole perennial solution in such dire situations.

As a ballpark comparison, I had a look to the Australian Grid batteries map who provide energy storage and contribute to grid regulation.
The total power that the 28 battery power plants in operation (green icons) can provide is 3.09 GW. This figure assumes that all of these batteries plants were on standby (no power delivery) just before the outage.
This therefore represents approximately the power of three nuclear power plants and about a fifth of the outage suffered by Spain.

[Innovatia]

Reported by an electronics engineer from whose back yard the Pyrenees can be seen:

The power outage did touch a small part of the French Pyrenees, in the Basque country and for a few hours. ... Our production systems are highly interconnected in Europe and I read that the 400-kV connection between France and Spain - in Catalonia - was suddenly disconnected which did cascade to the rest of Spain and Portugal. I know RTE - the French operator for the network - has procedure to isolate the problem and avoid contaminating the rest of the network which is why it was constrained for a few minutes to the southern part. I did not see any explanation yet. Of course, self-designated experts are blaming solar and wind turbines as guilty for corrupting the mains frequency (Spain is often 100% supplied by renewable) but there are no evidence yet. We will see what comes out from the on-going investigation.

[DavidAlfa]

Some notes:
-After blackout only people with cash were able to purchase at some supermarkets which had diesel generators

I could pay with credit card just fine, however contactless didn't work.

[G7PSK]

The UK is re introducing 1920's technology to help stabilise the grid. Namely Syncronous condensers with massive flywheels.

https://www.quinbrook.com/news-insights/quinbrook-commissions-first-new-synchronous-condenser-for-the-uk-power-market/#:~:text=LONDON%20%E2%80%93%2015%20February%202022%20%E2%80%93%20Quinbrook,at%20Rassau%2C%20in%20Ebbw%20Vale%2C



These put real rotating mass back into the system.
[...]

It would be interesting to know how much production-consumption imbalance all these synchronous condenser could compensate, and for how much time (seconds, minutes ?).
Just to compare with the jolt that occured in Spain (15 GW production loss in 5 seconds).
The purpose of these synchronous condensers is probably not to compensate the total amount of loss, but just give additional time (let's say a couple of minutes) to organize shedding, which is actually the sole perennial solution in such dire situations.

As a ballpark comparison, I had a look to the Australian Grid batteries map who provide energy storage and contribute to grid regulation.
The total power that the 28 battery power plants in operation (green icons) can provide is 3.09 GW. This figure assumes that all of these plants were on standby (no power delivery) just before the outage.
This therefore represents approximately the power of three nuclear power plants and about a third of the outage suffered by Spain.

This is what I found online. In the UK, synchronous condensers typically have a reactive power capacity ranging from 20 to 200 MVAr. Some manufacturers can tailor solutions for power grid stability, potentially reaching up to 350 MVAr. They are also known for providing short-circuit power, inertia, and reactive power for dynamic loads 

[David Hess]

Some notes:
-After blackout only people with cash were able to purchase at some supermarkets which had diesel generators

I could pay with credit card just fine, however contactless didn't work.

When I lived near St. Louis, I only had to go shopping during power outages a couple times, and cash was not so much of a problem as dim lighting in the store, but I had my EDC flashlight so that worked out.

A larger problem was buying fuel.  I routinely use cash anyway, however if the station has no power, then they cannot operate their pumps.  I understand this has been a problem along the gulf during hurricane evacuations.  I have never seen a fueling station with backup power for their pumps, but I assume that they exist.

[DavidAlfa]

Here only low cost stations went down, most "big brands" had generators.

[coppercone2]

it can't hurt to revisit alot of old electromechanical technology, particularly ones that are big, because there is like obscene improvements in bearings, shafts, lubricants, modern magnets, high temperature enamel and contacts.

Magnetic bearings, super alloy shafts, tungsten silver contacts, extremely durable lubricants/greases. Once something gets big and expensiv e enough, using a few of those modern technologies actually might blow reliability through the roof.


I mean, even if you did something as simple as sending a laser shaft aligner back in time to 1920, it would probobly change the entire coarse of invention history. They \had little problems on big things that made them look else where.

Simply putting some of that old stuff on giant hard drive quality bearings and balancing it with something like a multi axis grinder could totally change the wear profile.

Am I thinking that some stuff got taken out of use because the overall picture when you include silk wire insulation, low quality oil and crooked setup? Especially the medium sized things, because you know the assemblers/implementers could not afford the tools you perhaps got better performance from in a expensive project like a hydroelectric dam (which probobly had obscenely expensive mechanical metrology.

[Someone]

...

Due to lack of "rotating reserve" (literal rotating generators with mass) this fault was insufficiently dampened, and the high percentage of PV on the grid then causes a cascade failure.

This sounds more plausible than "atmospheric phenomena".

Emulating newtons laws with inverters is difficult.
I suspect there will be amendments to EN 50549 due to this.

Emulating Newton's laws with inverters is trivial, at least on a small scale.  There are two cases:

1. The inverter provides a current into the grid proportional to the voltage.  A load draws current inversely proportional to its load resistance.  In this case the current is reversed, negative, so the inverter is synthesizing a negative resistance and connecting it to the grid to become a source instead of a load.

2. The inverter provides a current into the grid proportional to the voltage as above, but with phase lag or phase lead.  This requires phase locking a local frequency reference to the grid, and using the phase difference between the reference and the grid to adjust the current.  This is what a rotating source effectively does.  With a time reference, the inverter can now attempt to regulate grid frequency by adjusting the phase of its current compared to the line voltage.  With an external time reference, the inverter can attempt to regulate the grid phase.

The difficulty is one of scale.  A rotating source has enormous inertia and peak power capability which is either difficult to implement or uneconomical.  A 1000 watt grid-tie inverter with a 1000 watts of solar panels is not going to provide much if any overload or reserve capacity.  An inverter with a battery bank like the large Tesla systems could however, but at considerably extra cost.

Agreed. Solar grid inverters can't do much in case real power is involved. A solar grid inverter can't supply more energy than it is already supplying (a typical solar grid inverter is always maxed out) and it can't absorb (much) real power either. A spinning mass inside a generator however has the ability to convert electric power to and from kinetic energy (accumulator function).

Wow, you're back on the same plainly false claims again?

Plant (regardless of energy source) cannot run above the input source, that is true of all generators. On the grid some generators are run below their full power output as "reserve" so that if there is a change in demand they can respond. Solar inverters can do this. But of course you know this already and are trying to play semantics rather than provide the full picture.

Slewing down (shedding production) that's the thing that inverters are far better at doing than spinning mass! Other plant doesnt need to absorb power if the generation side is able to curtail quickly enough (or the demand side increase quickly enough).

How much energy storage is needed in different timeframes (capacitors: cycle by cycle to seconds, vs batteries hours) depends on the designed dynamics of the grid. What has been designed around hours of slew from nuclear/coal no longer makes sense both on the demand and supply sides.