Lightning detection

[microsfx]

Hi all,

New member here, not sure I'm posting in the right forum but here goes:

Anyone interested in becoming partner in a project revolving around real-time lightning detection and locating?



You'd have to be an EE or posses equivalent skills and be willing to take over responsibility for the electronics hardware that is already designed by my former partner, who recently decided leave the project. This includes some light technical end-user support but most importantly, continued development to improve our product. I myself work on firmware (STM32/PIC/AVR/ESPxx) and C/S (SBC/Win/Linux) software side of things.

We currently have some 60 lightning sensors evenly deployed across North America and Europe, most operated by weather enthusiasts. The goal is to expand the numbers a lot over time and geographically, ultimately reaching a level of data quality that is suitable for commercial purposes. You should ideally reside in US, Canada or Europe.

If you have an interest in weather sensing applications and/or lightning detection and feel that this might be something for you, please let me know via PM. For general questions, shoot below and I'll try to answer the best I can.

[Syntax Error]

Do you have any technical details of the detector hardware and supporting network?

I should also ask, does your network have any association with Blitzortung?

[microsfx]

Syntax Error:

The hardware consists of a custom designed mini-whip type E-field antenna sensitive in the VLF/LF range and separate controller board for signal conditioning and sampling. The board hosts an STM32F4 micro, GPS and OLED display. The micro will process and compress signal data and then feed this via USB to a PC client (which can be regular Windows/Linux PC or a single board computer such as Raspberry Pi). The client's main purpose is to forward data over Internet to one of two central processing servers (US/EU) and also enable the user to adjust a few settings and monitor system performance.

Network is basically a standard TCP/IP configuration with clients connecting to a central server where time-of-arrival computations are done. One server is in Germany and handles the European sensor sites, and another in Wisconsin, USA and for North American sites.

This is not associated with Blitzortung (BO). The concept is similar but we use E-field detection and are able to locate and discriminate lightning discharge polarities, whereas BO is mainly H-field detection and limited to locating. I believe our system as a whole is technically more robust and we have observed approximately the same level of locating accuracy with half the number of sensors compared to BO, the latter requiring 11 sensors to co-detect a lightning strike before a TOA solution is computed.

[Syntax Error]

@microsfx Thank you for your detailed reply. Sounds an interesting project for an EE. I must admit, it's a little out my scope as a SW geek but, as a life long lightning/weather watcher, I'd certainly like to learn more. Do you have a website? You can link it in the forum

Thinks... Have you thought about reaching out to the guys at weather-underground, as this sounds like their market?

[microsfx]

Hey @Syntax Error

We have a website of course but given the state of project some info might not be relevant and things will change in the near future:
https://microsferics.com

I'm also testing out a live beta map with websocket interface for improved real-time experience, give it a load on this link:
https://astrogenic.net/mapper/toamapper.html

What kind of software do you do?

As for WeatherUnderground... I know the ex-CEO and founder plus some other guys at the original, good, WU. Had lots of great cooperation with them some years ago. A while ago The Weather Channel bought them up and soon after, IBM ingested both and got rid of most of the people in leading positions. Since then WU has been a bit of mess, I'm guessing due to major restructuring. I did reach out but my only remaining contact at WU said he was not allowed to make a decision due to IBM's strict control over what their subsidiaries are allowed to publish. He sent it up the chain but so far *crickets*.

[Nuno_pt]

@microsfx nice project, will follow with interests.

I was looking at Blitz, but the leading times for having that are very long on the waiting list, so yours could be an interesting project.

I can't help you with nothing, but if you need a place in Portugal to place a sensor I can put it here. 

[microsfx]

@Nuno_pt

Thanks and it looks like we are making progress in terms of finding a new partner, which is good news.

Yes, Blitz has some 6000 people on the waiting list, estimated time is 3 years to get a kit. I think we can get at least 5-10% of those to join us instead.

Of course we need people in Southern Europe too. I'm near Malaga, Spain and there are too few participants around here but I plan to have at least one sensor up and running at my location soon. If you want I can PM you once we get back on track with our project.

[Nuno_pt]

@Mmicrosfx

I'm happy that you're on the right track with this project.

About Blitz, I know when I join I was #63xx from what I remember, they are to slow in terms of moving parts, from what I was able to find on the forum, they order/make a batch, and only when they have all that batch sold is the time they order another batch of PCB and components, that is way too long.

I'm near Cascais, Lisbon, about 1.5km (1 mile) from the sea, so if you want I've no problem in putting here a sensor.

Yes PM me when you're ready, or when you've news about something that will move forward this project.

I'll send you my email by PM, and that way we can stay in touch if you want.

[Syntax Error]

@Mmicrosfx : Thanks for the info about WU. I did not know they were in the clutches of the men in blue, but that would explain a lot. Back in the dial-up days, WU was my goto bookmark. Mainly because of the storm chasers.

Any thoughts about getting your detectors into South America and Africa? With Blitz, Europe and the USA are over detected, but there are some big not spots on the other continents.

A wider question about lightning detection. Is it possible to detect sprites, jets and elves with this technology? Do the intense lightning discharges that spawn mesospheric phenomena have a subtly different EM signature?

[TK]

Have you compared your custom HW solution to the AS3935?

[floobydust]

Franklin AS3935 I found is an overpriced piece of junk. It's a great idea- but the 500kHz antenna tank circuit picks up AM radio stations. It's only one-axis and sometimes detects nothing. BW is too low and it constantly false triggers mainly with male speech from AM radio. I think the part is no longer carried by distributors now.

The lightning research I looked at found AM band H-detection is OK for cloud-ground strikes, but by then the storm is on top of you. VHF E-field is superior for providing early warning of approaching thunderstorm or tornado, it is quite active with cloud-cloud discharges. Many researchers are using SDR to study the EM spectrum in detail near storms.

I considered making a Blitzortung node but I think it's expensive (goal is under 300€, $USD 325, $CAD 458, $AUD506) and endless complaints about supply issues for the boards. They are (understandably) very strict about what hardware connects to their network. Although the H-field antennas seem to vary widely.
You have to first register your interest, wait for pc boards to come in. It seems Gerbers and parts kits are protected and the backlog is a bit too much. There are a lot of parts and I think this discourages most people from getting into it, it's a major electronics build. You don't even know if a new system/pcb rev is being developed.


From OP's description, the only thing I dislike is the MCU requiring connection to a PC.
If you have GPS time sync, with event time stamps, I would think the MCU can do some pre-processing or just send the sampled waveforms out. I think node cost is a huge factor in determining if people will adopt, set one up. At near $500 I'm out.

edit: the paper I mentioned is New Mexico Tech, VHF Mapping Observations of Lightning Discharge Processes, Lorentz Center International Workshop Streamers, sprites, leaders, lightning: from micro- to macro-scales, 8-12 October, 2007. Leiden, Holland
Continuous broadband lightning VHF mapping array using MUSIC algorithm Jan. 2020 is more recent

[microsfx]

@Nuno_pt: Got it, will let you know as soon as I have some news.

@Syntax Error: Europe and North America are the primary target areas for now and expansion to other continents will certainly happen but I can't say when. A problem in Africa (same as inland Australia) is the Internet and power grid infrastructure. It is certainly possible to run on solar power and modem data comms although this hasn't been extensively tested.

Sprites/Jets etc.: My knowledge about these phenomena is limited but I think that technically it should be possible, at least to some degree. Sprites, Jets and other TLE's do no necessarily emit signals in the same frequencies or the band that we are listening to. As an example, it could work for Sprites but would require a specialized antenna and modded firmware for Jets. One other issue is that most of these events occur coincidentally with regular lightning. So you'd have a positive Cloud-Ground discharge immediately followed by a Sprite. A DSP algorithm would have to be designed to reliably discriminate between those two signatures.

@TK: AS3935 is a toy, it has 40 km max range and is very sensitive to disturbers. Our system will detect lightning at 4000 km under optimal conditions with the average being somewhere between 1500-2000 km.

@floobydust: The PC requirement is for historic reasons. When we started the system was designed to work with an existing single point lightning detection system which in itself required a Windows PC to work. Our custom antenna came at a much later date. Nowadays, a "PC" is still required but it can be a single board computer such as RPi, NanoPi (w. Ubuntu Core) or Beaglebone. Client software is available for all these platforms, as well as Ubuntu/Debian and Windows PC's. Some may see this separation as a drawback but our existing users seem to appreciate to have a local user interface to play with, upgrade firmware through etc. instead of having to go online to do all of this. I think that in future revisions we will probably integrate a NanoPI or equivalent on the PCB board.

Also pricing, our system is shipped assembled and with everything in custom enclosures, comes with an active E-field antenna, power feed unit, coax cable and 1 year warranty. It is basically plug-and-play which is in stark contrast to BO's assemble-and-fail kit 

[Nuno_pt]

I think node cost is a huge factor in determining if people will adopt, set one up. At near $500 I'm out.

Hi @floobydust, I've found this on the web, < https://astrogenic.com/pub/microsferics_info_bg.pdf > I don't know if it's up-to-date or not (@microsfx can tell us, and if it need also the Boltek or PCI card or it's a standalone unit like Blitz), about the pricing < http://microsferics.com/index.php/Pricing > at 248€ it's on the same price tag that Blitz < https://forum.blitzortung.org/attachment.php?aid=2805 > this is the latest system from Blitz, from what I've read Egon has release the Green system for general use you can see here < http://en.blitzortung.org/forum.php > but with no support.

the only thing I dislike is the MCU requiring connection to a PC.

For me also, I would like a all in one box solution( even if you would have to buy the box that you like), that way you would only have to connected to the Ethernet, even if with a Raspi, it would be preferable to have all in the same box, that in two separate box's, that's the way I like most.

I would also like to see

" endless complaints about supply issues for the boards."

Yes, that's the true, When I sign I was #6xxx on the line, the expected time like @microsfx said is about 3 years, but this year they will only ship the large percentage of the boards for country's will low coverage, so more time, one of the reasons for this is they only make new batch's of PCB's and components orders when they have sold out the previous batch.

I'm also on the waiting list for a Blitz, but if a solution at the same price range, similar performance and more quickly is available (the key here is time of deliver, from the time you sign your interest, till the time you receive the unit), I could jump the train, also many members of the waiting list of Blitz.


 The E-field antenna occupies less space then a H-field antenna, but would it be usefull to have both , or just a E-antenna, any tests on this?

Let's see what @microsfx has to say about this and some more ideas that you could share here, or maybe create a new topic for discussing ideas, if @microsfx want that input, let's see what @microsfx has to say about some ideas and this discussion.

[microsfx]

Hi @Nuno_pt

The link you found is outdated and for the TOA Compute box only. This was before we had a custom antenna option available. By itself this box is useless and requires either a complete Boltek system or the special custom antenna to be able to capture lightning signals.

Because the project is currently in a state of flux I can't say exactly what the final cost will be but floobydust is most likely close to the mark. I'll contact you directly with proper pricing info once I have this myself.

E-field and H-field antennas detect the same thing, both methods have advantages and drawbacks. In our case it would be pointless to also have an H-field antenna as the system is designed to detect what we need using E-field only.