Author Topic: Hacking the Rigol MSO5000 series oscilloscopes  (Read 167034 times)

0 Members and 2 Guests are viewing this topic.

Offline NoisyBoy

  • Regular Contributor
  • *
  • Posts: 165
  • Country: us
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1200 on: August 20, 2019, 08:04:16 pm »
It is self-healing!

To try out some of the latest finding you have discovered, I removed the firewall rule on my router, plug the scope into the network, and tried to recreate the problem.  Guess what, now the scope boots fine with the LAN attached, and rigol.com no longer blocked.

Not sure if some flags were reset in my scope or something happened on rigol.com end. 

One thing I did earlier was booting the scope with the firewall rule in place, when the Online Upgrade button light up, I push it for an online upgrade.  I backed out at the first screen when it asked me to accept the terms and condition.  Not sure if that might have somehow reset the flag, but for now, everything is back to normal.

BTW, the Online Upgrade button lights up whenever the scope is connected to the LAN, it does not matter if it can actually reach the rigol.com site.

On the safe side, I am keeping it off the Internet.  If I need remote control, it will only go on the physically isolated network at my hobby lab.
 

Offline mabl

  • Regular Contributor
  • *
  • Posts: 100
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1201 on: August 20, 2019, 08:05:51 pm »
It's been a long time since I saw that but I have the impression that all those constant writings are in the FRAM copy and not in the NAND.

I hope so... But just by looking at the file modification date suggests it changes repeatedly....
 

Offline tv84

  • Frequent Contributor
  • **
  • Posts: 907
  • Country: pt
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1202 on: August 20, 2019, 08:12:13 pm »
The other is a "upload" function but  it's not set to upload anything that would reveal it's hacked since that is only in appEntry with no file system modifications.

Not everyone patches.  ;)

Code: [Select]
<firmware>http://www.rigol.com/Support/ProductUpgradeFile?sn=%1$hardware=%2$behaviour=%3$software=%4</firmware>
<uploadurl>http://www.rigol.com/up.aspx?act=%1$filename=%2</uploadurl>

That upload-url makes me a bit squeezy.

This is what I was referring as "calling home". It shouldn't be too hard to see, with a well placed breakpoint, what is being sent.

 

Offline tv84

  • Frequent Contributor
  • **
  • Posts: 907
  • Country: pt
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1203 on: August 20, 2019, 08:17:30 pm »
It is self-healing!

To try out some of the latest finding you have discovered, I removed the firewall rule on my router, plug the scope into the network, and tried to recreate the problem.  Guess what, now the scope boots fine with the LAN attached, and rigol.com no longer blocked.

 ;D

Let's admit that they keep track of the last date when you reported your scope status. And, from time to time, they require you to report the status. If you don't...

Now, you've good for a certain period of time.

(This is a just a feeling...)
 

Offline mabl

  • Regular Contributor
  • *
  • Posts: 100
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1204 on: August 20, 2019, 08:23:27 pm »
Intriguing idea  |O. Possibly there is some kind of counter for power-ups or time persisted in /rigol/data/stat.dat. Next time I debug this, I'll set a breakpoint and try to trigger that auto-connect.
 

Offline tv84

  • Frequent Contributor
  • **
  • Posts: 907
  • Country: pt
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1205 on: August 20, 2019, 08:37:11 pm »
Intriguing idea  |O. Possibly there is some kind of counter for power-ups or time persisted in /rigol/data/stat.dat. Next time I debug this, I'll set a breakpoint and try to trigger that auto-connect.

That's all in the FRAM.

Answering your previous thoughts, I think the scope keeps all the status in the FRAM and then (after boot or in any specific periods/events) creates a copy in the NAND of that info in the /data files. Maybe just to ease the "calling home" task.

You can zero the whole FRAM and the scope is able to work (unlicensed).
 

Offline Fungus

  • Super Contributor
  • ***
  • Posts: 10096
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1206 on: August 20, 2019, 08:40:43 pm »
Let's admit that they keep track of the last date when you reported your scope status. And, from time to time, they require you to report the status. If you don't...

What if you never connect it to a network?  :popcorn:
 

Offline tv84

  • Frequent Contributor
  • **
  • Posts: 907
  • Country: pt
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1207 on: August 20, 2019, 08:44:44 pm »
Let's admit that they keep track of the last date when you reported your scope status. And, from time to time, they require you to report the status. If you don't...

What if you never connect it to a network?  :popcorn:

Of course, that's one solution. Better one is to patch the hyperlinks that try to dump the XML link page and do the upload.

BTW, we also must remember that in the code there is an email client, so the reporting can exist via email. The email address to which it reports can be easily found in the code.

 

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1208 on: August 20, 2019, 11:16:42 pm »
Let's admit that they keep track of the last date when you reported your scope status. And, from time to time, they require you to report the status. If you don't...

What if you never connect it to a network?  :popcorn:

Of course, that's one solution. Better one is to patch the hyperlinks that try to dump the XML link page and do the upload.

BTW, we also must remember that in the code there is an email client, so the reporting can exist via email. The email address to which it reports can be easily found in the code.

Fun fact about the email function. They hardcoded smtp credentials into appEntry.
 
The following users thanked this post: thm_w

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1209 on: August 21, 2019, 12:41:23 am »
Does Rigol have a security team email? they dun fucked up with their smtp credentials.

You can uh, log into their IBM iNotes instance using them and at minimum send email as rigol.com

You can even see their full internal employee directory.


They do have 4x "IT Operations, Process and Maintenance" addresses, maybe I'll send them a nice email from their own IBM instance  :-DD
« Last Edit: August 21, 2019, 12:49:22 am by delfinom »
 
The following users thanked this post: thm_w, bd139

Offline mabl

  • Regular Contributor
  • *
  • Posts: 100
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1210 on: August 21, 2019, 04:03:37 am »
Oh dear.... I did see the hard coded credentials, but thought nobody was crazy enough to do this. I once even checked the credentials on an older firmware and they did not work for sending mails (maybe the server was down or I made a mistake or they even changed).

Now you get into their company mail server? And see the employee directory? :palm: Maybe even see all the old sent emails by that account?

This is where the fun ends. Please contact Rigol as soon as possible.

EDIT: I tried sending them a warning message using their own SMTP server - it did not answer on port 25, which the scope also uses. So as I said before, their mail credentials seems broken. I wonder if this is due to your last message now, or if they removed plain text SMTP...
« Last Edit: August 21, 2019, 04:39:25 am by mabl »
 

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1211 on: August 21, 2019, 11:55:24 am »
Oh dear.... I did see the hard coded credentials, but thought nobody was crazy enough to do this. I once even checked the credentials on an older firmware and they did not work for sending mails (maybe the server was down or I made a mistake or they even changed).

Now you get into their company mail server? And see the employee directory? :palm: Maybe even see all the old sent emails by that account?

This is where the fun ends. Please contact Rigol as soon as possible.

EDIT: I tried sending them a warning message using their own SMTP server - it did not answer on port 25, which the scope also uses. So as I said before, their mail credentials seems broken. I wonder if this is due to your last message now, or if they removed plain text SMTP...
Are you sure it's not due to your ISP blocking port 25 like mine does?
 

Offline Kean

  • Supporter
  • ****
  • Posts: 915
  • Country: au
  • Embedded systems & IT consultant
    • Kean Electronics
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1212 on: August 21, 2019, 02:11:08 pm »
Oh dear.... I did see the hard coded credentials, but thought nobody was crazy enough to do this. I once even checked the credentials on an older firmware and they did not work for sending mails (maybe the server was down or I made a mistake or they even changed).

Now you get into their company mail server? And see the employee directory? :palm: Maybe even see all the old sent emails by that account?

This is where the fun ends. Please contact Rigol as soon as possible.

EDIT: I tried sending them a warning message using their own SMTP server - it did not answer on port 25, which the scope also uses. So as I said before, their mail credentials seems broken. I wonder if this is due to your last message now, or if they removed plain text SMTP...
Are you sure it's not due to your ISP blocking port 25 like mine does?

Why would they include credentials?  If it is sending email to @rigol.com, then it just needs to connect to the rigol.com MX.
Of course, as you mention they cannot rely on port 25 being open outbound (rarely is these days) so they'd want to use 587 or similar, and I guess their server configuration may then require auth.
In any case, sending email from the scope is just stupid when they could more reliably use a web API for notifications or data exfiltration.
 

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1213 on: August 21, 2019, 02:28:40 pm »
Oh dear.... I did see the hard coded credentials, but thought nobody was crazy enough to do this. I once even checked the credentials on an older firmware and they did not work for sending mails (maybe the server was down or I made a mistake or they even changed).

Now you get into their company mail server? And see the employee directory? :palm: Maybe even see all the old sent emails by that account?

This is where the fun ends. Please contact Rigol as soon as possible.

EDIT: I tried sending them a warning message using their own SMTP server - it did not answer on port 25, which the scope also uses. So as I said before, their mail credentials seems broken. I wonder if this is due to your last message now, or if they removed plain text SMTP...
Are you sure it's not due to your ISP blocking port 25 like mine does?

Why would they include credentials?  If it is sending email to @rigol.com, then it just needs to connect to the rigol.com MX.
Of course, as you mention they cannot rely on port 25 being open outbound (rarely is these days) so they'd want to use 587 or similar, and I guess their server configuration may then require auth.
In any case, sending email from the scope is just stupid when they could more reliably use a web API for notifications or data exfiltration.

They are using the credentials as a smtp relay which is normal. There's a giant list of reasons why nobody sends mail directly to domains these days (i.e. 90% chance the receiving mail server or your ISP or corporate network will block you)
It's just providing their own credentials and their own relay is stupid.
And then doing so by creating an user in their IBM Notes instance without blocking the user from login is extra icing on the cake.


Their intention is to allow users to enter in their own SMTP settings such as a corporate network relay or personal service like gmail. But yea they screwed up.



This is where the fun ends. Please contact Rigol as soon as possible.


Well I sent an email to their IT groups internally via the account.....hopefully they understand english.....  :D

Otherwise it's not like they have a security email address for such reports.
« Last Edit: August 21, 2019, 02:40:16 pm by delfinom »
 

Offline Kean

  • Supporter
  • ****
  • Posts: 915
  • Country: au
  • Embedded systems & IT consultant
    • Kean Electronics
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1214 on: August 21, 2019, 03:06:05 pm »
They are using the credentials as a smtp relay which is normal. There's a giant list of reasons why nobody sends mail directly to domains these days (i.e. 90% chance the receiving mail server or your ISP or corporate network will block you)

Yes, the fact it will likely be blocked, even if using a non-standard SMTP port, is why this whole thing is bonkers.
The fact that they then included credentials that can be used to do more than sending emails to their domain, just shows they shouldn't be writing networking code, or probably any embedded applications.

Please update us if you hear back from them.  If you don't hear that they've closed up the security hole in a pretty short time, then this should be reported to a security researcher to name & shame them.  e.g Brian Krebs or Troy Hunt.
 

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1215 on: August 21, 2019, 03:28:52 pm »
They are using the credentials as a smtp relay which is normal. There's a giant list of reasons why nobody sends mail directly to domains these days (i.e. 90% chance the receiving mail server or your ISP or corporate network will block you)

Yes, the fact it will likely be blocked, even if using a non-standard SMTP port, is why this whole thing is bonkers.
The fact that they then included credentials that can be used to do more than sending emails to their domain, just shows they shouldn't be writing networking code, or probably any embedded applications.

Please update us if you hear back from them.  If you don't hear that they've closed up the security hole in a pretty short time, then this should be reported to a security researcher to name & shame them.  e.g Brian Krebs or Troy Hunt.

Meh, you are making it out to worse than it is. Shit I've seen worse at major defense contractors and the stories I could tell.

Is it bad? Yes.
But does it mean they shouldn't be writing networking code or embedded applications? Hah
« Last Edit: August 21, 2019, 03:39:11 pm by delfinom »
 

Offline Kean

  • Supporter
  • ****
  • Posts: 915
  • Country: au
  • Embedded systems & IT consultant
    • Kean Electronics
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1216 on: August 21, 2019, 03:42:07 pm »
Meh, you are making it worse than it is. Shit I've seen worse at major defense contractors and the stories I could tell.

Possibly, but they've just compromised their own email server.  Think about that, and then what concerns there might be in putting this scope on an engineering network in a large corporation or university.

These scopes connect to external web servers via http (not https, so no encryption and no way to validate certificate to ensure there isn't a MITM attack), and then writes downloaded data to the filesystem.  I wouldn't be surprised if there was a serious vulnerability there that allowed malicious code to be injected and run on the scope, and not just via the firmware update process.

Is it bad? Yes.
But does it mean they shouldn't be writing networking code or embedded applications? Hah

The fact that a lot worse happens elsewhere doesn't mimimise how bad this is.

Of course... if it wasn't for these same developers this thread wouldn't exist at all.  And Rigol would be selling a lot less of their latest scopes.
 

Offline Mr. Scram

  • Super Contributor
  • ***
  • Posts: 8039
  • Country: 00
  • Display aficionado
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1217 on: August 21, 2019, 03:54:25 pm »
Maybe Keysight is willing to pay a pretty penny for some Rigol company secrets they can dig up with those credentials.  :-DD  Making vulnerable firmware on purpose is one thing but this doesn't look like being done on purpose at all.
 

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1218 on: August 21, 2019, 05:12:28 pm »
Meh, you are making it worse than it is. Shit I've seen worse at major defense contractors and the stories I could tell.

Possibly, but they've just compromised their own email server.  Think about that, and then what concerns there might be in putting this scope on an engineering network in a large corporation or university.

These scopes connect to external web servers via http (not https, so no encryption and no way to validate certificate to ensure there isn't a MITM attack), and then writes downloaded data to the filesystem.  I wouldn't be surprised if there was a serious vulnerability there that allowed malicious code to be injected and run on the scope, and not just via the firmware update process.

Is it bad? Yes.
But does it mean they shouldn't be writing networking code or embedded applications? Hah

The fact that a lot worse happens elsewhere doesn't mimimise how bad this is.

Of course... if it wasn't for these same developers this thread wouldn't exist at all.  And Rigol would be selling a lot less of their latest scopes.

Why are you concerned about it being a vulnerability? It's a feature.  :-DD
The literal design of the download mechanism is download via http and just applying the gel file. And uh, you can see we can generate our own gel files by hand pretty easily.
So you could inject the malicious code yourself if you wanted to via a MITM easily over http.

But uh, I'll take my $1k scope that's worth far more since being patched and sit in the corner hugging it ;)


Possibly, but they've just compromised their own email server.  Think about that, and then what concerns there might be in putting this scope on an engineering network in a large corporation or university.


I still like relative comparisons. Why?
Because I'm seen so much shit in different industries that everything is fucking terrible to the point you can't escape it, there's just different levels of terrible. You can only work to compartmentalize your network and security to minimize damage when shit goes wrong. Shit, even my home network is running 5 VLANs with 2 dedicated just for IoT devices.

So I rank the scopes with some poor security decisions as less than say, the backdoored Cisco hardware those large corps or univerisities are most likely running ;)
« Last Edit: August 21, 2019, 10:02:25 pm by delfinom »
 
The following users thanked this post: thm_w

Offline tv84

  • Frequent Contributor
  • **
  • Posts: 907
  • Country: pt
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1219 on: August 21, 2019, 05:12:40 pm »
Unsecurity by design!  :popcorn:
 

Offline nimish

  • Regular Contributor
  • *
  • Posts: 52
  • Country: gb
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1220 on: August 21, 2019, 10:05:40 pm »
Anyone allowing this thing to connect to external networks is a fool.

 

Offline NoisyBoy

  • Regular Contributor
  • *
  • Posts: 165
  • Country: us
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1221 on: August 22, 2019, 12:19:30 am »
That's why mine is only going on a physically isolated network if remote control is required, there's really no reason for it to ever go on the Internet.  I will make all firmware update manually through an USB Drive, especially given how infrequent they come out.
 

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1222 on: August 22, 2019, 12:38:00 am »
It appears they disabled the account and no longer works to login via iNotes and SMTP. No reply to me, I assume there was some internal yelling after I CCed 4x internal IT distribution groups on my "you dun fucked up" email.

You know what's interesting digging further (and avoiding the temptation to anally probe some infrastructure), for a Chinese company they sure like to escape the great firewall by using an email provider who proceeds to host all their infrastructure on DigitalOcean (which has no Chinese datacenters).
« Last Edit: August 22, 2019, 01:58:11 am by delfinom »
 
The following users thanked this post: thm_w, Kean, tv84

Offline Kean

  • Supporter
  • ****
  • Posts: 915
  • Country: au
  • Embedded systems & IT consultant
    • Kean Electronics
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1223 on: August 22, 2019, 02:56:56 am »
Dave tweeted about it, so the word has got out  >:D

Glad they closed the hole so quickly.
 

Offline delfinom

  • Contributor
  • Posts: 42
  • Country: 00
Re: Hacking the Rigol MSO5000 series oscilloscopes
« Reply #1224 on: August 22, 2019, 03:00:15 am »
Dave tweeted about it, so the word has got out  >:D

Glad they closed the hole so quickly.

They fixed it way before the tweet ;)
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf