Author Topic: To the Linux Gurus - Problems with HSTS in Firefox  (Read 3430 times)

0 Members and 1 Guest are viewing this topic.

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
To the Linux Gurus - Problems with HSTS in Firefox
« on: October 30, 2019, 05:42:02 am »
Well, I already talked with my VPN provider (the dared NordVPN) but their own solution is to test somethings that I especially tested already and shown in the original mail  :palm:.

The problem is the following:

Since the October crackdown on server from China, all IPs from the NordVPN servers were blocked. After two weeks they got a solution using the OpenVPN app instead of their own app. It always happen with the OpenVPN on Linux but I remember having the same problem sometimes after 20:00 in China with the NordVPN app before the crackdown, where the speed to the servers located outside of Mainland China gets really trottled (we are talking from using my full bandwidth to less that 800kps. For what I know is a problem of the links outside in the 3 exit points in China - Shanghai, Beijing and Shenzhen not had been upgraded for years).

Sometimes when connecting via Windows and constantly when connected via Linux I get the following message on Firefox:



Other times I get this:



It's the Google address but under it it shows that Facebook detected a problem (?????).

That only happens in websites that use specifically HSTS as authentication, or HTTP Strict Transport Security. Websites as google.com, youtube.com and amazon.com to say some.

On Windows the problem mostly never happens because my Antivirus suite, Kaspersky Small Office Security signs the certificate authenticity:



But the problem is persistent on Linux, it always fail during the TLS handshake. I tried changing the following parameters on the Firefox about:config without success in the end result:

security.tls.version.max from 4 to 2 or even 1, nothing
network.proxy.proxy_over_tls from true to false same thing, nothing

The OpenVPN files were edited to add the option block-outside-dns, but that option is a Windows only, in Linux the OpenVPN app as soon as it catches that like it stops loading the file with a Fatal Error message.

The reality is that when checking for DNS Leak, the Windows perform a lot better than the Linux one:

Windows 10 1903


Fedora 30 Workstation 5.3.7-200.fc30.X86_64


How can I solve this problem on Linux using the OpenVPN app? If more info is needed please ask for it.
« Last Edit: October 30, 2019, 08:42:46 am by Black Phoenix »
 

Offline I wanted a rude username

  • Frequent Contributor
  • **
  • Posts: 627
  • Country: au
  • ... but this username is also acceptable.
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #1 on: October 30, 2019, 06:47:50 am »
Hope I'm wrong, but it looks like something is MITMing your connection, and Kaspersky is hiding this from you. But maybe that is just the nature of the OpenVPN app.

Personally I would never trust a certificate authority other than those on Mozilla's list. Including especially those of corporate firewalls.

The Great Firewall of China does traffic analysis based on machine learning algorithms. It can't see into your TLS sessions, but it can see the data flows, and stochastically matches them to known VPN data flows. VPN connections work for some time, but then it throttles them.

The surest way around the problem (other than, of course, leaving the country) is to use an obfuscating algorithm, such as TorBrowser's obfs4. This distorts the data flows to fool the traffic analysis. Not guaranteed, but the best tool you have.

Edit: The forum software is broken and can't handle the correct link. This one should work, after a redirect.
« Last Edit: October 30, 2019, 06:59:52 am by I wanted a rude username »
 

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #2 on: October 30, 2019, 08:41:07 am »
Hope I'm wrong, but it looks like something is MITMing your connection, and Kaspersky is hiding this from you. But maybe that is just the nature of the OpenVPN app.

Personally I would never trust a certificate authority other than those on Mozilla's list. Including especially those of corporate firewalls.

The Great Firewall of China does traffic analysis based on machine learning algorithms. It can't see into your TLS sessions, but it can see the data flows, and stochastically matches them to known VPN data flows. VPN connections work for some time, but then it throttles them.

It doesn't surprise me that this is the cause but in reality in Windows it's running better than using the app from NordVPN when it worked. Even at night I'm able to have reasonable speeds, slow but not as slow as I said in the second paragraph. So the servers are from the same provider, but the apps to access are different, and from what I see, if I'm able to configure it better It doesn't look like I will keep using the NordVPN app in the future since I don't have to constantly shutdown the connection and reconnect to the same server to get a increase of speed or when the app simply makes my traffic crawl without any reason whatsoever.

The surest way around the problem (other than, of course, leaving the country) is to use an obfuscating algorithm, such as TorBrowser's obfs4. This distorts the data flows to fool the traffic analysis. Not guaranteed, but the best tool you have.

Edit: The forum software is broken and can't handle the correct link. This one should work, after a redirect.

The TorBrowser is just for web browsing, it doesn't include other kinds of traffic that I use, it doesn't create a virtual network adapter. Nor without installing a pfSense or similar machine to control all my traffic outside via the onion. It solves part of the problem but then other kind of access in terms of traffic that I may need stops working as it is working now.

Use ExpressVPN instead. It's been working for me consistently, including around the two meetings, and around June 4th.

Probably what I had should have done instead of buying 3 years of NordVPN last December in preparation to come here. Sorry but since I have a signature with still 2 years remaining, I prefer to keep using it and not waste the money spend. I prefer to try to understand what I have misconfigured on the Linux part. Specially when both DNS leaking tests report something different.

Again as I told, when before September the NordVPN app was connecting to the servers in question I never had this problem with the TLS handshakes. So definitely something regarding the DNS Spoofing/Poisoning/highjacking done by the GFW analysis of traffic.

https://www.crowdstrike.com/blog/cyber-kung-fu-great-firewall-art-dns-poisoning/
https://en.greatfire.org/faq/what-does-dns-poisoning-mean

A note from the last link:
Quote
TLS (SSL) certificate filtering

This time GFW knows you are using encryption to evade censorship and has decided to censor your certificate which is sent in plain text before an encryption tunnel is established. Two counter measures as before: 1. Because your certificate is completely free and assigned automatically by robots, there's nothing stopping you from changing it constantly as the filtering list of TLS (SSL) certificates updates rarely 2. Use CloudFlare's SSL option. CloudFlare replied to me that multiple sites may use the same certificate. Each site needs to have its own subject alternate name (SAN) and  the common name can be a variation of SSL#.cloudflare.com. So as in the case of IP addresses, GFW can't filter your certificates without blocking a bunch of innocent sites. (Again that didn't stop them before. They could block SSL connections of a bunch of sites and then filter only your domain. With this method other sites could be reached via http only.)
 

Offline OwO

  • Super Contributor
  • ***
  • Posts: 1250
  • Country: cn
  • RF Engineer.
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #3 on: October 30, 2019, 09:00:09 am »
If you have a VPS you can try SSH port forwarding. After you SSH into the server run "top" to keep the connection from becoming idle (long running idle connections get killed). Other than that openvpn on a VPS also seems to work well. I wouldn't touch commercial VPN services because these are under constant watch by the CCP since laymen can easily access them. Remember the purpose of the GFW isn't to keep everyone out, it's to keep the average joe from reading random fake news/populist propaganda. Tools with higher barrier of entry or only accessible to the technically inclined will not get targeted. SSH is safe for now until commercial VPN services start using it too.
Email: OwOwOwOwO123@outlook.com
 

Offline OwO

  • Super Contributor
  • ***
  • Posts: 1250
  • Country: cn
  • RF Engineer.
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #4 on: October 30, 2019, 09:05:12 am »
As to speed, it usually isn't throttling but just congestion. All websites outside of china are slow after 5-6pm (and often in the afternoon too) even on plain HTTP. There *are* ways around it that I'm sure the commercial services/proprietary protocols do, which is to simply use UDP with aggressive retransmit/congestion control. TCP is very "polite" on the network and will back off the transmit rate as soon as there is any packet loss, and you can easily get a "bigger slice of the pie" by simply not backing off your transmit rate as easily and dealing with the packet loss. That may be one of the reasons they are blocking these VPN services but not targeting openvpn as much.

EDIT: when I say slow I'm talking 20-30KByte/s (150-200Kbit/s).
« Last Edit: October 30, 2019, 09:06:56 am by OwO »
Email: OwOwOwOwO123@outlook.com
 

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #5 on: October 30, 2019, 09:31:27 am »
If you have a VPS you can try SSH port forwarding. After you SSH into the server run "top" to keep the connection from becoming idle (long running idle connections get killed). Other than that openvpn on a VPS also seems to work well. I wouldn't touch commercial VPN services because these are under constant watch by the CCP since laymen can easily access them. Remember the purpose of the GFW isn't to keep everyone out, it's to keep the average joe from reading random fake news/populist propaganda. Tools with higher barrier of entry or only accessible to the technically inclined will not get targeted. SSH is safe for now until commercial VPN services start using it too.

I don't have a VPS, I also heard about it a lot while searching after arriving here. Probably when I get better with Linux configuration I will think of renting a VPS server and configure it. But currently this is what I have and it's what I'm trying to work with.

As to speed, it usually isn't throttling but just congestion. All websites outside of china are slow after 5-6pm (and often in the afternoon too) even on plain HTTP. There *are* ways around it that I'm sure the commercial services/proprietary protocols do, which is to simply use UDP with aggressive retransmit/congestion control. TCP is very "polite" on the network and will back off the transmit rate as soon as there is any packet loss, and you can easily get a "bigger slice of the pie" by simply not backing off your transmit rate as easily and dealing with the packet loss. That may be one of the reasons they are blocking these VPN services but not targeting openvpn as much.

EDIT: when I say slow I'm talking 20-30KByte/s (150-200Kbit/s).

One of the reasons Chinese internet is so slow (when accessing sites outside of China) is that there are only three main gateways that connect China's "local" internet to the rest of the world – one in the north, around Beijing, one in the center, in Shanghai, and the third in the south, in Guangzhou. As traffic passes through these gateways, packets are "mirrored" and monitored by government computers, which could further slow the process.[check out James Fallows' article on the Great Firewall for more: “The Connection Has Been Reset”]

China's internet infrastructure is also just relatively undeveloped. China's size is comparable to the US (i.e. it's a big country) so it's difficult and costly to build such systems at that scale, especially when many of China's areas are still underdeveloped in many other ways. Also, while China's demand for internet is certainly growing, the government's interest in satisfying that demand is still low, so it's not in their interest to invest in making access to information even easier for its citizens (the government being the ones that control telecoms in China).

China average speeds are, as reported by Akamai Technologies, are 7.6Mb/s on average, on the 74th position of all countries analysed. https://www.akamai.com/us/en/multimedia/documents/state-of-the-internet/q1-2017-state-of-the-internet-connectivity-report.pdf

Plus with 1.3 Billion people where around 75% use some kind of equipment that access the Internet, is normal to have congestions. It's totally normal to see outside people using WeChat VideoCall each and everyday, and always looking at their phones watching movies and series.

But that you already know, for sure.
« Last Edit: October 30, 2019, 10:25:24 am by Black Phoenix »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6260
  • Country: fi
    • My home page and email address
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #6 on: October 30, 2019, 11:28:55 am »
The OpenVPN files were edited to add the option block-outside-dns, but that option is a Windows only, in Linux the OpenVPN app as soon as it catches that like it stops loading the file with a Fatal Error message.
Does your OpenVPN config file contain dhcp-option DNS 103.86.96.100 or pull so that the NordVPN server can pass their DNS server? Does the /etc/openvpn/update-resolv-conf script contain foreign_option_ and /sbin/resolv.conf ?  These are basically the facility how the name resolver is updated when an OpenVPN connection is made.

If you do, I have an inkling that systemd is involved (specifically, that it rejects the openvpn DNS settings just because |O), but maybe that is paranoia.
« Last Edit: October 30, 2019, 11:30:57 am by Nominal Animal »
 

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #7 on: October 30, 2019, 02:44:36 pm »
Does your OpenVPN config file contain dhcp-option DNS 103.86.96.100 or pull so that the NordVPN server can pass their DNS server? Does the /etc/openvpn/update-resolv-conf script contain foreign_option_ and /sbin/resolv.conf ?  These are basically the facility how the name resolver is updated when an OpenVPN connection is made.

If you do, I have an inkling that systemd is involved (specifically, that it rejects the openvpn DNS settings just because |O), but maybe that is paranoia.

Well regarding that, as said in the PM, yes it have, here the log from the field in question:
Code: [Select]
PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 103.86.96.100,dhcp-option DNS 103.86.99.100,sndbuf 524288,rcvbuf 524288,explicit-exit-notify,comp-lzo no,route-gateway (retracted),topology subnet,ping 60,ping-restart 180,ifconfig (retracted) 255.255.255.0,peer-id 31,cipher AES-256-GCM'
So the DNS are being applied correctly.

Although now Youtube.com opens OK, google.co.uk also, but google.com reports this:



Why is Google.com showing a certificate related with Facebook???
« Last Edit: October 30, 2019, 02:46:23 pm by Black Phoenix »
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7764
  • Country: de
  • A qualified hobbyist ;)
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #8 on: October 30, 2019, 03:29:24 pm »
Hope I'm wrong, but it looks like something is MITMing your connection, and Kaspersky is hiding this from you. But maybe that is just the nature of the OpenVPN app.

I agree, looks like a nosy middlebox. Regarding Kaspersky, most security solutions basically install a MITM kit to check https traffic. To prevent certificate warnings they also add their root certificate to the OS. Unfortunately they are doing a poor job of checking the original certs and alerting users. This way they can create a false impression of a secure connection.
 

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #9 on: October 30, 2019, 03:36:47 pm »
Hope I'm wrong, but it looks like something is MITMing your connection, and Kaspersky is hiding this from you. But maybe that is just the nature of the OpenVPN app.

I agree, looks like a nosy middlebox. Regarding Kaspersky, most security solutions basically install a MITM kit to check https traffic. To prevent certificate warnings they also add their root certificate to the OS. Unfortunately they are doing a poor job of checking the original certs and alerting users. This way they can create a false impression of a secure connection.

Then the noisy middlebox as you said affect the PC connected via WIFI to the China Mobile Router, affects the same PC connected via USB to the Phone tethering the LTE connection from China Mobile and also affects when connected to the WIFI network in the Library. So 3 different access points, same results.

What is common again? GFW and OpenVPN. What I'm trying to find is where.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6260
  • Country: fi
    • My home page and email address
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #10 on: October 30, 2019, 03:38:45 pm »
So the DNS are being applied correctly.
Well, not exactly: your VPN client does receive the DNS servers, but there is no guarantee the DNS settings are applied.

One way you can verify this is to run
  sudo tcpdump port domain
in one terminal window, and in another, look up a host name, say
  host www.google.com
In the tcpdump terminal, you'll see exactly which DNS servers your machine connects to.

Like I said, I suspect systemd is fuggering up the actual update of the DNS settings.  OpenVPN uses /etc/openvpn/update-resolv-conf script and the /sbin/resolvconf utility (from the resolvconf package) to update the DNS settings.  The issue could be that the DNS settings do not get updated correctly.

Why is Google.com showing a certificate related with Facebook???
View the certificate, and check out the Issuer/Certificate Authority.  For example, this here forum uses a Let's Encrypt certificate, issued by Let's Encrypt Authority X3 certificate.  (So do I, BTW, on my own site.)
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7764
  • Country: de
  • A qualified hobbyist ;)
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #11 on: October 30, 2019, 04:13:13 pm »
What is common again? GFW and OpenVPN. What I'm trying to find is where.

I just did a quick search, the GFW is able to detect OpenVPN connections for some time. Solutions could be running OpenVPN over SSL,  SSH or Obsfsproxy (https://community.openvpn.net/openvpn/wiki/TrafficObfuscation).
 

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #12 on: October 31, 2019, 06:34:23 am »
Well, not exactly: your VPN client does receive the DNS servers, but there is no guarantee the DNS settings are applied.

One way you can verify this is to run
  sudo tcpdump port domain
in one terminal window, and in another, look up a host name, say
  host www.google.com
In the tcpdump terminal, you'll see exactly which DNS servers your machine connects to.

Like I said, I suspect systemd is fuggering up the actual update of the DNS settings.  OpenVPN uses /etc/openvpn/update-resolv-conf script and the /sbin/resolvconf utility (from the resolvconf package) to update the DNS settings.  The issue could be that the DNS settings do not get updated correctly.

Why is Google.com showing a certificate related with Facebook???
View the certificate, and check out the Issuer/Certificate Authority.  For example, this here forum uses a Let's Encrypt certificate, issued by Let's Encrypt Authority X3 certificate.  (So do I, BTW, on my own site.)

Ok, I've done some changes and solved the problem at least at first glance. I was getting other strange errors when trying to access google.com and youtube.com. On youtube the page would load but no video would load, or start download. On google.com this happened:



Then sometimes the error as I shown before where the certificate used was not the google.com but the facebook.com one:





Ok, here the log for when you asked me to do the tcpdump to the host www.google.com:

Code: [Select]
phoenix@localhost ~]$ sudo tcpdump port domain
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlp1s0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:07:53.203649 IP localhost.localdomain.38905 > OpenWrt.lan.domain: 8370+ A? location.services.mozilla.com. (47)
11:07:53.203741 IP localhost.localdomain.38905 > OpenWrt.lan.domain: 19666+ AAAA? location.services.mozilla.com. (47)
11:07:53.206416 IP localhost.localdomain.45239 > OpenWrt.lan.domain: 37948+ PTR? 254.2.168.192.in-addr.arpa. (44)
11:07:53.208023 IP OpenWrt.lan.domain > localhost.localdomain.38905: 8370 4/0/0 CNAME locprod1-elb-eu-west-1.prod.mozaws.net., A 54.72.168.141, A 34.253.23.107, A 52.215.71.87 (147)
11:07:53.208095 IP OpenWrt.lan.domain > localhost.localdomain.38905: 19666 1/0/0 CNAME locprod1-elb-eu-west-1.prod.mozaws.net. (99)
11:07:53.208761 IP OpenWrt.lan.domain > localhost.localdomain.45239: 37948* 1/0/0 PTR OpenWrt.lan. (69)
11:07:53.209206 IP localhost.localdomain.42910 > OpenWrt.lan.domain: 46190+ PTR? 210.2.168.192.in-addr.arpa. (44)
11:07:53.210467 IP OpenWrt.lan.domain > localhost.localdomain.42910: 46190 NXDomain* 0/0/0 (44)
11:07:53.491442 IP localhost.localdomain.45719 > OpenWrt.lan.domain: 37061+ A? push.services.mozilla.com. (43)
11:07:53.491539 IP localhost.localdomain.45719 > OpenWrt.lan.domain: 44258+ AAAA? push.services.mozilla.com. (43)
11:07:53.493674 IP localhost.localdomain.48971 > OpenWrt.lan.domain: 46292+ A? push.services.mozilla.com. (43)
11:07:53.493818 IP OpenWrt.lan.domain > localhost.localdomain.45719: 37061 2/0/0 CNAME autopush.prod.mozaws.net., A 34.214.229.245 (97)
11:07:53.494657 IP OpenWrt.lan.domain > localhost.localdomain.45719: 44258 1/0/0 CNAME autopush.prod.mozaws.net. (81)
11:07:53.495204 IP OpenWrt.lan.domain > localhost.localdomain.48971: 46292 2/0/0 CNAME autopush.prod.mozaws.net., A 34.214.229.245 (97)
11:08:02.903703 IP localhost.localdomain.41786 > OpenWrt.lan.domain: 22828+ A? [url=http://www.google.com]www.google.com[/url]. (32)
11:08:02.906169 IP OpenWrt.lan.domain > localhost.localdomain.41786: 22828 1/0/0 A 31.13.72.54 (48)
11:08:02.906541 IP localhost.localdomain.37372 > OpenWrt.lan.domain: 64147+ AAAA? [url=http://www.google.com]www.google.com[/url]. (32)
11:08:02.907886 IP OpenWrt.lan.domain > localhost.localdomain.37372: 64147 1/0/0 AAAA 2404:6800:4008:800::2004 (60)
11:08:02.908135 IP localhost.localdomain.54107 > OpenWrt.lan.domain: 30974+ MX? [url=http://www.google.com]www.google.com[/url]. (32)
11:08:03.908741 IP6 localhost.localdomain.40837 > OpenWrt.lan.domain: 30974+ MX? [url=http://www.google.com]www.google.com[/url]. (32)
11:08:03.909067 IP localhost.localdomain.43483 > OpenWrt.lan.domain: 35935+ PTR? 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.e.e.c.4.6.4.d.0.8.d.f.ip6.arpa. (90)
11:08:03.912132 IP OpenWrt.lan.domain > localhost.localdomain.43483: 35935* 1/0/0 PTR OpenWrt.lan. (115)
11:08:03.912588 IP localhost.localdomain.50423 > OpenWrt.lan.domain: 24784+ PTR? 9.4.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.e.e.c.4.6.4.d.0.8.d.f.ip6.arpa. (90)
11:08:03.913932 IP OpenWrt.lan.domain > localhost.localdomain.50423: 24784 NXDomain* 0/0/0 (90)
11:08:08.557930 IP OpenWrt.lan.domain > localhost.localdomain.54107: 30974 ServFail 0/0/0 (32)
11:08:08.558272 IP6 localhost.localdomain.40837 > OpenWrt.lan.domain: 30974+ MX? [url=http://www.google.com]www.google.com[/url]. (32)
11:08:09.991507 IP6 OpenWrt.lan.domain > localhost.localdomain.40837: 30974 0/0/0 (32)
11:08:10.746061 IP localhost.localdomain.38081 > OpenWrt.lan.domain: 38125+ AAAA? fedoraproject.org. (35)
11:08:10.748403 IP OpenWrt.lan.domain > localhost.localdomain.38081: 38125 3/0/0 AAAA 2605:bc80:3010:600:dead:beef:cafe:fed9, AAAA 2610:28:3090:3001:dead:beef:cafe:fed3, AAAA 2604:1580:fe00:0:dead:beef:cafe:fed1 (119)
^C
29 packets captured
29 packets received by filter
0 packets dropped by kernel


Forcing the dns-options on the .ovpn file doesn't change anything, the problems still presist.

The OpenVPN /etc/openvpn/update-resolv-conf was not present in the folder in question, nor even inside the Client or Server Folder. Without the .ovpn files the folder would be totally empty.



I even instead of using the wireless from the OpenWRT router I am using (the Phicomm PSG1218 K2 that I cracked as shown in this post - https://www.eevblog.com/forum/networking/phicomm-psg1218-k2-hw-rev-a2-openwrt-how-to/) and connected directly to the Huawei one provided by China Mobile, thinking it could be anything on the OpenWRT firmware. Nothing still the same.

Since the new Fedora 31 Workstation was released, and I didn't know if I had f#ck this installation by any reason after most of the testing that I've done before posting this problem and other things I tried to other stuff to try to convert gradually from Windows to Linux, I nuked the Installation from Fedora 30 to the sky by deleting the partitions and keeping the bootloader. Windows worked fine, no need to fix the boot.

Then downloaded the Fedora 31 Live Iso, checked the SHA-256, and install again in the free space, not partitioned available on the disk as last time when I installed the Fedora 30.

After the Fedora 31 was working first thing without even running any dnf update I copied one of the .ovpn files and tested. Same problems, same errors as before. So it wasn't my fault.

Then I remembered that there was this option in the Settings/Network GUI:



And in the VPN zone it lets import files from .ovpn. So I imported one of the files, filled the spaces and voila... The problems cease to exist. The pages started loading perfectly and correctly and the certificates are loading perfectly, youtube videos work too (after you install the dnf -y install ffmpeg codecs of course). Even the DNS Leaking now have an acceptable state (adding the media.peerconnection.enabled on Firefox from true to false.



So now I ask: Why it works in the GUI but doesn't work on the terminal window?

If you do, I have an inkling that systemd is involved (specifically, that it rejects the openvpn DNS settings just because |O), but maybe that is paranoia.

So probably you were right all the time Nominal Animal... Well thank you to everyone who tried to help me, I hope I don't have to return here to report that everything gone apesh#t again.
« Last Edit: October 31, 2019, 07:04:21 am by Black Phoenix »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6260
  • Country: fi
    • My home page and email address
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #13 on: October 31, 2019, 08:56:29 am »
So now I ask: Why it works in the GUI but doesn't work on the terminal window?
The Settings panel was reworked in Fedora 27, and Fedora being the ones who push systemd the hardest, I bet they changed the way it constructs VPNs: from files to DBUS-only communications.  The systemd folks hate user-accessible configuration files, and insist on everything using dbus.

If so, the only way to create working VPNs in Fedora 31 is to configure it via DBUS, and that is what the Network Settings applet does.  Fits into my paranoid picture perfectly, too -- but I could be wrong.
 

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #14 on: October 31, 2019, 09:45:05 am »
Well Nominal Animal, I believe more in you that in the NordVPN support (last message they sent me was a link to a WeTransfer with all the current server .ovpn files with the block-outside-dns flag inserted when I specially mentioned in 2 mails before that the option is exclusive of Windows and my problem is not on Windows).

And not lying but I remember sawing something like DBUS in some file when I was seaching this morning. I can't remember the file in question, as I said after that I nuked the installation but I remember clearly that because I was trying your solutions on the posts. Thank you, I learned something new today. Basically that I really need to learn a lot more.
 

Offline OwO

  • Super Contributor
  • ***
  • Posts: 1250
  • Country: cn
  • RF Engineer.
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #15 on: October 31, 2019, 09:55:08 am »
I usually just delete the /etc/resolv.conf symlink, replace it with hardcoded nameservers, and chattr +i to prevent bullshit from messing with it.
Email: OwOwOwOwO123@outlook.com
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6260
  • Country: fi
    • My home page and email address
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #16 on: October 31, 2019, 12:07:41 pm »
I have been told my dislike for systemd is irrational, because it is so useful to some, but I think this is a good example of the reasons of my dislike.
A centralized system with a silly bus interface through a single-provider library is not a good long-term solution.

The resolver, like pluggable authentication modules (PAM), is a dynamic part of the actual userspace processes, even statically linked ones.  The old /etc/resolv.conf with the C library handling it internally worked for simple static configurations.  Then came Name Service Switch, which complicated things a lot.  Now, some of us want the resolver to be configured per control group, or even per process (useful for e.g. VPN'ing specific applications).  The obvious solution is to use a resolver daemon that the C library connects through an unix domain socket, providing the process' credentials as ancillary data (kernel verifies these).  This way the resolver daemon can treat each client process separately.

It would even be possible to provide fully backwards compatible support for old binaries examining /etc/resolv.conf directly, via a FUSE module, or a simple kernel module that helps the resolver daemon provide each process a separate view into a pseudofile exposed as /etc/resolv.conf.

But nooo, that would make too much sense.  So, instead, we get this abortion of systemd-resolved/systemd-resolve, which has four internal "modes", and ... well, :rant:.
 
The following users thanked this post: OwO, Black Phoenix

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #17 on: November 01, 2019, 03:33:03 am »
New Reply from NordVPN (I'm not going to try this but just for info):

Code: [Select]
Try disabling IPv6 first:

    Open a terminal window.
    Change to the root user.
    Issue the commands:


sysctl
-w net.ipv6.conf.all.disable_ipv6=1
sysctl
-w net.ipv6.conf.default.disable_ipv6=1
sysctl
-w net.ipv6.conf.tun0.disable_ipv6=1

Afterwards, instal dnsmasq and set your DNS servers to NordVPN's:

    Install dnsmasq:

    sudo
    apt update


    sudo apt install dnsmasq
    Disable systemd-resolved:

    sudo
    systemctl disable systemd-resolved.service


    sudo systemctl stop systemd-resolved.service
    Remove /etc/resolv.conf and create a new one:

    sudo
    rm /etc/resolv.conf


    sudo nano /etc/resolv.conf
    Enter into your empty resolv.conf file:

    nameserver
    103.86.96.100


    nameserver 103.86.99.100
    Press Ctrl+x to exit the editor. Enter y to save and then press Enter to overwrite your new resolv.conf file.
    Edit your NetworkManager.conf file:

    sudo
    nano /etc/NetworkManager/NetworkManager.conf

    and add the following:

    dns=dnsmasq

    beneath the lines (navigate using arrow keys) [main] and plugins=ifupdown, keyfile exactly like this with the new line added.

    [main]


    plugins=ifupdown, keyfile


    dns=dnsmasq
    Press Ctrl+x to exit the editor. Enter y to save and then press Enter to overwrite the file.
    Back out of the terminal, and reboot the system


Once all of that is done, try connecting to NordVPN servers via OpenVPN without the block-outside-dns variable and check if you leak.

Let us know how it goes!
 

Offline soldar

  • Super Contributor
  • ***
  • Posts: 3158
  • Country: es
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #18 on: November 01, 2019, 10:20:07 pm »
I cannot be of help with the VPN configuration but a few related comments regarding my experience in China.

Your location says HK but I have felt any restrictions while in HK so either they are new or, more likely, you are in mainland China.

When in China, for a month or two, we have an American phone with unlimited data and, while I don't know the technical details, it works like we are in America. I guess the phone company has an agreement with China and they just VPN directly to America, automatically and transparently. This would allow Americans, especially business people, visiting China to have full access without problem.

I am guessing this solution might also be used by those with contacts outside China although I wonder if the authorities might become suspicious if a foreign phone is used for many months in China or if the foreign phone company might also object to that kind of use.
All my posts are made with 100% recycled electrons and bare traces of grey matter.
 

Offline I wanted a rude username

  • Frequent Contributor
  • **
  • Posts: 627
  • Country: au
  • ... but this username is also acceptable.
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #19 on: November 01, 2019, 11:34:26 pm »
Does EdgeInsider support DNS over HTTPS? If so, enabling it would be the easiest solution. Or just use Firefox.

Good to know about SSH tunnels. I haven't been recommending that solution because I figured the Great Firewall would throttle them the same way based on traffic analysis. Maybe there are enough geeks within the Ministry of Truth that they can't tolerate the inconvenience.
 

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #20 on: November 02, 2019, 12:29:04 am »
I cannot be of help with the VPN configuration but a few related comments regarding my experience in China.

Your location says HK but I have felt any restrictions while in HK so either they are new or, more likely, you are in mainland China.

When in China, for a month or two, we have an American phone with unlimited data and, while I don't know the technical details, it works like we are in America. I guess the phone company has an agreement with China and they just VPN directly to America, automatically and transparently. This would allow Americans, especially business people, visiting China to have full access without problem.

I am guessing this solution might also be used by those with contacts outside China although I wonder if the authorities might become suspicious if a foreign phone is used for many months in China or if the foreign phone company might also object to that kind of use.

Nope, the location flag is just a flag for me, as some people have 00 or Antarctica as their flag. Yes I'm in Mainland China.

And yes, your phone with an American SIM card will work normally as if in America because of Roaming. When in Roaming, the Home HLR is contacted by the Remote (Roaming) HLR of your info and your HLR gives a small percent of your subscriber info, just enough to make the connection through him, using the Roaming Access Network as a path. Same with Chinese Roaming SIM in China, if you have a Chinese SIM card connected to the Roaming Network, you will not be able to open google.com, since the traffic is being redirected by the Home HLR, in China.

Since all foreigners have to register in China on the Police I'm sure that they already have a database who joins images together with signatures, passports together with IMEI/IMSI and habits together with location.
 

Offline OwO

  • Super Contributor
  • ***
  • Posts: 1250
  • Country: cn
  • RF Engineer.
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #21 on: November 02, 2019, 06:01:32 am »
Since all foreigners have to register in China on the Police

Don't do that!!! The rules are on paper only and no one has gotten in trouble for not registering. Also when going through customs always leave "intended address in China" blank and when asked say someone is hosting you and you don't know the address. They will know your whereabouts anyway by face and gait recognition, and foreign SIM info can be correlated with your passport info anyway.
I'm sure they know I'm a "resident" by now because at customs checkpoint they don't even bother asking me questions anymore.
Email: OwOwOwOwO123@outlook.com
 
The following users thanked this post: I wanted a rude username

Offline Black PhoenixTopic starter

  • Super Contributor
  • ***
  • Posts: 1129
  • Country: hk
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #22 on: November 02, 2019, 06:26:08 am »
Since all foreigners have to register in China on the Police

Don't do that!!! The rules are on paper only and no one has gotten in trouble for not registering. Also when going through customs always leave "intended address in China" blank and when asked say someone is hosting you and you don't know the address. They will know your whereabouts anyway by face and gait recognition, and foreign SIM info can be correlated with your passport info anyway.
I'm sure they know I'm a "resident" by now because at customs checkpoint they don't even bother asking me questions anymore.

Really it doesn't bother me. To get the resident visa for 2 years I had to register in the police and provide documentation that I'm married in China to get the sticker on the passport so... I don't have anything to hide so whatever.

As I said in another topic, I'm not a Chinese Citizen, I'm Portuguese. I have the right to question and talk bad about my government, because I vote on the ones I want in power. Here I'm simply a foreigner, I don't have any right to say anything. I have my freedom of thought, that no one will take me. There are a lot of things I see here that are basically wrong in my own though, but I'm no one to say anything specially because I can't change anything and no one likes people imposing their own believes.

The day I feel uncomfortable of being here, the only thing I have to do is pack my stuff and exit the country with my family. So they can know were I go, were I am, what I do, were I exit and enter when I go to Hong Kong, what I bring, whatever. I'm not going to go against the laws of the country.

Even they can read what I am writing now or access my internet traffic. I'm not writing stuff against the government, nor I'm inciting someone to do something against. I just use it to read what I want to read, from various places and different perspectives and create my own opinion about it. But that is my opinion and I don't have the obligation to say it.
« Last Edit: November 02, 2019, 06:29:49 am by Black Phoenix »
 

Offline soldar

  • Super Contributor
  • ***
  • Posts: 3158
  • Country: es
Re: To the Linux Gurus - Problems with HSTS in Firefox
« Reply #23 on: November 02, 2019, 09:20:46 am »
Since all foreigners have to register in China on the Police I'm sure that they already have a database who joins images together with signatures, passports together with IMEI/IMSI and habits together with location.


Ha ha, you give them too much credit. As we say in Spanish "the wolf is not as fierce as it is portrayed". If what you say were true I know a lot of people who would be in serious trouble.

I have been visiting China for over twenty years now and have never registered with the police. I assume when I stay in hotels it is done automatically by the hotel but most of the time I stay in private homes and I have never registered.

In my wife's hometown every day we would pass in front of the police station and I mentioned to her that maybe I should go in and register until one day we finally went in and had a funny and amiable exchange with the police chief there.  My wife translating, obviously.

- (me) Ask him how I can register that I am staying with your family.
- (wife) He says why would you want to register. Are you planing on committing any crimes?
- (me) Um, no.
- (Wife) He says to run along and come back to register before committing any crimes. :)

So after some smiles and chitchat we parted. A very friendly guy who seemed to not know what to do with me.

If Chinese authorities kept close tabs on small administrative infractions I know a lot of people who would be in trouble. I could tell lots of cases and stories but I won't, for obvious reasons.

My interaction with the Chinese police and other authorities has always been friendly and cordial. Even when I get pissy in airport security controls, which I often do because I hate them, they have always been firm but polite.

I feel China is a lot like Spain where there are a ton of rules which are not really enforced and you are not really expected to abide by but if someone runs afoul of the authorities then they can get him on all sorts of things. Keep your head low and you will be fine.

I once read a description of Cuba as "a ruthless dictatorship tempered by total chaos". In many ways this applies to China and to Spain. The rules book says we are playing one game but we all know we are playing a different game.

Getting back to the issue of Internet access, using a roaming foreign phone has always worked for me in the last few years. I have also used other ways like VPN or accessing my home computer using TeamViewer. I have thought of setting up my own VPN server at home or at somebody else's computer but I never did it as it has some complication for me and I never needed it that badly.

I have never really had any serious problem getting around the firewall. Maybe they are tightening up now.
« Last Edit: November 02, 2019, 10:23:33 am by soldar »
All my posts are made with 100% recycled electrons and bare traces of grey matter.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf