Getting a lot of "502 Bad Gateway" errors

[Kean]

I got this captcha a couple of times now, but not nearly as often as the combined captchas yesterday.
Don't get me wrong. I understand the neccessity and i don't really mind that single captcha.
But it *is* changed behaviour.

Yes, I am regularly seeing the CrowdSec captcha.  Note that this is very much not the same as CrowdStrike!

I presume it is happening because with CloudFlare enabled the requests are reverse proxied, so the IP seen by the EEVblog servers is not the originating IP but one from CF.

As mentioned in another post, there are configuration settings that can be made to better handle this and restore the originating IP.
https://www.eevblog.com/forum/news/getting-a-lot-of-502-bad-gateway-errors/msg6357054/#msg6357054

Edit: Yes, in fact I can see the IP logged against this post is not my companies public IP address, but one associated with CF.

[EEVblog]

Don't get me wrong. I understand the neccessity and i don't really mind that single captcha.
But it *is* changed behaviour.

gnif may have changed something with the server side protection. CDN level Cloudflare authentication is turned off.

[EEVblog]

The number of unique daily page requests to Cloudflare for the website is bonkers.

[EEVblog]

US mostly.

[Kean]

87k unique visitors PER HOUR!

What percentage are being passed through?

[EEVblog]

87k unique visitors PER HOUR!
What percentage are being passed through?

I only have those public Guests and Users numbers in the stats.

[KungFuJosh]

I still have "Block AI training bots" enabled in Cloudflare (in addition to the multitude of filters gnif has set up server side), I'll disable that right now as a trial and see if the Guests increase again. 10am Monday Sydney time for reference.

There's no reason to turn that off. I'd keep all the invisible bot protections active.

[peter-h]

WTF

I must be misunderstanding the sentiment behind "WTF" but I was only trying to help.

Precursor gives you the same DDOS protection as the Under Attack mode, but is transparent to users.

[EEVblog]

WTF

I must be misunderstanding the sentiment behind "WTF" but I was only trying to help.
Precursor gives you the same DDOS protection as the Under Attack mode, but is transparent to users.

I meant that it's not available to Australians. I've looked, my account doesn't have it.

[EEVblog]

They have a Bot Fight mode:
https://developers.cloudflare.com/bots/get-started/bot-fight-mode/

[EEVblog]

Precursor is only available on the Enterprise plan, which is quote based which means "if you have to ask, you can't afford it".

https://blog.cloudflare.com/introducing-precursor/

[KungFuJosh]

Precursor is only available on the Enterprise plan, which is quote based which means "if you have to ask, you can't afford it".

https://blog.cloudflare.com/introducing-precursor/

It might be a regional deployment thing. I hadn't seen the option before, but I followed the link from the article, and the option was there in my free plan.

[EEVblog]

Precursor is only available on the Enterprise plan, which is quote based which means "if you have to ask, you can't afford it".

https://blog.cloudflare.com/introducing-precursor/

It might be a regional deployment thing. I hadn't seen the option before, but I followed the link from the article, and the option was there in my free plan.

I also followed the link, not there 

[lowimpedance]

Guest numbers creeping up , over 60k !
Just the usual after work spike or ...

[peter-h]

The CF site says

"Precursor will be free to use until our GA release later this year."

Must be how we got it for free.

The big Q is whether they will start charging to people who got it for free early on.

Our site is virtually hosted with the usual ton of bw so we would have to move to Anubis.

[EEVblog]

Guest numbers creeping up , over 60k !
Just the usual after work spike or ...

Dammit, yep, that didn't last long.

[gnif]

Guest numbers creeping up , over 60k !
Just the usual after work spike or ...

Because we have changed to using CF entirely I have had to disable/remove the CrowdSec layer we had implemented, these to systems are not compatible.
Unfortunately the result is many bots that I was already properly filtering is now relying on CF to filter, as a result of which I'd expect to see numbers rise again.

[EEVblog]

This is interesting, look who is the biggest.
Looks like I can block individual bots.

[gnif]

The bots that advertise who they are, are not the problem, it's those that hide as real users that are.

[EEVblog]

I just blocked Apple and Bing as a test. Will see if the Guests drop over the next few hours.

[gnif]

I just blocked Apple and Bing as a test. Will see if the Guests drop over the next few hours.

This is a bad idea, these bots already identified themselves properly and we have been explicitly allowing them. The major bots that identify themselves actually behave properly and respect crawl rates, and we can throttle them at the server side. The bots that pretend to be real users using proxy services are the issue, we can't identify them as bots or throttle them. CloudFlare wont give you a "switch" for these bots, we are just hoping their heuristics are better then ours at identifying them.

The guest count is not a good metric to test with, SMF retains this count for hours, and does not reflect actual server load.

[EEVblog]

The bots that advertise who they are, are not the problem, it's those that hide as real users that are.

Is there a chance they are bypassing the DNS and going to the server IP directly?

[gnif]

The bots that advertise who they are, are not the problem, it's those that hide as real users that are.

Is there a chance they are bypassing the DNS and going to the server IP directly?

Yes, but we have technical reasons why we need to keep it this way. Lets discuss this via email.

[EEVblog]

I just blocked Apple and Bing as a test. Will see if the Guests drop over the next few hours.

This is a bad idea, these bots already identified themselves properly and we have been explicitly allowing them. The major bots that identify themselves actually behave properly and respect crawl rates, and we can throttle them at the server side. The bots that pretend to be real users using proxy services are the issue, we can't identify them as bots or throttle them. CloudFlare wont give you a "switch" for these bots, we are just hoping their heuristics are better then ours at identifying them.

The guest count is not a good metric to test with, SMF retains this count for hours, and does not reflect actual server load.

Ok, thanks, have re-enabled them.

[peter-h]

Yes, but we have technical reasons why we need to keep it this way. Lets discuss this via email.

This is a normal issue; you set up a bypass in CF.

The real server should be firewalled to accept only CF traffic, plus obviously any other stuff you need internally.