Getting a lot of "502 Bad Gateway" errors

[Analog Kid]

Title says it all. This has been happening the last 2 days.
Yesterday I couldn't access the site for a couple hours.
Is this on your end or my end?

[Stray Electron]

  I've been getting them today and it wouldn't post a reply that I made a few hours ago.

[HwAoRrDk]

Same. I assume the servers are getting blasted by AI bot scrapers or something like that.

[themadhippy]

similar here, not had any prolonged outage,but almost every other page gives a 502 bad gateway

[tautech]

+ 1

EEVblog has been painfully slow for a few weeks and I suspect starting when Dave changed the forum ads. 

But there's also another story:
331297 Guests, 112 Users

Near record guests #'s.......Bots ?

Is it time for gnif to be let loose to tidy things up some ?

[Fraser]

I have had the same issue for a couple of days.

I looked at the forum guest stats tonight and it said today was the highest number seen on this forum. It is almost like a DoS attack, overloading the servers. If it is AI scrapers, the situation can only get worse unless Dave knows how to block them.

Fraser

[CatalinaWOW]

This did not start for me until today, but now well over half the times I click on anything on the site I get a bad gateway error.  Including two tries at posting this message.  Maybe the timing and my location will provide some clues for someone.

[xrunner]

This did not start for me until today, but now well over half the times I click on anything on the site I get a bad gateway error.  Including two tries at posting this message.  Maybe the timing and my location will provide some clues for someone.

Same thing for me for a couple of days now ... 

[Analog Kid]

Now it's literally every other click on this site gives me that error.

[thm_w]

Dropping now: " 218236 Guests, 66 Users "

I'm impressed that the server can handle that many views. We used to be able to see what thread they were looking at but no more.

[PCB.Wiz]

Yup, seems a bit variable.
Sometimes a refresh will spin-a-while then work, other times an instant 502 reappears.

[EEVblog]

EEVblog has been painfully slow for a few weeks and I suspect starting when Dave changed the forum ads. 
But there's also another story:
331297 Guests, 112 Users

331k guest requests will do it alright. Nothing to do with the ads.

[EEVblog]

Getting hammered!

[Analog Kid]

Please pardon my ignorance of how that Intertube stuff works, but what's the relationship between all those "guests" (=bots?) and the 502 errors?

[EEVblog]

Baidu bots

[Analog Kid]

Can they be blocked?
If so, what would the consequences of that be?

[thm_w]

Can they be blocked?
If so, what would the consequences of that be?

At a certain point you block IP ranges of legitimate users and they can't use the forum:
https://www.eevblog.com/forum/chat/baidu-is-trawling-the-eevblog-forum/msg6316476/#msg6316476

[EEVblog]

Update: the culprit is Shanghai263 again, and our anti-bot service was attacked and crashed. The infrastructure has a fail-safe configuration so that if this occurs it doesn't block the entire site.
Looks to be back up now.

[EEVblog]

Can they be blocked?
If so, what would the consequences of that be?

At a certain point you block IP ranges of legitimate users and they can't use the forum:
https://www.eevblog.com/forum/chat/baidu-is-trawling-the-eevblog-forum/msg6316476/#msg6316476

Correct, we have extensive block lists and have unfortunately accidently blocked legit users before.

[fourfathom]

Still getting regular 502 errors, so I guess it's not solved yet.

[iMo]

..mee too..

[chilternview]

Still getting the 502 errors, and the forum is very slow.

[gnif]

Still working on it, getting close now.

We do not block IPs at all, we used to. Now we force bad IPs/subnets through a captcha validation, otherwise we end up blocking VPN users completely.

[CatalinaWOW]

Any idea on how you got on someone's shit list?

[Analog Kid]

Question:

Would it help to require all users to solve a CloudFlare captcha (like the simplest "click on the box" one) before first logging on or accessing the site?

I wouldn't mind having to do this at all if it helped reduce the bot attacks.