Author Topic: Bad gateway 502  (Read 1013 times)

0 Members and 1 Guest are viewing this topic.

Offline Monkeh

  • Super Contributor
  • ***
  • Posts: 4638
  • Country: gb
Re: Bad gateway 502
« Reply #25 on: March 17, 2017, 04:30:06 AM »
Sounds like Dave's in for a server upgrade again.

Time for a couple SSDs?
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 3935
  • Country: us
  • DavidH
Re: Bad gateway 502
« Reply #26 on: March 17, 2017, 05:18:05 AM »
Sounds like Dave's in for a server upgrade again.

Time for a couple SSDs?

Let us hope it goes better than Drew's server upgrades. :)

https://beerspill.fark.com/drunkfarksysadmin.jpg
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 7932
  • Country: nz
    • Taupaki Technologies Ltd.
Re: Bad gateway 502
« Reply #27 on: March 17, 2017, 06:44:49 AM »
@gnif
Some more info in another thread:
http://www.eevblog.com/forum/chat/502-forum-errors/

Along with the info other incidences I've had could involve any individual thread and in some cases a whole board.

I get that you don't think it's browser based but the more info we can send your way the better.
When we get consistent errors we can generally work around it some...say we want to look at a particular thread in the TE board but trying to open the TE board produces errors....if we know we've posted in it we can look up our posts in our profile and access the thread that way or from New or Last Post links.
Avid Rabid Hobbyist & NZ Siglent Distributor
 

Offline apelly

  • Supporter
  • ****
  • Posts: 903
  • Country: nz
Re: Bad gateway 502
« Reply #28 on: March 17, 2017, 07:24:51 AM »
I really don't know why people are challenging the answer to the issue
Neither do I. I guess there's always someone thinks they're smarter.

I haven't noticed any issues serious enough to bother mentioning, so it doesn't matter to me, but thanks for taking the time to explain.

FWIW I had assumed this occasional 502 was a cloudflare issue. Even though the message specifically explains there is no response from the server, it is proudly covered in their branding.
I'd rather a Google clue, link, or some theory than "do this" (generally)
 

Offline gnif

  • Administrator
  • *****
  • Posts: 560
  • Country: au
Re: Bad gateway 502
« Reply #29 on: March 17, 2017, 08:11:08 AM »
FWIW I had assumed this occasional 502 was a cloudflare issue. Even though the message specifically explains there is no response from the server, it is proudly covered in their branding.

CloudFlare is just reflecting the 502 that the server is sending, if it was only a CloudFlare issue I would not see the 502 responses in the server's logs.

I have no issue at all with people reporting information, in fact it is encouraged, but unless someone actually sends me a PM, I don't always see or know if there is an issue (In this instance I stumbled across this thread).

Along with the info other incidences I've had could involve any individual thread and in some cases a whole board.

It really depends on what has been locked, there are tons of different updates that occur across the database all the time, even thread read counts.

Sounds like Dave's in for a server upgrade again.

Time for a couple SSDs?

This only makes sense if there is a HDD related bottleneck, this has not been established yet, nor do the current metrics point to such an issue.

I can think of a couple of reasons.  Your initial explanation was not consistent with observed symptoms.

How so? I stated the server is experiencing a load issue, this has not changed.

Your force/refresh (Ctrl+F5) or clearing cookies/cache is only working by coincidence, its just the fact that you made another request to the server when it was a little less loaded and as such it was able to return a result.
« Last Edit: March 17, 2017, 08:13:36 AM by gnif »
HostFission - Full Server Monitoring and Management Solutions.
https://hostfission.com/
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 3935
  • Country: us
  • DavidH
Re: Bad gateway 502
« Reply #30 on: March 17, 2017, 09:22:09 AM »
I can think of a couple of reasons.  Your initial explanation was not consistent with observed symptoms.

How so? I stated the server is experiencing a load issue, this has not changed.

If the problem is a load issue, then improving performance will rectify it.  If a limited number of processing threads are getting stuck because database rows are stuck in a locked condition, then the problem is why they are locked or why they were not unlocked.

Or are you suggesting that specific database rows get locked for hours at a time and this is just because of insufficient server capacity?  It seems odd that such a state would persist for such a long time.
 

Online wilfred

  • Super Contributor
  • ***
  • Posts: 3911
  • Country: au
Re: Bad gateway 502
« Reply #31 on: March 17, 2017, 09:33:34 AM »
I really don't know why people are challenging the answer to the issue
Neither do I. I guess there's always someone thinks they're smarter.
It is because the problem hasn't been fully explained. Regardless of the justification for that, that is the reason people question the answer. If this was a forum where people swapped Quinoa and Kale recipies that might not be the case. But it isn't such a forum, and everyone has an opinion.
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 7932
  • Country: nz
    • Taupaki Technologies Ltd.
Re: Bad gateway 502
« Reply #32 on: March 17, 2017, 09:36:23 AM »
Or are you suggesting that specific database rows get locked for hours at a time and this is just because of insufficient server capacity?  It seems odd that such a state would persist for such a long time.
This.

Not just hours, days in some cases and a browser cleanup always fixes it.....until the next time it happens.

I won't challenge gnif on any of this, only reporting observed behaviour.
I might add at times my connection speeds are painfully slow (10's of Kb/s), even at full speed my DSL is only 1.7 Mb/s.  :(
Tethered to my phone it's heaps better however if I'm getting errors already, nothing changes.
Avid Rabid Hobbyist & NZ Siglent Distributor
 

Online wilfred

  • Super Contributor
  • ***
  • Posts: 3911
  • Country: au
Re: Bad gateway 502
« Reply #33 on: March 17, 2017, 09:41:12 AM »
I have no issue at all with people reporting information, in fact it is encouraged, but unless someone actually sends me a PM, I don't always see or know if there is an issue (In this instance I stumbled across this thread).

Do you really want a PM? I thought since you volunteered your time that would be an unwelcome intrusion. And not all forum members know that you are willing to receive PM's or that you are the person to send reports to.

Besides, since the system apparently logs the error and the error is related to some database performance issues it seems that raising an alert automatically at that point would be better. Especially since it resolves itself when load drops and isn't demanding immediate attention.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 3935
  • Country: us
  • DavidH
Re: Bad gateway 502
« Reply #34 on: March 18, 2017, 01:48:10 AM »
Or are you suggesting that specific database rows get locked for hours at a time and this is just because of insufficient server capacity?  It seems odd that such a state would persist for such a long time.
This.

Not just hours, days in some cases and a browser cleanup always fixes it.....until the next time it happens.

I am assuming there is more than one problem which returns the 502 gateway error.

In the past I got that error for days at a time and clearing cookies and cache fixed it; it would be awfully coincidental if after days, I just happened to clear cookies and cache at the same time that the problem was resolved on the server side.

The current error and most of them in the past were not affected by clearing cookies and cache.

And then there is a third thing going on where Cloudflare issues the redirect destroying the browser history so you cannot recover your post or reply.
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 7932
  • Country: nz
    • Taupaki Technologies Ltd.
Re: Bad gateway 502
« Reply #35 on: March 18, 2017, 05:22:36 AM »
Or are you suggesting that specific database rows get locked for hours at a time and this is just because of insufficient server capacity?  It seems odd that such a state would persist for such a long time.
This.

Not just hours, days in some cases and a browser cleanup always fixes it.....until the next time it happens.

I am assuming there is more than one problem which returns the 502 gateway error.

In the past I got that error for days at a time and clearing cookies and cache fixed it; it would be awfully coincidental if after days, I just happened to clear cookies and cache at the same time that the problem was resolved on the server side.

The current error and most of them in the past were not affected by clearing cookies and cache.

And then there is a third thing going on where Cloudflare issues the redirect destroying the browser history so you cannot recover your post or reply.
I may have had that once IIRC but Chrome pretty much plays nice with the forum.
I get 2 gateway fails, Chrome reports 502 (most common) and Cloudflare much less often.
Avid Rabid Hobbyist & NZ Siglent Distributor
 

Offline StillTrying

  • Frequent Contributor
  • **
  • Posts: 544
  • Country: gb
  • 100% Brand New and High Quality, in theory.
Re: Bad gateway 502
« Reply #36 on: March 21, 2017, 10:00:11 AM »
502s all over the place 5 mins ago, noticed site slowing just before the 502s.

Didn't do any clearing of cookies or caches - never have,  just waited 2 mins.
 

Online Iwanushka

  • Regular Contributor
  • *
  • Posts: 153
  • Country: lt
Re: Bad gateway 502
« Reply #37 on: March 21, 2017, 11:12:02 AM »
looks like we have to many users especially when entire US/AUS/NZ jumps in maybe it's time for nginx and load balancing if Dave has the money for multiple Servers/DB clustering
When all you've got is a hammer, everything starts looking like a nail.- Attrition.
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 1236
  • Country: us
Re: Bad gateway 502
« Reply #38 on: March 21, 2017, 11:20:07 AM »
I frequently get the Bad Gateway error, usually I can resolve it by clearing my browser cache. I tried this after noticing when it starts happening in one browser I can load the site just fine in a different browser on the same PC.
 

Offline StillTrying

  • Frequent Contributor
  • **
  • Posts: 544
  • Country: gb
  • 100% Brand New and High Quality, in theory.
Re: Bad gateway 502
« Reply #39 on: March 21, 2017, 12:41:10 PM »
Right. Page 1 is stuck on 502, but page 2 is fine of:
http://www.eevblog.com/forum/beginners/change-of-capacitor-capacitance-in-a-circuit/

Later...
I used ALL in the end to see pages 1+2.
« Last Edit: March 21, 2017, 01:04:22 PM by StillTrying »
 

Online dimkasta

  • Regular Contributor
  • *
  • Posts: 63
  • Country: gr
Re: Bad gateway 502
« Reply #40 on: March 21, 2017, 12:42:15 PM »
Gnif may I ask what maintenance operation locks the db for so long? I don't know if you want to reveal that info publicly though.

You could try increasing the nginx timeout (duh) temporarily until you put your finger on the db issue.
Also, have a look in case there is any forgotten MyISAM causing table locks somewhere that it shouldn't.
And you could fiddle with innodb io capacity if you see any issues there and you can spare disk i/o.

An operation taking so long on rows rather points to some kind of i/o bottleneck. Are you using a swap file ?
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 7932
  • Country: nz
    • Taupaki Technologies Ltd.
Re: Bad gateway 502
« Reply #41 on: Today at 10:12:51 AM »
Had 502's again a few minutes ago, right on midday NZ time, 9am Aussie.
Net was slow so switched to a USB tethered cell for service....problem gone.  :)

Just letting you know so you can check logs.
Avid Rabid Hobbyist & NZ Siglent Distributor
 

Online wilfred

  • Super Contributor
  • ***
  • Posts: 3911
  • Country: au
Re: Bad gateway 502
« Reply #42 on: Today at 10:14:28 AM »
It just happened to me @10:02am AEST. I decided to try waiting and 3 minutes later it came good. Is 10am the time the DB maint gets done?
 

Offline StillTrying

  • Frequent Contributor
  • **
  • Posts: 544
  • Country: gb
  • 100% Brand New and High Quality, in theory.
Re: Bad gateway 502
« Reply #43 on: Today at 10:22:31 AM »
http://www.eevblog.com/forum/login2/

Was stuck on 502, for a couple of minuettes.
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 7932
  • Country: nz
    • Taupaki Technologies Ltd.
Re: Bad gateway 502
« Reply #44 on: Today at 11:19:16 AM »
Avid Rabid Hobbyist & NZ Siglent Distributor
 

Online xrunner

  • Super Contributor
  • ***
  • Posts: 2075
  • Country: us
Re: Bad gateway 502
« Reply #45 on: Today at 11:32:39 AM »
I get it a lot lately too but I never say anything here because the message states that the main players know about the problem.  :-//
I am a Test Equipment Addict (TEA) - by virtue of this forum signature, I have now faced my addiction
 

Offline tooki

  • Frequent Contributor
  • **
  • Posts: 853
  • Country: ch
Re: Bad gateway 502
« Reply #46 on: Today at 11:49:29 AM »
Not just hours, days in some cases and a browser cleanup always fixes it.....until the next time it happens.
No, it doesn't. What fixes it is waiting a few moments and trying again — which is exactly what is happening when people go to their settings, clear the cache, etc. It's the time delay that's fixing the problem, not the cache clearing!!
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 7932
  • Country: nz
    • Taupaki Technologies Ltd.
Re: Bad gateway 502
« Reply #47 on: Today at 11:58:25 AM »
Not just hours, days in some cases and a browser cleanup always fixes it.....until the next time it happens.
No, it doesn't. What fixes it is waiting a few moments and trying again — which is exactly what is happening when people go to their settings, clear the cache, etc. It's the time delay that's fixing the problem, not the cache clearing!!
I'll stand by what I have said.  :P

And this won't work ATM either: http://www.eevblog.com/forum/unreadreplies/?start=0

The forum has been trouble free for a few days.....until this morning.  :scared:

All we can do is relay as much info as possible to gnif while he tries to get to the bottom of it.
« Last Edit: Today at 12:00:17 PM by tautech »
Avid Rabid Hobbyist & NZ Siglent Distributor
 

Offline tooki

  • Frequent Contributor
  • **
  • Posts: 853
  • Country: ch
Re: Bad gateway 502
« Reply #48 on: Today at 12:27:00 PM »
Not just hours, days in some cases and a browser cleanup always fixes it.....until the next time it happens.
No, it doesn't. What fixes it is waiting a few moments and trying again — which is exactly what is happening when people go to their settings, clear the cache, etc. It's the time delay that's fixing the problem, not the cache clearing!!
I'll stand by what I have said.  :P

And this won't work ATM either: http://www.eevblog.com/forum/unreadreplies/?start=0

The forum has been trouble free for a few days.....until this morning.  :scared:

All we can do is relay as much info as possible to gnif while he tries to get to the bottom of it.
You made an absolute statement, and as such, a single point of conflicting data disproves the claim. I've never seen a forced refresh fix it, ergo, the claim is disproven.

Regardless, the fact that the unread replies page isn't currently working for you, but is working for me, lends credence to it being a server-side issue involving specific record locks, as gnif has explained.
 

Offline StillTrying

  • Frequent Contributor
  • **
  • Posts: 544
  • Country: gb
  • 100% Brand New and High Quality, in theory.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf