NEWS: Forum reverted to backup

[EEVblog]

We had to revert to a backup of the forum last night.
I'm still need to evaluate the aftermath, this is just a thread to discuss what happened.

[tautech]

Found this on Discord however even that thread has vanished:

[Whales]

Yes, that thread was made yesterday.  https://www.eevblog.com/forum/news/did-everyones-forum-post-counts-just-drop-by-between-1-and-5/msg3561390/#msg3561390

Hoping that the beast is tamed.  Thankyou Dave.

[I wanted a rude username]

Thanks for the update. Glad I created that thread, if it's what tipped you off.

> TE thread is down to 18 pages.
> Our entire lives are gone

RIP all the comments we made between the backup time and restoration time. Feels like an alternate universe just got snuffed out.

[tom66]

Does the SMF forum not support database transactioning?  Before doing a big operation, start a SQL transaction, see if it is bad, if it is, roll it back...

You don't want to transaction every operation (huge performance hit) but if you're ever unsure about what something will do, it should be done.

Not sure if you can tell SQL 'any operation in the future is to be put in the transaction log' and then turn that off.

[rsjsouza]

If it helps finding the extent of the data loss... I usually post once or twice a day and my post history shows a gap between a post in April 24th and the next on April 29th.

[Hannibal_Hector]

This is what backups are for. You can put al kinds of levers in place to ensure nothing ever breaks, nothing is ever lost, nothing ever goes down... but there is always that one thing that gets you. And that's where the backups come in, and putting things in perspective, nobody got hurt! (I think?)

[andy3055]

Earlier today, it was saying that the site is down for maintenance. Is that also related?

[JohnnyMalaria]

Earlier today, it was saying that the site is down for maintenance. Is that also related?

Yes, Dave shut the site to address his...um...nuclear attack on some threads.

I was reading the thread as it was all unfolding. Most surreal. Though for some this happened yesterday, it was earlier today for me. Either that, or I can't account for a day

[Whales]

Earlier today, it was saying that the site is down for maintenance. Is that also related?

Probably.  Loading a backup for a massive database takes some time and you don't want people interacting with an incomplete database, that can lead to all sorts of inconsistencies and craziness (assuming it even works at all).  Even viewing a topic before the full import is complete could lead to thousands of errors (eg posts and thread exists, but certain user's profiles do not).

[EEVblog]

Unfortunately I don't think the backup went back far enough, I'm still seeing huge numbers of deleted threads, about half as many as before, so progress, but still, I wouldn't rule out another reversion.
In this case we coincidentaly had a partial day backup and I believe that was used to recover. We might need to go back full day at least, or try an fix the threads manually from here.

[gnavigator1007]

Completely forgot about the discord. I was wondering what was going on when I received 35 email notifications all at once for each topic I was subscribed to that had been removed. Received more some hours later for topics being moved. At least it was nothing malicious. Figure Dave is probably too busy to answer, but does anybody know how often the forum is backed up & how much storage is required to maintain?

[Whales]

[...] but still, I wouldn't rule out another reversion.

Can you temporarily freeze?  That gives time for TEA people to backup their new posts and time for you to scrounge through backups, all the while avoiding us making new stuff.

[EEVblog]

Yep, will freeze posting until a decision is made on what direction to take.

[EEVblog]

I've re-enabled posting, and am currently going through restroing some accidently deleted theads.
As a result you may see your post count increase back toward normal level as threads are restored if you have posted in them.

[magic]

RIP all the comments we made between the backup time and restoration time. Feels like an alternate universe just got snuffed out.

Yep, a good chunk of Sunday (6AM~6PM UTC or so) is MIA.

[EEVblog]

I'm pretty sure I have now undeleted almost every thread that was accidently deleted.
I may have missed a couple, and if you are aware of any please let me know.
Everyone's post counts should be back to almost normal, although there will almost certainly be a few casulty posts, sorry about that.
The recovery from my goof was not 100%, but should be pretty darn close.
Thanks to gnif for a late night backup recovery and writing some script to fix some things.
Sorry to everyone for the goof up, it was entirely my fault 

Although having said that, I do believe it was ultimately caused by forum bug.
For those interested, I thought I'd be clever and search for keywords from a particular user which allowed me to mass delete posts 50 at a time (at request of the owner).
So although I was selecting individual posts from a user, the forum decided to delete the entire thread that contained that post. And I didn't realise this until someone posted that thread asking why their post count had dropped 5% or something.

[tautech]

Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/

[Ian.M]

For those interested, I thought I'd be clever and search for keywords from a particular user which allowed me to mass delete posts 50 at a time (at request of the owner).
So although I was selecting individual posts from a user, the forum decided to delete the entire thread that contained that post. And I didn't realise this until someone posted that thread asking why their post count had dropped 5% or something.

Obviously, if a user gets banned and all or most of their posts  have unacceptable content, you may need to mass delete them, but why bend over backwards to <expletive> up the forum at the request of a user?   Deleting posts that weren't unacceptable content, at best causes confusion, and if done in bulk for/by a single user smacks of "I'll take my ball and go home!".  Obviously, given the tools you choose to make available to us users, you cant easily prevent users stripping their content from the forum, but if they want to be immature, at least let them have to do all the grunt work to do so rather than wasting your and Gnif's time.

IMHO when you post a message on a public forum, you are granting the forum the right to display the content you have created in perpetuity^*.  Once you hit [Post] or [Send] its out of your control. 

*  unless the forum offers users a more restrictive content creators license, granting the users more rights.

[EEVblog]

For those interested, I thought I'd be clever and search for keywords from a particular user which allowed me to mass delete posts 50 at a time (at request of the owner).
So although I was selecting individual posts from a user, the forum decided to delete the entire thread that contained that post. And I didn't realise this until someone posted that thread asking why their post count had dropped 5% or something.

Obviously, if a user gets banned and all or most of their posts  have unacceptable content, you may need to mass delete them, but why bend over backwards to <expletive> up the forum at the request of a user?   Deleting posts that weren't unacceptable content, at best causes confusion, and if done in bulk for/by a single user smacks of "I'll take my ball and go home!".  Obviously, given the tools you choose to make available to us users, you cant easily prevent users stripping their content from the forum, but if they want to be immature, at least let them have to do all the grunt work to do so rather than wasting your and Gnif's time.

IMHO when you post a message on a public forum, you are granting the forum the right to display the content you have created in perpetuity^*.  Once you hit [Post] or [Send] its out of your control. 

*  unless the forum offers users a more restrictive content creators license, granting the users more rights.

Sorry but I won't go into details about this.
This stuff up was entirely my doing, I tried to do something fancy and it backfired.
If it wasn't for the technical stuff up/bug, it would have been trivial work and no one would have noticed.
A similar thing could happen in theory during any routine admin cleanup task like spam account purging etc.

[BradC]

Thankfully nothing I post is worth reading!

A nice reminder in "whoopsie" recovery though.

[Ed.Kloonk]

Thanks for all you do, Dave.

Thanks gnif.

 

[EEVblog]

Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/

Who's posts are missing?

[Muttley Snickers]

I'm still down around 100 posts but not that concerned about it. I do wonder though, what would happen if a member had reached 1000 posts and gained access to the supporters lounge but are now back under 1000 due to the glitch, do they still have access?

[EEVblog]

I do wonder though, what would happen if a member had reached 1000 posts and gained access to the supporters lounge but are now back under 1000 due to the glitch, do they still have access?

I can fix that manually.