EEVblog Electronics Community Forum

EEVblog => News => Topic started by: EEVblog on May 02, 2021, 09:40:39 pm

Title: NEWS: Forum reverted to backup
Post by: EEVblog on May 02, 2021, 09:40:39 pm
We had to revert to a backup of the forum last night.
I'm still need to evaluate the aftermath, this is just a thread to discuss what happened.
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 02, 2021, 09:58:47 pm
Found this on Discord however even that thread has vanished:

(https://cdn.discordapp.com/attachments/662871717667930142/838397721513427004/unknown.png)
Title: Re: NEWS: Forum reverted to backup
Post by: Whales on May 02, 2021, 10:11:00 pm
Yes, that thread was made yesterday.  https://www.eevblog.com/forum/news/did-everyones-forum-post-counts-just-drop-by-between-1-and-5/msg3561390/#msg3561390 (https://www.eevblog.com/forum/news/did-everyones-forum-post-counts-just-drop-by-between-1-and-5/msg3561390/#msg3561390)

Hoping that the beast is tamed.  Thankyou Dave.
Title: Re: NEWS: Forum reverted to backup
Post by: I wanted a rude username on May 02, 2021, 10:18:47 pm
Thanks for the update. Glad I created that thread, if it's what tipped you off.

> TE thread is down to 18 pages.
> Our entire lives are gone

RIP all the comments we made between the backup time and restoration time. Feels like an alternate universe just got snuffed out.
Title: Re: NEWS: Forum reverted to backup
Post by: tom66 on May 02, 2021, 10:27:44 pm
Does the SMF forum not support database transactioning?  Before doing a big operation, start a SQL transaction, see if it is bad, if it is, roll it back...

You don't want to transaction every operation (huge performance hit) but if you're ever unsure about what something will do, it should be done.

Not sure if you can tell SQL 'any operation in the future is to be put in the transaction log' and then turn that off.
Title: Re: NEWS: Forum reverted to backup
Post by: rsjsouza on May 02, 2021, 10:34:52 pm
If it helps finding the extent of the data loss... I usually post once or twice a day and my post history shows a gap between a post in April 24th and the next on April 29th.
Title: Re: NEWS: Forum reverted to backup
Post by: Hannibal_Hector on May 02, 2021, 10:35:54 pm
This is what backups are for. You can put al kinds of levers in place to ensure nothing ever breaks, nothing is ever lost, nothing ever goes down... but there is always that one thing that gets you. And that's where the backups come in, and putting things in perspective, nobody got hurt! (I think?)
Title: Re: NEWS: Forum reverted to backup
Post by: andy3055 on May 02, 2021, 10:40:19 pm
Earlier today, it was saying that the site is down for maintenance. Is that also related?
Title: Re: NEWS: Forum reverted to backup
Post by: JohnnyMalaria on May 02, 2021, 10:49:36 pm
Earlier today, it was saying that the site is down for maintenance. Is that also related?

Yes, Dave shut the site to address his...um...nuclear attack on some threads.

I was reading the thread as it was all unfolding. Most surreal. Though for some this happened yesterday, it was earlier today for me. Either that, or I can't account for a day :)
Title: Re: NEWS: Forum reverted to backup
Post by: Whales on May 02, 2021, 10:50:24 pm
Earlier today, it was saying that the site is down for maintenance. Is that also related?

Probably.  Loading a backup for a massive database takes some time and you don't want people interacting with an incomplete database, that can lead to all sorts of inconsistencies and craziness (assuming it even works at all).  Even viewing a topic before the full import is complete could lead to thousands of errors (eg posts and thread exists, but certain user's profiles do not).
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 02, 2021, 11:17:57 pm
Unfortunately I don't think the backup went back far enough, I'm still seeing huge numbers of deleted threads, about half as many as before, so progress, but still, I wouldn't rule out another reversion.
In this case we coincidentaly had a partial day backup and I believe that was used to recover. We might need to go back full day at least, or try an fix the threads manually from here.
Title: Re: NEWS: Forum reverted to backup
Post by: gnavigator1007 on May 02, 2021, 11:20:04 pm
Completely forgot about the discord. I was wondering what was going on when I received 35 email notifications all at once for each topic I was subscribed to that had been removed. Received more some hours later for topics being moved. At least it was nothing malicious. Figure Dave is probably too busy to answer, but does anybody know how often the forum is backed up & how much storage is required to maintain?
Title: Re: NEWS: Forum reverted to backup
Post by: Whales on May 02, 2021, 11:54:18 pm
[...] but still, I wouldn't rule out another reversion.

Can you temporarily freeze?  That gives time for TEA people to backup their new posts and time for you to scrounge through backups, all the while avoiding us making new stuff.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 12:09:05 am
Yep, will freeze posting until a decision is made on what direction to take.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 02:40:18 am
I've re-enabled posting, and am currently going through restroing some accidently deleted theads.
As a result you may see your post count increase back toward normal level as threads are restored if you have posted in them.
Title: Re: NEWS: Forum reverted to backup
Post by: magic on May 03, 2021, 05:22:28 am
RIP all the comments we made between the backup time and restoration time. Feels like an alternate universe just got snuffed out.
Yep, a good chunk of Sunday (6AM~6PM UTC or so) is MIA.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 05:26:41 am
I'm pretty sure I have now undeleted almost every thread that was accidently deleted.
I may have missed a couple, and if you are aware of any please let me know.
Everyone's post counts should be back to almost normal, although there will almost certainly be a few casulty posts, sorry about that.
The recovery from my goof was not 100%, but should be pretty darn close.
Thanks to gnif for a late night backup recovery and writing some script to fix some things.
Sorry to everyone for the goof up, it was entirely my fault  |O

Although having said that, I do believe it was ultimately caused by forum bug.
For those interested, I thought I'd be clever and search for keywords from a particular user which allowed me to mass delete posts 50 at a time (at request of the owner).
So although I was selecting individual posts from a user, the forum decided to delete the entire thread that contained that post. And I didn't realise this until someone posted that thread asking why their post count had dropped 5% or something.
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 03, 2021, 06:00:51 am
Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/ (https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/)
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/ (https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/)
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/ (https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/)
Title: Re: NEWS: Forum reverted to backup
Post by: Ian.M on May 03, 2021, 06:04:15 am
For those interested, I thought I'd be clever and search for keywords from a particular user which allowed me to mass delete posts 50 at a time (at request of the owner).
So although I was selecting individual posts from a user, the forum decided to delete the entire thread that contained that post. And I didn't realise this until someone posted that thread asking why their post count had dropped 5% or something.
Obviously, if a user gets banned and all or most of their posts  have unacceptable content, you may need to mass delete them, but why bend over backwards to <expletive> up the forum at the request of a user?   Deleting posts that weren't unacceptable content, at best causes confusion, and if done in bulk for/by a single user smacks of "I'll take my ball and go home!".  Obviously, given the tools you choose to make available to us users, you cant easily prevent users stripping their content from the forum, but if they want to be immature, at least let them have to do all the grunt work to do so rather than wasting your and Gnif's time.

IMHO when you post a message on a public forum, you are granting the forum the right to display the content you have created in perpetuity*.  Once you hit [Post] or [Send] its out of your control. 

*  unless the forum offers users a more restrictive content creators license, granting the users more rights.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 06:10:32 am
For those interested, I thought I'd be clever and search for keywords from a particular user which allowed me to mass delete posts 50 at a time (at request of the owner).
So although I was selecting individual posts from a user, the forum decided to delete the entire thread that contained that post. And I didn't realise this until someone posted that thread asking why their post count had dropped 5% or something.
Obviously, if a user gets banned and all or most of their posts  have unacceptable content, you may need to mass delete them, but why bend over backwards to <expletive> up the forum at the request of a user?   Deleting posts that weren't unacceptable content, at best causes confusion, and if done in bulk for/by a single user smacks of "I'll take my ball and go home!".  Obviously, given the tools you choose to make available to us users, you cant easily prevent users stripping their content from the forum, but if they want to be immature, at least let them have to do all the grunt work to do so rather than wasting your and Gnif's time.

IMHO when you post a message on a public forum, you are granting the forum the right to display the content you have created in perpetuity*.  Once you hit [Post] or [Send] its out of your control. 

*  unless the forum offers users a more restrictive content creators license, granting the users more rights.

Sorry but I won't go into details about this.
This stuff up was entirely my doing, I tried to do something fancy and it backfired.
If it wasn't for the technical stuff up/bug, it would have been trivial work and no one would have noticed.
A similar thing could happen in theory during any routine admin cleanup task like spam account purging etc.
Title: Re: NEWS: Forum reverted to backup
Post by: BradC on May 03, 2021, 06:14:25 am
Thankfully nothing I post is worth reading!

A nice reminder in "whoopsie" recovery though.
Title: Re: NEWS: Forum reverted to backup
Post by: Ed.Kloonk on May 03, 2021, 06:22:59 am
Thanks for all you do, Dave.

Thanks gnif.

 :-+
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 06:31:28 am
Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/ (https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/)
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/ (https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/)
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/ (https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/)

Who's posts are missing?
Title: Re: NEWS: Forum reverted to backup
Post by: Muttley Snickers on May 03, 2021, 06:33:53 am
I'm still down around 100 posts but not that concerned about it. I do wonder though, what would happen if a member had reached 1000 posts and gained access to the supporters lounge but are now back under 1000 due to the glitch, do they still have access?
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 06:46:12 am
I do wonder though, what would happen if a member had reached 1000 posts and gained access to the supporters lounge but are now back under 1000 due to the glitch, do they still have access?

I can fix that manually.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 06:49:57 am
I'm still down around 100 posts but not that concerned about it.

That's interesting, your account shows deleted sposts for threads like "your pet peeve", yet that thread is still intact. Strange...
Title: Re: NEWS: Forum reverted to backup
Post by: Ian.M on May 03, 2021, 06:54:41 am
My missing post (from browser history):
3561403 in https://www.eevblog.com/forum/beginners/12v-l-e-d-light-bar/ (https://www.eevblog.com/forum/beginners/12v-l-e-d-light-bar/)
Title: Re: NEWS: Forum reverted to backup
Post by: Muttley Snickers on May 03, 2021, 07:03:58 am
The snapshot below taken on 10/04/21 shows 2135 posts and I posted a couple more after that and hadn't deleted any. I have no clue what threads they were posted in and they may not be recent posts either, in fact they could have been from years back.
Title: Re: NEWS: Forum reverted to backup
Post by: wilfred on May 03, 2021, 07:15:14 am


IMHO when you post a message on a public forum, you are granting the forum the right to display the content you have created in perpetuity*.  Once you hit [Post] or [Send] its out of your control. 

*  unless the forum offers users a more restrictive content creators license, granting the users more rights.

Well there is this in the forum's rules sticky.

"Removal of Data:
You are free to delete or edit your own posts at any time."

It has been there as long as I can recall. It certainly is in accord with Dave's free approach in the early days. Perhaps he's a little battle weary these days but I wouldn't be encouraging him to change it. I cleaned up a bunch of my own rubbish which was of no enduring value (if it even ever had any). I doubt it negatively affected anyone.


Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 03, 2021, 07:37:30 am
Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/ (https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/)
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/ (https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/)
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/ (https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/)

Who's posts are missing?
Mine and other members.
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 03, 2021, 07:42:18 am
Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/ (https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/)
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/ (https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/)
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/ (https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/)

Who's posts are missing?
Mine and other members.
TurboTom typed a extensive and detailed reply yesterday and that's gone too:
https://www.eevblog.com/forum/testgear/rigol-ds1074z-s-function-generator-help/ (https://www.eevblog.com/forum/testgear/rigol-ds1074z-s-function-generator-help/)
Title: Re: NEWS: Forum reverted to backup
Post by: SeanB on May 03, 2021, 07:49:29 am
Everybody makes mistakes, just that Dave has been consistent in having a backup that is current. More importantly, that backup works! +1000% for Dave having that, and for Gnif spending his weekend hours and into the wee hours to fix things.

Dave, you do not pay Gnif enough.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 08:44:28 am
Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/ (https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/)
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/ (https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/)
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/ (https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/)

Who's posts are missing?

Mine and other members.

This is very strange.
When I look at the first 3 pages of your post history, there are none in the Deleted folder.
Mutley's profile on the other hand I see a few on the first page:

(https://i.imgur.com/UWOkT00.png)

Yet if I hit the Restore button, it doesn't seem to work, and a search for April 1st in that thread doesn't show the posts.

Yet the thread exists just fine:
https://www.eevblog.com/forum/chat/your-pet-peeve-technical-or-otherwise/ (https://www.eevblog.com/forum/chat/your-pet-peeve-technical-or-otherwise/)

So I have no idea how the post got deleted and why it's not restoring  :-//
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 08:46:22 am
Everybody makes mistakes, just that Dave has been consistent in having a backup that is current. More importantly, that backup works! +1000% for Dave having that, and for Gnif spending his weekend hours and into the wee hours to fix things.
Dave, you do not pay Gnif enough.

With hindsight maybe we should have gone back several days in the backup and just ate a couple of thousand new posts maybe (the forum gets about 800 new posts a day IIRC), it seems that something was screwed up.
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 03, 2021, 08:53:20 am
Posts MIA from these threads:
https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/ (https://www.eevblog.com/forum/repair/wavetek-2025a-0-2-2-200mghz-rf-sig-gen-repair/)
https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/ (https://www.eevblog.com/forum/testgear/sourcing-the-fork-terminals-used-on-oscilloscope-probe-ground-leads/)
https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/ (https://www.eevblog.com/forum/testgear/design-a-new-precision-lcr-tweezers/)

Who's posts are missing?

Mine and other members.

This is very strange.
When I look at the first 3 pages of your post history, there are none in the Deleted folder.
Not even the new topic I started which also is MIA ?   

TBH I'm very reluctant to make any further detailed posts to help members until the forum proves its trustworthiness.  :-BROKE
Title: Re: NEWS: Forum reverted to backup
Post by: Muttley Snickers on May 03, 2021, 08:53:51 am
Sorry Dave, I did delete that pet peeve post a day after posting it as I felt the language might have been out of order in retrospect.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 09:00:44 am
Sorry Dave, I did delete that pet peeve post a day after posting it as I felt the language might have been out of order in retrospect.

Ah right, still doesn't explain why the restore function doesn't work.
I've used it before and it's always worked.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 09:03:36 am
TBH I'm very reluctant to make any further detailed posts to help members until the forum proves its trustworthiness.  :-BROKE

There is no evidence at all that there is any issues with future posts.
With regards to your missing thread, we did go to a backup, so anyone who posted anything after that backup point will have lost their post, that's just the way daily backups work.
Your post was almost certainly one of those.
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 03, 2021, 09:06:58 am
Everybody makes mistakes, just that Dave has been consistent in having a backup that is current. More importantly, that backup works! +1000% for Dave having that, and for Gnif spending his weekend hours and into the wee hours to fix things.
Dave, you do not pay Gnif enough.

With hindsight maybe we should have gone back several days in the backup and just ate a couple of thousand new posts maybe (the forum gets about 800 new posts a day IIRC), it seems that something was screwed up.
And piss members off even more that have posted in great detail over that time ?
Whatever time your little faux pas has cost you members collectively have lost more.

Please remember this any time admin work is carried out.
Title: Re: NEWS: Forum reverted to backup
Post by: lowimpedance on May 03, 2021, 09:30:52 am
 Still down 30 posts, with no idea what threads they would have been from   :-// . Since I don't post often I'm not certain how far back this "cleansing" has gone. The show posts seems to show all and a quick look back to 2018 at those threads appear to be intact. Oh well at least I'm still above  1000  !, Which only took a decade to get  :P.
 At least total FUBAR has been mostly averted. :-+.
Title: Re: NEWS: Forum reverted to backup
Post by: Ed.Kloonk on May 03, 2021, 09:32:18 am
Everybody makes mistakes, just that Dave has been consistent in having a backup that is current. More importantly, that backup works! +1000% for Dave having that, and for Gnif spending his weekend hours and into the wee hours to fix things.
Dave, you do not pay Gnif enough.

With hindsight maybe we should have gone back several days in the backup and just ate a couple of thousand new posts maybe (the forum gets about 800 new posts a day IIRC), it seems that something was screwed up.
And piss members off even more that have posted in great detail over that time ?
Whatever time your little faux pas has cost you members collectively have lost more.

Please remember this any time admin work is carried out.

I don't think that is being very fair at all.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 10:02:01 am
Everybody makes mistakes, just that Dave has been consistent in having a backup that is current. More importantly, that backup works! +1000% for Dave having that, and for Gnif spending his weekend hours and into the wee hours to fix things.
Dave, you do not pay Gnif enough.

With hindsight maybe we should have gone back several days in the backup and just ate a couple of thousand new posts maybe (the forum gets about 800 new posts a day IIRC), it seems that something was screwed up.
And piss members off even more that have posted in great detail over that time ?
Whatever time your little faux pas has cost you members collectively have lost more.
Please remember this any time admin work is carried out.

Seriously?
It looks like this was a bug in the forum software, it wasn't me being reckless pushing the wrong button, it was an unforeseen issue. Shit happens and we did the best we could to revover from it.
It's been a rather streesful few days and I don't appreciate this response.
Title: Re: NEWS: Forum reverted to backup
Post by: SeanB on May 03, 2021, 10:14:47 am
Seriously?
It looks like this was a bug in the forum software, it wasn't me being reckless pushing the wrong button, it was an unforeseen issue. Shit happens and we did the best we could to revover from it.
It's been a rather streesful few days and I don't appreciate this response.

Dave, you cannot please all. Just smile, think of them as if they are again off the lithium, and carry on as usual. I would post him a pouch cell to take in place of it, but SAPO will not cooperate with sending dangerous goods in the post.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 10:15:56 am
My missing post (from browser history):
3561403 in https://www.eevblog.com/forum/beginners/12v-l-e-d-light-bar/ (https://www.eevblog.com/forum/beginners/12v-l-e-d-light-bar/)

Was that post in the last day or so?
I see no history of any deleted post, so I presume it was recent and lost in the reversion to backup.

For those unaware of how this work, the forum server runs on dual redundant boxes. The loss of either box hardware will not impact the forum.
The loss of both boxes as happened in the recent data centre fire results in the forum going down, but at least people can't post and add more stuff that might get lost.
Backups are peformed daily and stored in multiple offsite facilities.
If anything technical happens to the forum in terms of database problems, corruption etc, then the forum may actually continue to operate and we only find out at a later point that something is wrong, during which time people continue to post etc. If the issue can't be fixed, or fixed easily then we may have to resort to reinstalling the backup. That means we could lose anywhere from a few hours to 24 hours worth of posts, or more if we don't notice the issue in a timely fashion. In this case it's not possibel to not lose posts, it's just a matter of how many.
We had the unfortunate latter situation this time where I did not notice anything until i saw the "has anyone lost 5% of their posts?" thread, by which time many thousands of posts were likely to have been made, and it took time to anaylse what was happening and lock down the forum.
Title: Re: NEWS: Forum reverted to backup
Post by: tggzzz on May 03, 2021, 10:29:07 am
Whatever time your little faux pas has cost you members collectively have lost more.

Please remember this any time admin work is carried out.

I suspect Dave, gnif et al are more than aware of that!

I don't see your comment being either amusing, enlightening, entertaining, or helpful.
Title: Re: NEWS: Forum reverted to backup
Post by: Muttley Snickers on May 03, 2021, 10:31:57 am
We are all very grateful for the forum and acknowledge the effort involved in keeping it running smoothly but it's still a bit confusing as to what happened to many peoples posts. The 5% thread you made reference to which no longer exists included a few members who also experienced multiple missing posts or a questionable post count. At the time tautech had lost over five hundred posts in the matter of a day or so, I believe this is why he was so concerned.
Title: Re: NEWS: Forum reverted to backup
Post by: beanflying on May 03, 2021, 10:35:27 am
The opportunity to break an SMF or for that matter most forums in the backroom when you are the Admin is absolute and fairly easy to do as you have ALL permissions enabled. When I was developing my last SMF based project I was running a dummy forum with all the same code and mods fitted and anything potentially harmful got tested before getting near the main forum.

Bulk removing 10k+posts was always a dangerous thing and if that now I gather ex member wanted it done then it should have fallen on them to filter the content they wanted removed or it should have been left in place so the threads in time still read 'sensibly'.
Title: Re: NEWS: Forum reverted to backup
Post by: Ian.M on May 03, 2021, 10:44:44 am
My missing post (from browser history):
3561403 in https://www.eevblog.com/forum/beginners/12v-l-e-d-light-bar/ (https://www.eevblog.com/forum/beginners/12v-l-e-d-light-bar/)

Was that post in the last day or so?
I see no history of any deleted post, so I presume it was recent and lost in the reversion to backup.
It would have been within 12 hours of the last visible post in that topic.  Reply #2 on: April 29, 2021, 06:47:02 (https://www.eevblog.com/forum/beginners/12v-l-e-d-light-bar/msg3559460/#msg3559460)
IIRC there was a further reply from the O.P. as well.  I believe the O.P. got what he needed from the discussion, so please don't prioritize rescuing that thread manually.

It would seem that *either* you don't have any easy/automated way to extract new/edited posts made since the most recent backup so they can be reinstated after the restore, *or* the database was too badly corrupted for that to be practical.   

It would have been nice if the extremely terse SMF error message while the forum was fully down linked to a status update page similar to the one you put up during the full site outage.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 10:52:11 am
We are all very grateful for the forum and acknowledge the effort involved in keeping it running smoothly but it's still a bit confusing as to what happened to many peoples posts. The 5% thread you made reference to which no longer exists included a few members who also experienced multiple missing posts or a questionable post count.

That thread was posted atfer the last backup snapshot so of course it would be gone, no mystery there.
As to some other posts, I'm still trying to figure that out, but at the time we had to make a call to:
a) Use the backup or try to repair, we ultimately decided on the backup after some investigation.
and then
b) Which backup to use. We weren't sure of timing when the issues too place so we picked the last backup to minimise the number of lost posts. But as it now turns out that backup database may have still had some risidual issues we are only just now finding out about. At the time the backup did indeed look good from what we could see, so we ran with it.

Unfortunately we might now be in a position where there are some residual post issues leftover, and the only solution to that would be the go to an even older backup and losing even more posts. At some point we have to decide to just take the loss and move forward.
Title: Re: NEWS: Forum reverted to backup
Post by: tggzzz on May 03, 2021, 12:48:50 pm
There's an argument that if a poster can't tell you their post is missing, then the post was of little consequence.

That's certainly valid for my posts!
Title: Re: NEWS: Forum reverted to backup
Post by: beanflying on May 03, 2021, 12:52:43 pm
There's an argument that if a poster can't tell you their post is missing, then the post was of little consequence.

That's certainly valid for my posts!

In this case if all of Blueskulls 300+ topics for a start have been or are still removed then having other members lose several hundred posts is likely. Forums are about 'content' and the sum of that content is the reason for their ultimate existence. This SUM is of consequence!
Title: Re: NEWS: Forum reverted to backup
Post by: wilfred on May 03, 2021, 12:59:43 pm
Bulk removing 10k+posts was always a dangerous thing and if that now I gather ex member wanted it done then it should have fallen on them to filter the content they wanted removed or it should have been left in place so the threads in time still read 'sensibly'.
I can't find it now, but I thought I read a post where Dave explained that a member requested Dave's help to remove some posts. What is wrong with that? It is not like they did anything malicious. They did nothing wrong.

I don't know why they wanted it, I don't care. If they ask and Dave is OK with it then as far as I am concerned they have done all they needed to do.

Dave sure as hell isn't trying to shift the blame onto them. Not from anything I've seen. As far as I can see Dave has been completely upfront in accepting responsibility.

I can't help but wonder if it is not possible to journal posts between backups so that no matter how far you must revert the database to get a good  copy for restore, you will be able to roll forward from the journal entries timestamped after the backup. If that was available this would be a much simpler problem to unravel.

I would like to see the changes made to the live forum verified on a clone copy. Whether it is database maintenance, or software upgrades. Then with the confidence of a verified procedure apply them to the alternate server and switch the databases with subsequent updates made to the live server applied from the journals. I'm sure database admins will express it better. Perhaps on these systems it isn't possible. IDK.

Another thing I saw was a comment Dave made about the time lag between a database corruption and the error being reported or noticed. This would cause posts made in the interval vulnerable to loss when the latest backup was applied. Aside from speculating on whether posts can be journaled, can a script be added to the backups to test (and fix) database errors? or at least notify an admin hopefully before users notice in any numbers.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 03, 2021, 01:14:38 pm
I would like to see the changes made to the live forum verified on a clone copy. Whether it is database maintenance, or software upgrades. Then with the confidence of a verified procedure apply them to the alternate server and switch the databases with subsequent updates made to the live server applied from the journals. I'm sure database admins will express it better. Perhaps on these systems it isn't possible. IDK.

Unfortunately in certain circumstances like we've seen her, the only way to "verify" that something worked ok is to wait and crowd source complaints.

Quote
Another thing I saw was a comment Dave made about the time lag between a database corruption and the error being reported or noticed. This would cause posts made in the interval vulnerable to loss when the latest backup was applied. Aside from speculating on whether posts can be journaled, can a script be added to the backups to test (and fix) database errors? or at least notify an admin hopefully before users notice in any numbers.

There are just so many variations on what can happen with a database of this size and magntiude, it's likely impossible to cover everything.#Like like I have imagined that deleting indivisual user posts under a specifc circumstance have caused entire threads to be deleted? Not in a million years, or the 15 years I've been using SMF anyway.
Title: Re: NEWS: Forum reverted to backup
Post by: Black Phoenix on May 03, 2021, 01:42:30 pm
Deleting something from a database in a working live platform is always a problematic endeavour as, if a mistake is made a lot more is lost and tracking back all that happened to recover what was removed by mistake is a PITA and very time consuming.

I've done similar stuff in telecommunications were I just YOLO a config into a NSN MGW (Media Gateway, kinda like a router but for telecommunications that is the size of 3 40U racks) and put half of Portugal out of communications. 10 min for 5 hours of recovery. It was my fault because I should had done a snapshot of the equipment (less than 1 hour) but since it was a small group of commands I thought it was safe. Low and behold, one of the delete commands spectrum was higher that was suppose to be and KAPUT everything gone. Had to recover from the day before snapshot and then get the rest of the changes made that day on the machine via Command Log History.

What Dave done was risky, but he had done the right decisions in terms of recovery he made. Yes, a day of posting was lost. Probably as soon as someone detected the missing posts Dave should had blocked the forum to read only and then made a snapshot of all new posts from the last backup until the moment. Then restore the backup and merge the changes present in the snapshot (don't know how hard it is or even if it is possible with the database and forum software being run).

It wasn't done. Posts were lost, discussions and lines of thought were lost since the last backup. But still Dave, Gnif and the rest of the team done the right choices for the problem in hand. I can't fault them for that. And I will not point the fingers on them because I perfectly know what is to screw up and clean the pile of sh#t you made.

What it makes me perplexing is why causing all this because of removing a user from existence on this forum, per request of said user. Yes he have the right to have his posts deleted. But did he?

I don't know the rules in Australia and if they have something like the Right To Be Forgotten (https://en.wikipedia.org/wiki/Right_to_be_forgotten) or GDPR (https://en.wikipedia.org/wiki/General_Data_Protection_Regulation). If it is the rules of the country from the community owner, then I don't know the regulations in Australia. If it is the rules of the citizen's country then that's a different beast - In China that right doesn't exist: https://en.wikipedia.org/wiki/Right_to_be_forgotten#China

But the main problem now is that we have a forum now that lost potentially lots of knowledge from that user in posts created by him and lots of quotes from his posts in other threads with line of tough lost in topics that the users didn't quote him for his comment.  Yes he was very extremist in his views regarding his country, call him whatever you want. But from his total amount of posts - 14803 at the time he was banned - I'm sure that only 10% of them was really problematic (and from that 10% a lot were dealt with at the time they were reported for sure). OK let's say 20 to 25% were problematic. It is still 75% of posts with info and knowledge that left the forum.

Again I will say, I'm not against how the forum team reacted in all this situation. On the opposite, if we weight the pros and cons of what was done, it was the right choice for the problem in question. What it saddens me is the info that was lost. Just that.

But again I reiterate - Comparing the problem (the mass deleted posts by mistake), the possible solutions and consequences of the decisions, the forum team done exactly the right choices for the problem in hand with the minimal loss.

It is easy being on this side and judge. But the other side is the one who knows what was the size of the turd that hit the fan.
Title: Re: NEWS: Forum reverted to backup
Post by: magic on May 03, 2021, 06:51:53 pm
So all of that for blueskull? :scared:
Surprising that Dave gave in, but whatever.

One thing that could have been done differently (if it hasn't) was to perhaps make a backup of the state before reversal. On the off chance that something of value had been posted worth recovering manually from the DB.
Title: Re: NEWS: Forum reverted to backup
Post by: thm_w on May 03, 2021, 09:13:59 pm
But the main problem now is that we have a forum now that lost potentially lots of knowledge from that user in posts created by him and lots of quotes from his posts in other threads with line of tough lost in topics that the users didn't quote him for his comment.  Yes he was very extremist in his views regarding his country, call him whatever you want. But from his total amount of posts - 14803 at the time he was banned - I'm sure that only 10% of them was really problematic (and from that 10% a lot were dealt with at the time they were reported for sure). OK let's say 20 to 25% were problematic. It is still 75% of posts with info and knowledge that left the forum.

Yes this is sad.

Bulk removing 10k+posts was always a dangerous thing and if that now I gather ex member wanted it done then it should have fallen on them to filter the content they wanted removed or it should have been left in place so the threads in time still read 'sensibly'.

Would have been nice if any post mentioning location/personal info etc were deleted and that was it. But it seems (https://www.eevblog.com/forum/microcontrollers/gigadevice-risc-v-microcontroller/msg3561686/#msg3561686) like more of a "ok im angry, I'm taking all my stuff back" deal. So to get back at the few idiots provoking him, a wealth of knowledge is lost for thousands of other readers. Poor form on both sides.

How long until we can train an AI to automatically screen posts for political opinions and throw up an error message.
Title: Re: NEWS: Forum reverted to backup
Post by: james_s on May 03, 2021, 09:58:31 pm
Well that explains what happened at least. There were several threads I could have sworn I had replied to already but my reply wasn't there. I couldn't think of any reason a mod would have deleted what I replied with deliberately.
Title: Re: NEWS: Forum reverted to backup
Post by: rsjsouza on May 03, 2021, 10:00:00 pm
So all of that for blueskull? :scared:
Surprising that Dave gave in, but whatever.
I had no idea that was the reason. Not knowing anything else, this is not a great move (deleting ALL posts). Way too many interactions that will crap the flow of conversations.

OTOH, his technical contributions were good but his rhetoric was extremely toxic. He was given many chances to cool off and, for a 28 year-old guy, I think he should have grown up by then.
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 03, 2021, 10:35:03 pm
So all of that for blueskull? :scared:
Surprising that Dave gave in, but whatever.
I had no idea that was the reason. Not knowing anything else, this is not a great move (deleting ALL posts). Way too many interactions that will crap the flow of conversations.

OTOH, his technical contributions were good but his rhetoric was extremely toxic.
Only as toxic as all the China hate on this forum and worldwide.

Blueskull and his parents sacrificed and worked hard for him to gain a doctorate in Power electronics and despite my wishes for him to display such qualifications in his profile he wouldn't flaunt what he had achieved yet the quality of his contributions here over the years grew with his experience.
I for one will miss him here although we do communicate privately so at least I still have that.

RIP Blueskull on EEVblog.  :(
Title: Re: NEWS: Forum reverted to backup
Post by: JohnnyMalaria on May 03, 2021, 10:40:56 pm
He sure got thanked a lot...

Title: Re: NEWS: Forum reverted to backup
Post by: gnavigator1007 on May 03, 2021, 10:50:18 pm
I'm mostly a lurker, but I appreciated a great many of his posts. It's a substantial loss for the forum. I was just looking at some of the forum stats and in addition to being one of the most thanked members, he was also one of the top topic starters. Interestingly, 367 topic posts are still viewable via his profile as well as attachments. 

Just noticed looking at the stats again that total page views have passed 1B! Congrats Dave
Title: Re: NEWS: Forum reverted to backup
Post by: james_s on May 03, 2021, 11:12:51 pm
OTOH, his technical contributions were good but his rhetoric was extremely toxic. He was given many chances to cool off and, for a 28 year-old guy, I think he should have grown up by then.


Frankly I had hardly noticed. Occasionally he would go off on some rant and I would just scroll past it, didn't really seem like THAT big of a deal. There are a few forum members I'm not going to name who have such toxic personalities that I put them on my ignore list a long time ago as they almost never say anything that is not rude and unhelpful but he is not one of them. I would take blueskull anti-western rants all day long over the toxic identity politics and other nonsense that pollute every other corner of the web.
Title: Re: NEWS: Forum reverted to backup
Post by: lowimpedance on May 04, 2021, 12:12:55 am
So all of that for blueskull? :scared:
Surprising that Dave gave in, but whatever.
I had no idea that was the reason. Not knowing anything else, this is not a great move (deleting ALL posts). Way too many interactions that will crap the flow of conversations.

OTOH, his technical contributions were good but his rhetoric was extremely toxic. He was given many chances to cool off and, for a 28 year-old guy, I think he should have grown up by then.
His strident views certainly shifted after he left the US, however removing all your posts is immature and he's not the first !. It is more sad that the knowledge passed on has been erased and even more so that this exercise has caused collateral damage that cannot be repaired.
Title: Re: NEWS: Forum reverted to backup
Post by: floobydust on May 04, 2021, 12:23:41 am
I found his fanaticism refreshing and an insight into that which is at war with democracy.
There are several papers written by Party defectors that give further insight. It's not looking good for us, nor is it all unicorns and rainbows in the Party as his future employer.

I have an issue with allowing an extremist who threatened to report forum members to the CC P to be here. The usual nutbar behaviour gets banned, but not this?
Where's the Forum's rules?
Title: Re: NEWS: Forum reverted to backup
Post by: bitseeker on May 04, 2021, 12:49:30 am
I'm pretty sure I have now undeleted almost every thread that was accidently deleted.
I may have missed a couple, and if you are aware of any please let me know.
Everyone's post counts should be back to almost normal, although there will almost certainly be a few casulty posts, sorry about that.
The recovery from my goof was not 100%, but should be pretty darn close.
Thanks to gnif for a late night backup recovery and writing some script to fix some things.
Sorry to everyone for the goof up, it was entirely my fault  |O

Thanks, Dave and gnif for getting everything mostly back to normal. Post count? I don't know exactly what my post count was. Close enough. ;D
Title: Re: NEWS: Forum reverted to backup
Post by: mnementh on May 04, 2021, 01:26:51 am
Wait... what...? This all happened cuz some dingle decided to ragequit; take his ball and go home...?  :palm:

(https://media1.tenor.com/images/8b1a9f1bb3962b8a8eccc08c86d1fe2a/tenor.gif)

I just figured gnif finally had enuf of us all whining aboot the slow pageloads and went a little Widlar on eevBlog's arse...  :-DD

mnem
oooh... smoky...
Title: Re: NEWS: Forum reverted to backup
Post by: rsjsouza on May 04, 2021, 01:30:34 am
OTOH, his technical contributions were good but his rhetoric was extremely toxic.
Only as toxic as all the China hate on this forum and worldwide.
As I have said in one of many occasions he was the subject of a banning/pardon decision: one can only control oneself's actions and he chose to respond to the provocations. Other members from the same country do a much better job at keeping their emotions at bay.

Blueskull and his parents sacrificed and worked hard...
Yes, just like many others here in this forum and worldwide, but that does not invalidate or justify his actions.

Reiterating: I appreciate a good discussion and the occasional insight about the Chinese culture that he provided, but it was way too often he was involved in imbroglios regarding nationality, political and economical POVs. Other members were banned by much less.

I feel sadened by two things: no more access to his valuable insights and the deletion of the majority of his contributions. May he have success in his future enterprises.

Anyways, my opinion is voiced. I now choose to ignore this thread. Bye y'all.
Title: Re: NEWS: Forum reverted to backup
Post by: Black Phoenix on May 04, 2021, 01:42:01 am
OTOH, his technical contributions were good but his rhetoric was extremely toxic.
Only as toxic as all the China hate on this forum and worldwide.
As I have said in one of many occasions he was the subject of a banning/pardon decision: one can only control oneself's actions and he chose to respond to the provocations. Other members from the same country do a much better job at keeping their emotions at bay.

Blueskull and his parents sacrificed and worked hard...
Yes, just like many others here in this forum and worldwide, but that does not invalidate or justify his actions.

Reiterating: I appreciate a good discussion and the occasional insight about the Chinese culture that he provided, but it was way too often he was involved in imbroglios regarding nationality, political and economical POVs. Other members were banned by much less.

I feel sadened by two things: no more access to his valuable insights and the deletion of the majority of his contributions. May he have success in his future enterprises.

Anyways, my opinion is voiced. I now choose to ignore this thread. Bye y'all.

Well let's not idolise the guy and godify him Ok? Rsjsouza is totally right. And as I said his contributions were good and is sad that they were lost because he wanted them removed.

But the toxic behaviour deserved what it arrived at his bay. Specially the last post that got him banned who make accusations to Dave and other members of this forum/companies. There is a say in the web that "Don't feed the troll". He had done that, lots of times.

If people talk bad about my country and whatever do you really think I will care about it? Why me justifying my country choices will change the idea of the person in question or even make any meaningful change in the overall big scope of things? It is a drop in the ocean. It doesn't change a thing, it only makes you feel better, nothing more.

And as a foreigner living in Shenzhen/Hong Kong and having a Chinese family from the side of my wife, both in Mainland and Hong Kong, I have a good view of both sides and different opinions.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 04, 2021, 02:22:26 am
And as I said his contributions were good and is sad that they were lost because he wanted them removed.

Correct. He wanted every single post removed and would have done it himself if he was given access back.
Everyone has the ability to delete all their posts on this forum, it's just that admins have tools to do it on a mass scale with greater ease.
Title: Re: NEWS: Forum reverted to backup
Post by: Ed.Kloonk on May 04, 2021, 06:40:36 am
it's just that admins have tools to do it on a mass scale with greater ease.

!
Title: Re: NEWS: Forum reverted to backup
Post by: Black Phoenix on May 04, 2021, 07:19:29 am
Wait... what...? This all happened cuz some dingle decided to ragequit; take his ball and go home...?  :palm:

(https://media1.tenor.com/images/8b1a9f1bb3962b8a8eccc08c86d1fe2a/tenor.gif)

I just figured gnif finally had enuf of us all whining aboot the slow pageloads and went a little Widlar on eevBlog's arse...  :-DD

mnem
oooh... smoky...
Lovely mnementh. I was just thinking why did you take so much time to grace us with your well chosen gifs! 5 star, specially for the ones who understand what the command in question do.

To be sincere I've done that plank to a new colleague with the exact same command (but before I made full disk backup of the system on a extra HDD for easy swap and low downtime plus a digital copy on the PC just in case) to show him the main rule from Sysadmins: Don't run commands without fully knowing what kind of implication such commands cause!
Title: Re: NEWS: Forum reverted to backup
Post by: magic on May 04, 2021, 08:18:13 am
It's for weeding out Ubuntu users >:D
Code: [Select]
$ sudo rm -rf /*
-bash: sudo: command not found

To be sincere I've done that plank to a new colleague
Not the smartest idea. There was time when it permanently bricked PCs thanks to the genius of UEFI and awesomeness of Linux. Since then I'm somewhat weary of running such things on /sys.

If it wasn't for the technical stuff up/bug, it would have been trivial work and no one would have noticed.
Haha, knowing now what really happened I have some doubts :D
Title: Re: NEWS: Forum reverted to backup
Post by: mnementh on May 04, 2021, 04:25:43 pm
Oh, I'll be the first to admit my Kung-Fu is weak; I know just enough to be really dangerous. Versions of that meme were popular aboot the second time I went back to school, and I figured that while it bordered on bad taste, we could maybe use a little frivolous diversion. ;)

Yes, I do know exactly what that command is supposed to do. I assume hope that in the many years since I was in school and had to do that kind of stuff daily, enough people got bitten by it that some err... protections have been put in place, at least in the more common distros...?

mnem
Being the class clown is hard work; but I have a lot of Karma to burn off...   >:D
Title: Re: NEWS: Forum reverted to backup
Post by: bd139 on May 04, 2021, 04:40:27 pm
Gah so much undue vitriol here.

Thanks to Dave and Gnif for managing to get through it. I think we all owe you both some space and a drink for actually giving enough of a crap about the community to respect a former user's request to have their posts removed  :-+
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 04, 2021, 04:55:32 pm
Ah, here it is!  :)

I knew there should be a thread about the outage somewhere, so this discussion should probably be in here, instead of ataradov's e-mail notification thread:

Something must still be amiss somewhere...

My post count says 1767 but if I view "Show Posts" in my profile, I can go back to page 73 and it shows post #1824...

It is 7455 vs 7406 for me.

Well, they're obviously still doing something behind the scenes on this...

My post count jumped from 1768 yesterday to 1783 today all by itself, and there are now 1841 in my "Show Posts" section in the profile.

They must still be un-munging the database and presumably will have to do some magic SQL incantation to re-populate people's post count field with the actual true value once the repairs are complete...

I'm pretty sure I was right around 1900 posts, as I was closing in on a 1 post-per-day average on the profile page, but I'm currently shown to be back at 0.864 per day.
Title: Re: NEWS: Forum reverted to backup
Post by: Muttley Snickers on May 05, 2021, 08:19:41 am
It appears that the post counter shown below a members avatar is incorrect, the number of posts shown on the last page of a members profile displays the true post count as mentioned above.

Title: Re: NEWS: Forum reverted to backup
Post by: tggzzz on May 05, 2021, 08:47:39 am
And as I said his contributions were good and is sad that they were lost because he wanted them removed.

Correct. He wanted every single post removed and would have done it himself if he was given access back.
Everyone has the ability to delete all their posts on this forum, it's just that admins have tools to do it on a mass scale with greater ease.

While blueskull may have made some useful technical contributions, some people are just more trouble than they are worth.

But overall I would have preferred to keep his contributions visible, so people could see his helpful posts and also to see what a painful immature twit he could be.

Overall I think you were too nice and obliging to him w.r.t. deleting his posts.

20:20 hindsight is a wonderful thing :)
Title: Re: NEWS: Forum reverted to backup
Post by: Ian.M on May 05, 2021, 09:38:39 am
OTOH, and this is pure speculation from an ordinary user and I specifically would *NOT* expect any admin or moderator to confirm or deny:

*IF* Dave agreed a deal with the ex-user in question that in exchange for eradicating his content he'd avoid any future interaction with this forum or the rest of Dave's online presence, it may have been well worth it (excluding the unforseen database corruption, which no doubt would eventually have take the forum down anyway). Its a simple choice to spend a few hours *now* and accept a limited amount of loss of content, to avoid a pissed-off and technically competent nutter person gunning for you, vs dealing with the sh!t they are likely to attempt to fling for the foreseeable future, if you don't give them a perceived 'victory' as they depart.

I do however believe the forum rules section:
Quote
Removal of Data:
You are free to delete or edit your own posts at any time. However, once someone has quoted your text (or it's archived by another website), you can't remove that, and only under exceptional circumstance will moderators remove it for you. This is at the discretion of the forum owner.
Basically, whilst you are allowed to withdraw your own posts, please be aware that it may have already been copied and duplicated elsewhere.
That is always the risk of posting on the Internet, and this forum is no different.
By posting on this public forum you are effectively giving up ownership of the text you wrote - it becomes public domain - people are free to quote it for perpetuity on this forum and other websites, bots are free to archive it and make it available in searches, and the forum owner is free to archive it and restore it if desired. You have no right to demand that what you wrote be removed from all parts of this forum and other people's posts. If you don't like this, don't post to a public forum like this.
could do with a tweak. changing "You are free to delete or edit your own posts at any time." to "Registered users in good standing are free to delete or edit their own posts at any time." which would bring it into line with the actual situation: that banned or suspended users cant delete/edit their own posts because they cannot log in to do so.

Edit: I retract any inferences about the ex-user's mental state.
Title: Re: NEWS: Forum reverted to backup
Post by: all_repair on May 05, 2021, 12:00:15 pm
Just stop bad mouthing a person when he has no way to reply or defend, or counter attack.   Do it when you can take his counter attack, toxic or not.  He has no way to delete his own posts, and Dave is doing what he has promised.  Banning a person and still want to keep his postings just doesn't sound right.
Title: Re: NEWS: Forum reverted to backup
Post by: Ian.M on May 05, 2021, 12:23:37 pm
He has no way to delete his own posts, and Dave is doing what he has promised.  Banning a person and still want to keep his postings just doesn't sound right.
That's what you, I and every other user agreed to.  See https://www.eevblog.com/forum/chat/forum-rules-please-read/ (https://www.eevblog.com/forum/chat/forum-rules-please-read/)

Dave *chose* to be nice and accommodate his request for bulk post deletion. 

By the terms of the user agreement, if a hypothetical disgruntled departing user utilized their posting history and the message modify or remove tools to go through and bulk delete their posts then abandoned their account, Dave *could* restore the deleted/modified messages and perma-ban them so they couldn't log back in and trash them again.
Title: Re: NEWS: Forum reverted to backup
Post by: beanflying on May 05, 2021, 12:33:17 pm
Not really the place for it but depending on the location of the entity and the location of the data the 'Author' retains a greater or lesser degree of control how their work is used or displayed. In reality no one is likely to push those buttons legally regardless of legal merit but it is a big world and laws vary.

Dave chose to follow the request covered by the forum rules in this case in spite of it being a net loss to the forums content as not only have Blueskulls topics and other posts gone but the other member replies to those 300+ topics have gone too. Opting to the taking his bat and ball and going home solution is still a fairly petty move and will only bring joy (small minded joy) to the former serial taunters of him and there were more than a few.
Title: Re: NEWS: Forum reverted to backup
Post by: magic on May 05, 2021, 03:24:57 pm
The topics and other people's replies are still up. Indeed, I have posted in one of them today.

*IF* Dave agreed a deal with the ex-user in question that in exchange for eradicating his content he'd avoid any future interaction with this forum or the rest of Dave's online presence, it may have been well worth it (excluding the unforseen database corruption, which no doubt would eventually have take the forum down anyway). Its a simple choice to spend a few hours *now* and accept a limited amount of loss of content, to avoid a pissed-off and technically competent nutter person gunning for you, vs dealing with the sh!t they are likely to attempt to fling for the foreseeable future, if you don't give them a perceived 'victory' as they depart.
He threatened to get his commie comrades to nuke all of Australia in retaliation. True story :D
Title: Re: NEWS: Forum reverted to backup
Post by: bsfeechannel on May 05, 2021, 04:14:15 pm
Yeah, I don't get it. There are still precisely 367 posts of his (https://www.eevblog.com/forum/profile/?u=101274;area=showposts;start=350). The post where I quoted his posts are still there. Dave said that blueskull would have deleted his own posts himself if he were allowed to return, but his profile is NOT banned (https://www.eevblog.com/forum/profile/?u=101274).

Blueskull had been at odds with the moderation for quite a few years, for him to have requested and Dave to have complied so promptly, there must have been a huge schism between the two. One can almost sense the fury with which Dave deleted his posts.

Was his threat of nuking Australia along with his commie homies the last straw?

Title: Re: NEWS: Forum reverted to backup
Post by: Bud on May 05, 2021, 04:29:35 pm
He studied in the US, after finishing the program he applied for a visa to stay. His  visa application was rejected, he returned to China and commited himself to a holy war against the West in revenge.
Title: Re: NEWS: Forum reverted to backup
Post by: bd139 on May 05, 2021, 05:35:59 pm
No. Realistically there’s immature dicks on both sides of the fence.

Everyone please just grow up and treat your fellow humans nicely. Then we won’t have to be here.
Title: Re: NEWS: Forum reverted to backup
Post by: Black Phoenix on May 05, 2021, 06:06:25 pm
He threatened to get his commie comrades to nuke all of Australia in retaliation. True story :D

Was his threat of nuking Australia along with his commie homies the last straw?

He studied in the US, after finishing the program he applied for a visa to stay. His  visa application was rejected, he returned to China and commited himself to a holy war against the West in revenge.

There wasn't any post relating with him saying that it was going to nuke Australia. What he done, and was said by Dave:

He had been baned before and has a history of years of provacations and reports from users, I was not going to allow this to continue.
Not to mention that one of his last posts was an admission that he directly targetted myself and the forum and reported me to forum advertisers in order to get their ads pulled.
Now please get this thread back on topic or it gets locked.

on https://www.eevblog.com/forum/microcontrollers/gigadevice-risc-v-microcontroller/msg3562369/#msg3562369 (https://www.eevblog.com/forum/microcontrollers/gigadevice-risc-v-microcontroller/msg3562369/#msg3562369)

And I provided Dave with a screenshot after reporting the message for his own reference that stated EXACTLY what Dave said in the comment.

I may have disliked how he simply gone into full angry mode every time someone talked bad about his country. I may not agree on Godify him as I see now happening, even if his past contributions on this forum were useful and he was one of the most thanked users, since a lot of rights don't make the wrongs disappear, but one thing I'm sure: Talking on the back without him being able to defend himself is not right, even if he was what he was and his views were what they were.

This is old news now, and bd139 just close the subject with his message:
No. Realistically there’s immature dicks on both sides of the fence.

Everyone please just grow up and treat your fellow humans nicely. Then we won’t have to be here.

And I must be also one of them since I feel that the hat fits me.
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 05, 2021, 06:45:01 pm
Something must still be amiss somewhere...

My post count says 1767 but if I view "Show Posts" in my profile, I can go back to page 73 and it shows post #1824...

It is 7455 vs 7406 for me.

Well, they're obviously still doing something behind the scenes on this...

My post count jumped from 1768 yesterday to 1783 today all by itself, and there are now 1841 in my "Show Posts" section in the profile.

They must still be un-munging the database and presumably will have to do some magic SQL incantation to re-populate people's post count field with the actual true value once the repairs are complete...

I'm pretty sure I was right around 1900 posts, as I was closing in on a 1 post-per-day average on the profile page, but I'm currently shown to be back at 0.864 per day.

It appears that the post counter shown below a members avatar is incorrect, the number of posts shown on the last page of a members profile displays the true post count as mentioned above.

Okay, guys...  back to the topic of forum database repair...  :)

Dave, for the incorrect post counts, if doing the various "check for errors" and then "recount forum statistics and stats" options in SMF's "forum maintenance" section of the admin panel still doesn't recount people's posts properly, you can try this mod:

https://custom.simplemachines.org/mods/index.php?mod=2324
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 05, 2021, 06:52:03 pm
(https://i.imgur.com/XOzldOf.png)
Title: Re: NEWS: Forum reverted to backup
Post by: magic on May 05, 2021, 08:11:01 pm
Was his threat of nuking Australia along with his commie homies the last straw?
No, it was a joke regarding all the crazy speculation here :P

That being said, maybe it wasn't so crazy if he really wrote letters to advertisers :scared:

But I'm afraid he will discover that all his whining will simply not have the same pull as similar actions by the usual suspects like LG@#$%RTY or B*M. And I think here lies the root of his anger: the absolute idiocy, doublefacedness and hypocrisy of all the Western SJW/liberal/mumbo/jumbo/egalitarian/democratic nonsense rhetoric.

The idea that blueskull is going through all that effort just because he had been denied a stupid visa which he could simply reapply for next year is IMHO laughable.
Title: Re: NEWS: Forum reverted to backup
Post by: magic on May 05, 2021, 08:16:34 pm
Yeah, I don't get it. There are still precisely 367 posts of his (https://www.eevblog.com/forum/profile/?u=101274;area=showposts;start=350). The post where I quoted his posts are still there. Dave said that blueskull would have deleted his own posts himself if he were allowed to return, but his profile is NOT banned (https://www.eevblog.com/forum/profile/?u=101274).
The 367 posts are his OPs. Removing those would entail nuking whole threads with all the content posted by others.

Surgically editing out quotes from your posts wouldn't be a trivial effort either, so no surprise.

It appears that the post counter shown below a members avatar is incorrect, the number of posts shown on the last page of a members profile displays the true post count as mentioned above.
Good catch, same for me. The number I see on my posts list is in the ballpark of what I remember from before the disaster, but the one in the profile is different.
Title: Re: NEWS: Forum reverted to backup
Post by: thm_w on May 05, 2021, 08:32:06 pm
No, it was a joke regarding all the crazy speculation here :P

*removed political opinions*

and you somehow feel the need to bring politics into this thread when that was the whole cause of the problem to begin with?
now this relatively civil discussion will be locked  :palm:

edit: the person you are quoting has had a number of racist posts deleted, so not the most reliable source.
Title: Re: NEWS: Forum reverted to backup
Post by: bsfeechannel on May 05, 2021, 09:09:21 pm
No, it was a joke regarding all the crazy speculation here :P

That being said, maybe it wasn't so crazy if he really wrote letters to advertisers :scared:

Yeah, I knew it had to be something really serious. Welp, he's got a lot to learn. But he's young, let's just give him time. I'll miss him.
Title: Re: NEWS: Forum reverted to backup
Post by: Ed.Kloonk on May 05, 2021, 09:45:08 pm
He'll be back.

Every good play needs a villain.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 12:57:39 am
(https://i.imgur.com/XOzldOf.png)

Ok, I'll run that now.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 01:00:29 am
(https://i.imgur.com/XOzldOf.png)

Ok, I'll run that now.

There is no such option:

(https://i.imgur.com/Nl1Gqqb.png)
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 06, 2021, 01:17:59 am
Ok, I'll run that now.
There is no such option:

Sorry, I should have been more clear...  You need to install this mod to add that option:

https://custom.simplemachines.org/mods/index.php?mod=2324

(https://i.imgur.com/XOzldOf.png)

P.S.  Also, we should chat sometime offline or PM or something re: extensions for things like the more modern post editors, etc. like being able to paste an image in using a simple [CONTROL]-[V], etc.
Title: Re: NEWS: Forum reverted to backup
Post by: Monkeh on May 06, 2021, 01:24:59 am
Ok, I'll run that now.
There is no such option:

Sorry, I should have been more clear...  You need to install this mod to add that option:

https://custom.simplemachines.org/mods/index.php?mod=2324

This is how we get into these situations..
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 01:25:21 am
Ok, I'll run that now.
There is no such option:

Sorry, I should have been more clear...  You need to install this mod to add that option:

https://custom.simplemachines.org/mods/index.php?mod=2324

Hmm, yet another mod, is it that important that everyones post count be exactly correct?  :-//
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 01:27:14 am
P.S.  Also, we should chat sometime offline or PM or something re: extensions for things like the more modern post editors, etc. like being able to paste an image in using a simple [CONTROL]-[V], etc.

I already installed an updated posting mod, and that one eventually screwed up images attachments in the database.
But if you know of a mod that allows simple CTRL-V please provide the link.
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 06, 2021, 01:36:09 am
The post re-counter mod is simple...  If you don't want to go messing with the SQL queries and statements by yourself to by hand to update the counts, it makes it easy.  It's only a like 2k long file, if you're good with DBs, could also just look at the source and do it manually I suppose, but that seems like a lot of effort for essentially one simple thing.   :o

I've never seen it mess anything up, and you can just remove it as soon as you're done the re-count since you're probably not going to be needing it again.  It doesn't have to stay on the system, you just need to run it once to fix the munged post total field...
Title: Re: NEWS: Forum reverted to backup
Post by: Muttley Snickers on May 06, 2021, 01:37:40 am

Hmm, yet another mod, is it that important that everyone's post count be exactly correct?  :-//

It's a technical engineering forum, if the numbers don't add up then nothing else counts for much either.   ::)
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 06, 2021, 01:39:28 am
I already installed an updated posting mod, and that one eventually screwed up images attachments in the database.

What one did you try, and when?

Your DB's whole schema pedigree was obviously very old originally vs. something now and been updated many times with SMF upgrades so you need to be cognizant of gotchas and try to be careful what you do willy-nilly...
Title: Re: NEWS: Forum reverted to backup
Post by: beanflying on May 06, 2021, 01:42:50 am
Ok, I'll run that now.
There is no such option:

Sorry, I should have been more clear...  You need to install this mod to add that option:

https://custom.simplemachines.org/mods/index.php?mod=2324


P.S.  Also, we should chat sometime offline or PM or something re: extensions for things like the more modern post editors, etc. like being able to paste an image in using a simple [CONTROL]-[V], etc.

Given there is still a fairly substantial SMF core upgrade due adding more mods than is absolutely needed is a bad idea. While it won't be as bad as the SMF 1.X to 2.0 best avoided. As to manipulating the DB you go do that on your own forum rather than suggesting it here.
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 06, 2021, 01:56:23 am
Given there is still a fairly substantial SMF core upgrade due adding more mods than is absolutely needed is a bad idea.

Indeed, and I've played with the latest release candidates here locally.  I have some understanding of what is in the pipeline.

Quote
While it won't be as bad as the SMF 1.X to 2.0 best avoided. As to manipulating the DB you go do that on your own forum rather than suggesting it here.

I didn't write the recount mod, it is the product of the SMFHacks guys who tend to maintain quality SMF mods, not some random schmoo, at least in my experience and opinion, but like you say, this is not my forum, I'm only offering suggestions.  (Yet this is why I suggested myself talking directly to Dave out-of-band regarding the situation as I simply don't have the time right this second to publicly rehash on a forum things I dealt with several years ago in a similar situation with SMF.)
Title: Re: NEWS: Forum reverted to backup
Post by: bitseeker on May 06, 2021, 02:05:24 am
It appears that the post counter shown below a members avatar is incorrect, the number of posts shown on the last page of a members profile displays the true post count as mentioned above.

Wow, I was once happily oblivious. :-DD

So, the profile post list keeps its own counter? Bizarre. Having worked on an SMF site before, I guess I shouldn't be surprised.
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 06, 2021, 02:09:37 am
Wow, I was once happily oblivious. :-DD

So, the profile post list keeps its own counter? Bizarre. Having worked on an SMF site before, I guess I shouldn't be surprised.

That is an intentional feature, so that if you are doing something like migrating from another forum or user group you can maintain members' post counts, for example.
Title: Re: NEWS: Forum reverted to backup
Post by: Monkeh on May 06, 2021, 02:10:10 am
It appears that the post counter shown below a members avatar is incorrect, the number of posts shown on the last page of a members profile displays the true post count as mentioned above.

Wow, I was once happily oblivious. :-DD

So, the profile post list keeps its own counter? Bizarre. Having worked on an SMF site before, I guess I shouldn't be surprised.

The list you see there is essentially the search results for all posts visible to you from that user, and therefore the number is relative to that. The post count attached to a user is a simple integer incremented for each post creation and decremented for each deletion. There's a few ways to get that out of sync with the actual number of posts (which is not the same as the number of posts located by an unprivileged search).
Title: Re: NEWS: Forum reverted to backup
Post by: bitseeker on May 06, 2021, 02:17:01 am
Wow, I was once happily oblivious. :-DD

So, the profile post list keeps its own counter? Bizarre. Having worked on an SMF site before, I guess I shouldn't be surprised.

That is an intentional feature, so that if you are doing something like migrating from another forum or user group you can maintain members' post counts, for example.

Or just support an offset to the actual count. But no matter. Now I know how many posts I'm missing. D'oh! I shall erase that knowledge from my head as soon as possible.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 03:02:48 am
The post re-counter mod is simple...  If you don't want to go messing with the SQL queries and statements by yourself to by hand to update the counts, it makes it easy.  It's only a like 2k long file, if you're good with DBs, could also just look at the source and do it manually I suppose, but that seems like a lot of effort for essentially one simple thing.   :o

I've never seen it mess anything up, and you can just remove it as soon as you're done the re-count since you're probably not going to be needing it again.  It doesn't have to stay on the system, you just need to run it once to fix the munged post total field...

It's currently running, but sitting at only 1% done.
If it screws up we know who to blame.
My post count went from 32216 to 32730, it processed mine first because I'm account #1
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 03:04:46 am
Aaaaand it's crashed already it seems.
After only a couple of minutes running it asked me to log back in to my account, and it's only at 2%. I did that now it's asking to log back in again after a bit. This is hopeless  :palm:
Maybe it's still running in the background, I have no idea.

UPDATE: Seems to be running now without logging me up, it's up to 6% done.
Title: Re: NEWS: Forum reverted to backup
Post by: lowimpedance on May 06, 2021, 03:22:02 am
 Perhaps it is , as my post count now equals the post count under show posts count and is correct!.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 03:25:05 am
I had two other random accounts open to check:
Ian.M was 9970 now 9993
Muttley was 2050 now 2151
Title: Re: NEWS: Forum reverted to backup
Post by: xrunner on May 06, 2021, 03:43:27 am
Mine is fixed - 5088 to 5156.
Title: Re: NEWS: Forum reverted to backup
Post by: drussell on May 06, 2021, 03:45:02 am
Yeah, no worries, you are totally free blame me if it (recount) somehow hoses your DB, I'm not worried!   8)

I've had it get stuck sometimes but it is essentially looking for mismatches in batches of 100, so you can always just restart it.  It is basically only just copying the length of the array of the [posts] for a user into the displayed "posts" field...

It is really rather harmless...

On other SMF topics, I would really rather discuss directly with Dave first, and NO, installing additional mods on the current 2.0.18 system is not what I'm suggesting.   :palm:
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 06, 2021, 04:31:59 am
On other SMF topics, I would really rather discuss directly with Dave first, and NO, installing additional mods on the current 2.0.18 system is not what I'm suggesting.   :palm:

Public or not at all.
Title: Re: NEWS: Forum reverted to backup
Post by: bsfeechannel on May 06, 2021, 02:46:54 pm
edit: the person you are quoting has had a number of racist posts deleted, so not the most reliable source.

Wow! If I'm reading right, I hope you're not talking about me, because a.f.a.i.k. I've never had a post of mine deleted, much less for racism.
Title: Re: NEWS: Forum reverted to backup
Post by: thm_w on May 06, 2021, 08:50:14 pm
Wow! If I'm reading right, I hope you're not talking about me, because a.f.a.i.k. I've never had a post of mine deleted, much less for racism.

not you
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 07, 2021, 02:40:32 am
The recount task is now complete, so all accounts should now be accurate.
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 07, 2021, 02:45:18 am
The recount task is now complete, so all accounts should now be accurate.
Oh good seems they are now.

Any maybe with less running in the background the server will let bd139 finish archiving the TEA thread:
https://www.eevblog.com/forum/testgear/test-equipment-anonymous-(tea)-group-therapy-thread/msg3564526/#msg3564526 (https://www.eevblog.com/forum/testgear/test-equipment-anonymous-(tea)-group-therapy-thread/msg3564526/#msg3564526)
Title: Re: NEWS: Forum reverted to backup
Post by: bitseeker on May 07, 2021, 03:39:28 am
The recount task is now complete, so all accounts should now be accurate.

Confirmed.

Thanks, Dave, for the fix.

Thanks, Muttley, for the count info.

Thanks, drussell, for the mod tip.
Title: Re: NEWS: Forum reverted to backup
Post by: ebastler on May 07, 2021, 08:35:20 pm
Dave said that blueskull would have deleted his own posts himself if he were allowed to return, but his profile is NOT banned (https://www.eevblog.com/forum/profile/?u=101274).

Not sure whether anyone has clarified this yet? The profile summary page never shows a user as banned, to my knowledge. If you click on any of blueskull's remaining posts (via the "show posts" function in his profile), the poster info in the thread will show him as banned.
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 08, 2021, 03:50:39 am
Dave said that blueskull would have deleted his own posts himself if he were allowed to return, but his profile is NOT banned (https://www.eevblog.com/forum/profile/?u=101274).

Not sure whether anyone has clarified this yet? The profile summary page never shows a user as banned, to my knowledge. If you click on any of blueskull's remaining posts (via the "show posts" function in his profile), the poster info in the thread will show him as banned.

Shows up as banned for me, but I'm an admin.
The one in the sidebar is a plugin IIRC.
Title: Re: NEWS: Forum reverted to backup
Post by: YetAnotherTechie on May 13, 2021, 09:10:58 pm
Since the fire i haven't received any notifications, i've gone back to the settings and choose to be "notified immediately but only for first" and to be notified of "replies and moderation", gone to the threads i was subscribed and clicked unnotify and notify again, and still nothing comes through...
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 14, 2021, 04:07:23 am
Since the fire i haven't received any notifications, i've gone back to the settings and choose to be "notified immediately but only for first" and to be notified of "replies and moderation", gone to the threads i was subscribed and clicked unnotify and notify again, and still nothing comes through...

Anyone else having notification issues?
Title: Re: NEWS: Forum reverted to backup
Post by: bitseeker on May 14, 2021, 06:12:50 am
Notifications have been OK for me.
Title: Re: NEWS: Forum reverted to backup
Post by: YetAnotherTechie on May 14, 2021, 06:44:14 pm
Just checked, asked a member to send me a personal message, and nope, no notifications for followed topics or personal messages  :'(
I'm using hotmail.
(http://)
Title: Re: NEWS: Forum reverted to backup
Post by: tautech on May 14, 2021, 09:49:33 pm
Just checked, asked a member to send me a personal message, and nope, no notifications for followed topics or personal messages  :'(
I'm using hotmail.
You need deeply check your profile settings.
Get PM's almost daily from all manner of EEVblog members ranging from Zero posters to those with 10k+ posts.
Title: Re: NEWS: Forum reverted to backup
Post by: bd139 on May 14, 2021, 10:17:25 pm
There's nothing arriving at outlook.com for reference here. Not getting PM notifications.

To note, outlook.com is notoriously dicky about what it will accept. Literally any deviant config in your SPF/DKIM and sender addressing and it'll shitcan you.

Worth whoever looks after the mail to check here: https://sendersupport.olc.protection.outlook.com/snds/index.aspx
Title: Re: NEWS: Forum reverted to backup
Post by: beanflying on May 14, 2021, 10:22:04 pm
Working fine on Gmail.

@Dave are you still using the native SMF email or are you doing something different with it that might be getting blacklisted by some?
Title: Re: NEWS: Forum reverted to backup
Post by: EEVblog on May 15, 2021, 03:53:23 am
Working fine on Gmail.

@Dave are you still using the native SMF email or are you doing something different with it that might be getting blacklisted by some?

Native SMF, unless gnif has changed it back, but I didn't think there were any plans for that.
Title: Re: NEWS: Forum reverted to backup
Post by: beanflying on May 15, 2021, 03:57:38 am
Working fine on Gmail.

@Dave are you still using the native SMF email or are you doing something different with it that might be getting blacklisted by some?

Native SMF, unless gnif has changed it back, but I didn't think there were any plans for that.

Quite possible the forum might be Blacklisted then. Have a look at this too https://wiki.simplemachines.org/smf/E-Mails_-_Why_are_members_not_getting_emails_sent_from_the_forum

Quick check of the error log will eliminate issues at the SMF send end at least.