General > General Technical Chat

Server Error Reports

<< < (140/142) > >>

gnif:
Bumping this thread so people see this.

We will be making some major changes soon that will likely keep the forum down for several hours. I have asked Dave to make an announcement and schedule a time for when this is to commence. The goals here are to address several things.

1) As the forum has grown so has the hosting requirements, we are reaching I/O saturation and have decided to upgrade to faster storage medium.
2) We are still seeing attachment issues which still seem to be the fault of some rare bug in GlusterFS. As a result we are moving away from this to DRBD + GFS2
3) Even if GlusterFS was not faulting with the attachments, it's starting to get very slow and causing random stalls when dealing with large directories.

The decision to move from GlusterFS to DRBD + GFS2 has been made after several months of evaluation, however as we do not have a testing environment, we wont know how well it will work until we pull the trigger. We do expect quite a performance improvement, however it will be hard to determine if that is due to the change in the software stack, or the hardware upgrade.

EEVblog:
Thanks Geoff.
Go time will be 11pm tinght AEST.
Feel free to touch grass for a few hours.
Thanks.

gnif:
Forums are back up (obviously) however attachments are still being copied over, so expect broken attachments until they have all moved over.

EEVblog:

--- Quote from: gnif on January 12, 2024, 01:33:43 pm ---Forums are back up (obviously) however attachments are still being copied over, so expect broken attachments until they have all moved over.

--- End quote ---

Thanks.
Forum seems crazy quick!
But it was always slick.

gnif:
Yes, there is certainly a huge performance boost here, IO load on average has dropped from 90+% to around 5-10%. DRBD + GFS2 is so much faster I can actually now count how many attachment files are on disk (for the record, over 810,000 files in one directory (SMF :palm:), and because it's so much less disruptive now to manage this directory we can start to look at migrating this into a proper directory structure.

Note the work is still not complete though, there is still a few things to shuffle around and there may be some minor outages lasting a few minutes while the work is completed.

Navigation

[0] Message Index

[#] Next page

[*] Previous page