Yahoo demise: how can information be preserved?

[tggzzz]

It is, finally, beginning to look like Yahoo! will be broken up and sold in the forseeable future. When that happens it is reasonable to presume that bits will be turned off because they aren't profitable, with the loss of all information therein. Sometimes that can happen very quickly, e.g. the first large social media site in the UK, FriendsReunited.

One or two Yahoo! groups contain valuable practical folklore and techniques, e.g. the TekScopes group. No doubt many other people will feel the same about their personal favourite groups.

So, if worst comes to worst, what's the best way of (a) extracting and preserving, and (b) making accessible the information in such groups?

[Rerouter]

If they are free to view and download attachments, i would recommend a site archiving package, though i wouldnt be suprised if its not already in the many internet archives out there

[German_EE]

Is it possible for the site administrators to download the message library to a zip file?

[rx8pilot]

This is why I do not use cloud services for anything that I care to hold on to. All information that I care about - including email  - I keep entirely on local drives.

I have lost too much stuff when various companies change something, sell off, or go out of business.

[MrSlack]

This is why the cloud is bad.

Scraping software is probably the way to go. Then reprocess it into plain HTML, archive the attachments and files, zip it and stick it on a torrent site. I might do this myself as I have written a ton of scraping software over the years.

[tggzzz]

This is why the cloud is bad.

My view is that it is the triumphant re-invention of timesharing bureaux.

Thise with long memories will recall the shouts of delight when PCs became widespread - because then the user owned and controlled their own data and processing, rather than having it under the control of white-coated magicians in freon-protected aircon halls.

The Hindus are right: it isn't time's arrow, it is time's wheel.

[SeanB]

Send a message to them to move here, they will likely get a new forum, and can only improve this one as well.

[Kilrah]

I still have a 4GB mostly-working archive of an MSN group I cared about when those closed about 10 years ago, grabbed with whatever web crawler that actually wasn't very intelligent and du(tri,...)plicated lots of stuff.

And yes, everything I can is stored on local hard drives. Would hate if Flickr went down though.

[Cubdriver]

Send a message to them to move here, they will likely get a new forum, and can only improve this one as well.

That was my first thought as well, though certainly not my place to do as this is Dave's house - it would be great to get all those HP and Tektronix guys on here.  I'm on both groups (mostly as a lurker), but I much prefer a forum format like this to the klunky Yahoo mailing list, and have never had much success searching on their main pages - it's painful to do in my experience at least.

-Pat

[edavid]

1. Yahoo itself does not provide any way to export or archive group content.  Most of the Yahoo group downloading scripts out there stopped working when Yahoo ruined everything with their "Neo" transition.

2. Many of the crotchety old members are adamant that they will never, ever use a forum instead of an email list.

3. groups.io looks like a good option, since they will suck over the Yahoo message archive, but who knows how long it will be around.  If I were a group owner, I would go with them if they would be willing to give me an offline archive.

4. But, if we're talking about TekScopes and hp_agilent_equipment, neither group owner is exactly proactive, so it's hard to say if they will get it together to do something before it's too late.

[MrSlack]

This is why the cloud is bad.

My view is that it is the triumphant re-invention of timesharing bureaux.

Thise with long memories will recall the shouts of delight when PCs became widespread - because then the user owned and controlled their own data and processing, rather than having it under the control of white-coated magicians in freon-protected aircon halls.

The Hindus are right: it isn't time's arrow, it is time's wheel.

This.

Then again I work for a financial cloud company so I'm a giant hypocrite.

[tggzzz]

Send a message to them to move here, they will likely get a new forum, and can only improve this one as well.

That was my first thought as well, though certainly not my place to do as this is Dave's house - it would be great to get all those HP and Tektronix guys on here.  I'm on both groups (mostly as a lurker), but I much prefer a forum format like this to the klunky Yahoo mailing list, and have never had much success searching on their main pages - it's painful to do in my experience at least.

Personally I like the email variant since it is trivial for me to personally archive whatever is interesting to me - and in a decent threaded format (cf the linear format on all web forums).

For newcomers and people with new interests, the Yahoo search facility is much more effective than that on this forum.

But most important is the ability to retain all the existing knowledge. Currently groups.io looks like a good starting point, but this is a good time to explore possibilities.

[Kilrah]

Personally I like the email variant since it is trivial for me to personally archive whatever is interesting to me - and in a decent threaded format (cf the linear format on all web forums).

To me email lists are an archaic unuseable thing, and the threaded format is just a thing that's misused 90% of the time and only leads to a lot of mess because going through topics within a topic is just never-ending.

Something about taste and points of views obviously makes a merge difficult...

[SeanB]

I still have a list server running........ Most of the time the only posts are the daily 2-3 spam emails sent to it from random senders, though I get around 1 non spam per year from somebody looking for info. Only reason it is still up is that storage space is cheap, and there is another list server or ten running on the same virtualised server anyway. Might be 0.01% extra CPU time on the machine. As it is a moderated list, and I am the BOFH admin, it is very quiet.....

[tggzzz]

Personally I like the email variant since it is trivial for me to personally archive whatever is interesting to me - and in a decent threaded format (cf the linear format on all web forums).

To me email lists are an archaic unuseable thing,

Which, of course, says as much about you as about email lists.

and the threaded format is just a thing that's misused 90% of the time and only leads to a lot of mess because going through topics within a topic is just never-ending.

Which indicates you don't understand.

You can't "misuse" a threaded format. It just is. The threading is completely and solely defined on the basis of which email is a response to which other email. There is no concept of a "topic within a topic".

The threading is implemented by your mail user agent. You control and select it independently of anyone else, and can change it on a whim. For example, one of my email folders contains 8958 emails,  going back to 2005. I flip between linear-date-order and threaded-date-order several times a day, and each flip takes a small fraction of a second.

The same is true of usenet postings, where I also get my reader to highlight emails by anyone I'm particularly interested in, e.g. one of the authors of "The Art of Electronics".

You cannot do that with any web-based forum.

[DimitriP]

This is why the cloud is bad.

Scraping software is probably the way to go. Then reprocess it into plain HTML, archive the attachments and files, zip it and stick it on a torrent site. I might do this myself as I have written a ton of scraping software over the years.

No need to re-invent the wheel...    http://www.personalgroupware.com/

[Wilksey]

Cloud! Yay! Did no-one learn their lesson with Geocities?

[tggzzz]

Cloud! Yay! Did no-one learn their lesson with Geocities?

Yes, they did learn.

Then new people come along and repeat old mistakes. That's compounded by people presuming old=bad, new=better. (Ah! The arrogance of youth!)

[station240]

I'm seeing an even bigger problem emerge with chatrooms being used for technical support, of course it's a cloud system.

There is no archive of anything, so unlike the forum, the technical knowledge is simply lost hours after it's created.

[MrSlack]

The joyous chat room situation was solved by our glorious support team by saving all chat transcripts, in jira, in the cloud, which we don't backup at all... Sigh...

[hendorog]

Google - ironically? - found this...
https://sourceforge.net/projects/grabyahoogroup/

and this
http://www.usenix.org.uk/content/yahoo_mbox.html

[German_EE]

From the web page of the first link:

"[ALERT] The new neo interface has broken the script and I have not found any easy way to work with it. For now the script is not being actively maintained till a solution materializes to access the relevant data easily."

It looks as if Yahoo have thrown a spanner in the works.

[rrinker]

 That horrible "Neo" interface is a couple of years old now. Yahoo threw a spanner in the works alright, that was the beginning of the end. Prio to "Neo", Yahoo Groups was actually pretty useable. Wasn't difficult to search or find specific posts. Neo broke all that, and overnight it all became an unusable mess. I'm on more than a dozen, mostly model railroad related, though I only get email from about 5 of them. I rarely post because it's almost impossible. And not even WYSIWYG, in the editor box all you see is your text you've types, if you miss the little check box you end up quoting the full contents of EVERY post in the thread that came before your response. Horrible UI design.
 Then they bring in an absolute boob to run the company - she might be pretty to look at but she has the business acumen of a rock. All of their news sources have turned into pure clickbait and downright malware links, it's all those sort of sites where a half page story is spread across 10 pages so they can present even more ads. They used to aggregate all the popular comics, they tossed that. Fine, no ad revenue for you, I get my Dilbert direct from the source now, let Scott Adams get the revenue to hits on his site.

[hendorog]

From the web page of the first link:

"[ALERT] The new neo interface has broken the script and I have not found any easy way to work with it. For now the script is not being actively maintained till a solution materializes to access the relevant data easily."

It looks as if Yahoo have thrown a spanner in the works.

Yeah I saw that too, I figured it was worth highlighting just in case someone was up for fixing it.

Hrm, what about this:
http://www.archiveteam.org/index.php?title=Yahoo!_Groups
https://archive.org/download/yahoo_groups

[XOIIO]

Now we can only hope bing goes next.