Author Topic: Why I can not upload .7z files here?  (Read 8882 times)

0 Members and 1 Guest are viewing this topic.

Offline ZuccaTopic starter

  • Supporter
  • ****
  • Posts: 4306
  • Country: it
  • EE meid in Itali
Why I can not upload .7z files here?
« on: October 12, 2019, 02:06:27 pm »
Any reasons for that?
Can't know what you don't love. St. Augustine
Can't love what you don't know. Zucca
 

Offline Ampera

  • Super Contributor
  • ***
  • Posts: 2578
  • Country: us
    • Ampera's Forums
Re: Why I can not upload .7z files here?
« Reply #1 on: October 12, 2019, 02:33:52 pm »
Because nobody's bothered to put 7z on the list of allowed file types.

Use zip or throw it in a tarball. Or just rename it to a supported file type and tell people to name it back. It's not that complicated.
I forget who I am sometimes, but then I remember that it's probably not worth remembering.
EEVBlog IRC Admin - Join us on irc.austnet.org #eevblog
 
The following users thanked this post: Zucca

Offline ZuccaTopic starter

  • Supporter
  • ****
  • Posts: 4306
  • Country: it
  • EE meid in Itali
Re: Why I can not upload .7z files here?
« Reply #2 on: October 14, 2019, 07:38:11 am »
Thanks Ampera,

it would be cool if someone could add the .7z to the magic list.
1 second Job and less irritation in this forum for the users.
Can't know what you don't love. St. Augustine
Can't love what you don't know. Zucca
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6242
  • Country: fi
    • My home page and email address
Re: Why I can not upload .7z files here?
« Reply #3 on: October 14, 2019, 10:35:47 am »
Adding SVG (.svg) would be darn nice as well, especially considering how much better (clearer, but small size) they are for circuit schematics and such.
 

Offline Ian.M

  • Super Contributor
  • ***
  • Posts: 12855
Re: Why I can not upload .7z files here?
« Reply #4 on: October 14, 2019, 11:53:20 am »
Obviously this forum cant support *every* filetype - there's a considerable benefit in standardising on a limited set so the readers don't have to install dozens of viewers or archive utilities to support a limited number of content creators who want to be different*.

Also, the increasing prevalence of in-browser SVG support is *NOT* a good thing. See: https://blog.online-convert.com/svg-file-and-its-danger/
A sandboxed external viewer that doesn't have internet access is *STRONGLY* recommended.  Its therefore unlikely that the admin team will allow user contributed directly viewable SVG files, for the same reasons as not permitting this forum to host directly executable user contributed Javascript files.

However, if you *must* be the 'nail that sticks out', may I suggest renaming 7z files, replacing the extension with .7z.zip, and SVG files with .svg.txt so its obvious to all that the file needs to be renamed without the final extension before use.

* As there are FOSS zip archivers for Windows, OSX, Linux and Android, its a voluntary choice, unless you are running some really odd-ball OS.
« Last Edit: October 14, 2019, 11:57:03 am by Ian.M »
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3893
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Why I can not upload .7z files here?
« Reply #5 on: October 14, 2019, 11:58:41 am »
Please don't use the freaking 7z archives. I won't bother to install thirdparty bullshitware to be able to unpack those. Probably others won't either.  There is little to no advantage of using this weirdo format over standard zip.
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3893
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Why I can not upload .7z files here?
« Reply #6 on: October 14, 2019, 12:01:58 pm »
Adding SVG (.svg) would be darn nice as well, especially considering how much better (clearer, but small size) they are for circuit schematics and such.

Would be enough to teach people not to use a lossy compressed formats for such drawings. There is already a plenty of choice already: bmp, gif, png,..

SVG is a pain in the arse to work with.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6242
  • Country: fi
    • My home page and email address
Re: Why I can not upload .7z files here?
« Reply #7 on: October 14, 2019, 12:42:27 pm »
Also, the increasing prevalence of in-browser SVG support is *NOT* a good thing. See: https://blog.online-convert.com/svg-file-and-its-danger/
Fuck that.  It is the absolute best format for diagrams, graphs, circuit designs et cetera, exactly because it is vector graphics and thus scalable.

What you mean is not using a sanitizer on the inputs is not a good thing.  Even JPEG and PNG libraries have had remote execution vulnerabilities; all file formats are somewhat a risk.

For SVG, Alister Norris' SVG Sanitizer is the simplest whitelist-based one that would work just fine; it keeps only known-safe SVG elements and their attributes, discarding everything else.  Drupal uses Daryll Doyle's svg-sanitizer, probably because it is more modular and allows easier configuration changes.

A quick look didn't show any SVG sanitizers provided with Simple Machines (this forum software), but it would be trivial to incorporate e.g. the alnorris one to e.g. the SVG As Image Attachment SMF mod.  Heck, if Dave were to disable #hashtags (they fuck up C code snippets!) and interested in adding such sanitized SVG support, I'd be happy to create the mod and push it through the SimpleMachines forums.

As it is, I currently just host the SVG files on my own site, and link them as images.  (I usually hand-optimize the SVG files, and even convert text to paths.  Mine never have any Javascript (although I might add some one day, if I make an animated or interactive SVG), and any CSS is strictly limited to element attributes; the files are entirely self-standing.  Any external references, or references to my server, are as user-visible links, but even those are exceedingly rare.  You see, I am not a hostile idiot, and neither are most of the users here.)
 
The following users thanked this post: tooki, SiliconWizard

Offline soldar

  • Super Contributor
  • ***
  • Posts: 3158
  • Country: es
Re: Why I can not upload .7z files here?
« Reply #8 on: October 14, 2019, 12:45:42 pm »
I would be against .SVG and against .7Z. And I would even be in favor of deleting .JPG which contain line drawings which should be done in .PNG

let's keep things simple.
All my posts are made with 100% recycled electrons and bare traces of grey matter.
 
The following users thanked this post: Yansi

Offline Ian.M

  • Super Contributor
  • ***
  • Posts: 12855
Re: Why I can not upload .7z files here?
« Reply #9 on: October 14, 2019, 01:21:37 pm »
@Nominal Animal,
You are a responsible member of this forum community, with an established reputation here.  The odds of you deliberately posting malicious code here are vanishingly small. 

@All,
However permitting SVG upload *WILL* encourage spammers, and as the drive-by variety never seem to check if their spam links actually work, filtering the SVGs will do little to discourage them.

Trusting forums not to serve you malicious code is not wise. Even with the best of intentions, slipups can happen.  e.g. When Dave first added Mathjax equation rendering support, I pointed out a arbitrary Javascript execution vulnerability, which he quickly secured.  However for a couple of days it was wide open and anyone could have registered a new user and added any arbitrary script to any thread, served from the eevblog.com domain. 

It may be worth setting up any script blocker you are running to prompt before opening scriptable content from www.eevblog.com/forum/*
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6242
  • Country: fi
    • My home page and email address
Re: Why I can not upload .7z files here?
« Reply #10 on: October 14, 2019, 03:32:38 pm »
The odds of you deliberately posting malicious code here are vanishingly small.
No, I meant that in general, any non-new member posting malicious stuff is very small; it affects what one should consider a reasonable balance between security and usefulness.

I am a very heavy SVG format user, and would like it to be more widely supported, because it solves two annoying problems for me: huge, and non-zoomable, illustrations.

Anyone who thinks SVG files are a pain to work with, just try Inkscape, available for all operating systems.

However permitting SVG upload *WILL* encourage spammers, and as the drive-by variety never seem to check if their spam links actually work, filtering the SVGs will do little to discourage them.
Okay, so instead of a sanitizer, make it a filter that rejects any and all SVG files that contain CSS or Javascript.  To conserve server resources, SVG files could be only permitted for users with enough posts (a limitation which IMHO makes sense for all attached files). Again, the alnorris implementation is quite robust, and very suitable for SVG filtering, and I am offering to do the work myself if the feature is considered.

With PNG and GIF files, the problem is that even with maximum compression and using indexed color mode to reduce file size, large enough images allowing zooming get easily into the megabyte file size range.  Most users do not do even that sort of optimization, and tools really cannot do it heuristically (heuristic/opportunistic methods are too slow and processor-intensive to run on a server). That means the image files are large and thus slow to load, and I find it aggravating.  If the image files are small, the details get lost.

Consider my home page: it is about 22 kB, but sharp at any magnification/zoom, because it consists of embedded SVG images.  That is, if you just copy the HTML file, the local copy will look exactly the same, and not depend on having a connection to the server: everything is contained in that one file.  Yet, it is not even "optimized"; it is quite human-readable.

For compression, I prefer xz over ZIP, because it yields smaller files with almost as fast decompression.  (There is even a public domain embeddable xz decompressor for limited use cases.)  However, I understand that many OSes don't have any tools to natively handle those, I accept ZIP files.

Trusting forums not to serve you malicious code is not wise. Even with the best of intentions, slipups can happen.  e.g. When Dave first added Mathjax equation rendering support, I pointed out a arbitrary Javascript execution vulnerability, which he quickly secured.  However for a couple of days it was wide open and anyone could have registered a new user and added any arbitrary script to any thread, served from the eevblog.com domain.
Fully agreed.  I never reuse passwords, because forum database security is never perfect.  My browser clears all cookies, cache, and site data whenever I close it, and I do that at least daily.  I use both NoScript and uBlock Origin, and similar other custom (strict) browser settings, but even then, I don't trust the browser to keep me safe.  It is all a matter of balance between security and usefulness.

Story time.

A bit over a decade ago, I created a small engine in PHP for displaying web pages from static files (HTML or PHP) managed via groups by several overlapping administrators/editors, with customizable navigation menus, with overall navigation roughly reflecting the directory structure.  It worked very, very well for several years, and was eventually used by the entire department.

One of its features was that as an engine, it would be executed for an entire subtree of URLs (for all matching prefixes).  To ensure it does not leak files, especially files or content not meant to be public, it was excessively paranoid about the URLs it handled.  For usability, it URL-decoded and converted accented UTF-8 letters to ASCII (so that /yhtiö/äänestys becomes /yhtio/aanestys and so on), then filtered the result accepting only known safe patterns (in particular, not allowing more than one consecutive . or /, removing all but the known safe characters (letters converted to lower case, digits, dash, underscore, slash, etc.), and so on).
The directory tree under which it looked for content was absolutely anchored, and although it accepted symbolic links, it checked the owner and group of the directory the content was in, to ensure only content from known-published directories was provided.

I would have preferred to write it in C instead of PHP, because then I could have added checks against runtime switch attacks (changing links in the hopes of hitting a race window), and the backwards directory tree walk (from target towards document root) could verify no ongoing shenanigans.  (The backwards walk also lets users use shared navigation menus for a shared subtree.)  However, for maintenance reasons, a scripting language was considered necessary, and all users with access to the files were "trusted" (under contractual obligations to do no such shenanigans), so PHP it was.  I am not sure, but I might have written a Python version also.  (I am writing this from memory, too lazy to dig up the actual code, so I might recall some details wrong too.)

Now, that entire scheme comes together, when you understand that the security approach relies on group ownership.  That is, users modify files using their own local user accounts (with personal default group), manipulate access controls via file and directory group, and had default umask of 002; i.e. by default, users grant the group rw access to the files and directories.  To not risk their personal files, they need personal default groups also, so that their own home directories are owned by their own group.  To grant access to additional groups, ACLs could be used, but IIRC were not needed in practice.

The security approach of that engine is essentially twofold: the URL it gets is not trusted in any way and is filtered for both usefulness and security, and the content is only considered if in a directory specially dedicated for public information, based on its group ownership.  Since system and user directories are never owned by the dedicated published-information groups, link/symlink/rename races are irrelevant, and there is no risk of any content administrator with no system administration access of publishing anything privileged or sensitive, other than copy-pasting such information explicitly.

In fact, I had even designed a WYSIWYG editor, loosely based on TinyMCE (I wanted a floating toolbar, not content boxes), but nobody was interested in that (unless I were to do it for free); most of that work would have been filtering HTML code (since most browsers stuff really silly stuff in DHTML, like FONT elements), and mapping content between different HTML snippet templates (for layout "chunking").  Technically, the approach would have allowed two different users modifying different parts of the same web page at the same time.  And the entire system would have worked even without any database at all.

I didn't just come up with it one morning.  Rather, it was a culmination of about a decade of web server maintenance, and trying to get webmasters to Do The Sensible Thing, often not cooperating with each other.. and basically the approach I could find with the least number of risks that did not unnecessarily limit the users.

Since then, I have looked at existing web forum software, and to be honest, I don't trust them at all.  I have considered writing my own from scratch, but because to do it in a secure manner, I would have to use several users and groups (to establish security boundaries), and current web hotels cannot provide such an environment, it would be limited to those running their own (virtual) servers, not just using a shared web hosting environment.  The way most allow the forum software to create new (script-)executable files and directories indistinguishable form installed forum software files, is a horribly security risk: it escalates all security bugs related to file creation to privilege escalation.  Me, I'd have a hierarchy of accounts, with the login and user account management isolated from normal operation, and all content files and directories owned by a dedicated user and group, with the service process belonging to that group, but running as a yet separate user.  This means that the forum service processes cannot create executable files, and thus avoids script drop problems.  The only downside is that the forum service cannot upgrade itself; it can only trigger it to be done by the real owner, either human or an automated script.

A secondary stopper is that I dislike using a separate database to hold all content.  For a forum, it is an unreasonable dislike; it is just that too often that dependence creates a fragile website, too often down due to limited database resources, or other issues with database connectivity.  I can do it with flat files quite efficiently, but that is considered caveman style... So, I would need to do a bit of research into the different database engines (PostgreSQL, MariaDB, MongoDB) and how to map the security features I'd need to database access.  (For SQL databases, a few database users would suffice, with proper read-write access rights to different tables. There are a few ways to pass the needed credentials safely, but it turns out that checking those credentials (password) is a significant part of the overall load in most cases, so finding a way to safely cache database connections would make it much more resource-friendly.)
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Why I can not upload .7z files here?
« Reply #11 on: October 14, 2019, 10:34:56 pm »
Please don't use the freaking 7z archives. I won't bother to install thirdparty bullshitware to be able to unpack those. Probably others won't either.  There is little to no advantage of using this weirdo format over standard zip.

How, exactly, is a tool which supports a wide variety of formats 'bullshitware'?

It's a common enough format. You want to cut yourself off from opening a whole load of common formats, that's your choice - but kindly don't shove it down my throat.
 
The following users thanked this post: NiHaoMike, Ampera, Jacon

Offline NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9008
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: Why I can not upload .7z files here?
« Reply #12 on: October 14, 2019, 11:11:28 pm »
7z offers better compression ratios than the antiquated zip format. I think we should support it since it is open source.
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 
The following users thanked this post: Ampera, Jacon

Offline mariush

  • Super Contributor
  • ***
  • Posts: 5018
  • Country: ro
  • .
Re: Why I can not upload .7z files here?
« Reply #13 on: October 14, 2019, 11:25:04 pm »
Please don't use the freaking 7z archives. I won't bother to install thirdparty bullshitware to be able to unpack those. Probably others won't either.  There is little to no advantage of using this weirdo format over standard zip.

7zip is an open source archive format (unlike zip or rar).

You can open them with the open source application 7-zip : https://www.7-zip.org/ and other applications support it.

As for benefits, there's actually quite good reasons to use it... if you check solid mode compression and a high compression preset, you can get high compression ratios, higher than what regular zip can do.
 
The following users thanked this post: NiHaoMike, Ampera, Jacon

Offline wilfred

  • Super Contributor
  • ***
  • Posts: 1252
  • Country: au
Re: Why I can not upload .7z files here?
« Reply #14 on: October 14, 2019, 11:37:19 pm »
Adding SVG (.svg) would be darn nice as well, especially considering how much better (clearer, but small size) they are for circuit schematics and such.

If not having .7z files is not because Dave never bothered then this would be the second reason. Or the first.
 

Offline SimonR

  • Regular Contributor
  • *
  • Posts: 122
  • Country: gb
Re: Why I can not upload .7z files here?
« Reply #15 on: October 15, 2019, 04:06:24 pm »
There is little to no advantage of using this weirdo format over standard zip.

There is an advantage as far I know it supports files bigger than 4Gig whereas standard zip does not. Or at least the windows built in zip tool does not.
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Why I can not upload .7z files here?
« Reply #16 on: October 15, 2019, 04:17:21 pm »
There is little to no advantage of using this weirdo format over standard zip.

There is an advantage as far I know it supports files bigger than 4Gig whereas standard zip does not. Or at least the windows built in zip tool does not.

ZIP64 dates to 2001 - Windows has apparently supported it since Vista (only took 5 years for a trivial quality of life update, fairly fast for Microsoft..). Can confirm 10 handles it. Handles it at a maximum of around 120MiB/s with zero compression, as opposed to that 'thirdparty bullshitware' at over 1GiB/s..
« Last Edit: October 15, 2019, 04:26:48 pm by Monkeh »
 

Offline soldar

  • Super Contributor
  • ***
  • Posts: 3158
  • Country: es
Re: Why I can not upload .7z files here?
« Reply #17 on: October 15, 2019, 04:26:25 pm »
Do we really need files larger than 4 GB?

There should be a very justified reason to post anything larger than 150KB. I hate opening a thread only to have my slow connection choke on a huge jpg which could have been converted and compressed down to a tiny png.

Maybe we could limit the size of attachments for newbies until the have enough posts and understand that posting huge files unnecessarily is lack of consideration for others.

I like the idea of 7Z being open source but at the same time ZIP has really become the standard.

With SVG I can't really see the point. Yes, I have Imagemagick and I can use them but for the purposes of this forum I just can't see the need and I can see plenty of people having problems with the. I mean, if someone posts a schematic in SVG I'll just skip the thread so it's not like it's a problem for me. I just think it is unnecesary.
All my posts are made with 100% recycled electrons and bare traces of grey matter.
 
The following users thanked this post: tooki

Online IanB

  • Super Contributor
  • ***
  • Posts: 11859
  • Country: us
Re: Why I can not upload .7z files here?
« Reply #18 on: October 15, 2019, 04:30:36 pm »
Why I can not upload .7z files here?
Any reasons for that?

Because the average user is unable to read .7z files. They are not good for public sharing.
 
The following users thanked this post: tooki

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Why I can not upload .7z files here?
« Reply #19 on: October 15, 2019, 04:32:21 pm »
Do we really need files larger than 4 GB?

No. But that's not a reason to dismiss the use of the format entirely in favour of sticking to barely functional tools.

7z exists. 7z works. There's no good reason to block it. Perhaps we should tell people not to upload software which runs on Windows, because some people don't use it? Or never supply source code in C++ because they don't like C++? Some people probably use PSpice, so we'd best ban LTSpice models!

Why I can not upload .7z files here?
Any reasons for that?

Because the average user is unable to read .7z files. They are not good for public sharing.

Uh, who here is average and incapable of installing software? A significant portion of files uploaded here require one piece of additional software or another to read.
 

Offline soldar

  • Super Contributor
  • ***
  • Posts: 3158
  • Country: es
Re: Why I can not upload .7z files here?
« Reply #20 on: October 15, 2019, 05:47:10 pm »
ZIP is much more widespread than 7Z. ZIP is opened natively by Windows and by Linux Mint.
All my posts are made with 100% recycled electrons and bare traces of grey matter.
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3893
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Why I can not upload .7z files here?
« Reply #21 on: October 15, 2019, 05:52:07 pm »
It is not about being incapable of installing software, but about installing just yet another completely unnecessary software, for some hippie compression format, that has zero to null advantages.
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Why I can not upload .7z files here?
« Reply #22 on: October 15, 2019, 06:12:15 pm »
ZIP is much more widespread than 7Z. ZIP is opened natively by Windows and by Linux Mint.

That's a reason to consider using zip. Not a reason to block the usage of 7z.

Speed alone is a good reason not to use Explorer to handle archives.
« Last Edit: October 15, 2019, 06:15:14 pm by Monkeh »
 

Offline Yansi

  • Super Contributor
  • ***
  • Posts: 3893
  • Country: 00
  • STM32, STM8, AVR, 8051
Re: Why I can not upload .7z files here?
« Reply #23 on: October 15, 2019, 06:17:00 pm »
But I do not have problem with speed of ZIP.

So what motivation is there to use 7z over any conventional format? So that I could compress a JPG image 5% better?
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Why I can not upload .7z files here?
« Reply #24 on: October 15, 2019, 06:27:36 pm »
But I do not have problem with speed of ZIP.

So what motivation is there to use 7z over any conventional format? So that I could compress a JPG image 5% better?

What motivation is there to use zip over any other conventional format?
« Last Edit: October 16, 2019, 12:25:09 pm by Monkeh »
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf