Products > Computers

Why CoreUtils/cp does need to fstat a file for a file-copy?

(1/3) > >>

DiTBho:
Why coreutils/cp does need to contain these lines?


--- Code: ---   /*
    * Compare the source dev/ino from the open file to the incoming,
    * saved ones obtained via a previous call to stat
    */
  if (! SAME_INODE (*src_sb, src_open_sb))
    {
      error (0, 0,
             _("skipping file %s, as it was replaced while being copied"),
             quote (src_name));
      return_val = false;
      goto close_src_desc;
    }

--- End code ---

Why a file-copy does need to "fstat" files and compare the source dev/ino from the open file to the incoming, saved ones obtained via a previous call to stat?!?

The result is that when you copy a file on some filesystems ... this happens


--- Code: ---#cp guinea-pig-1.txt guinea-pig-2.txt
skip file 'guinea-pig-1.txt', since it was replaced during copying

--- End code ---

/bin/cp returns FAILURE

This stuff gets really weird when copying a file, failing with a fancy error message.
CoreUtils/{ cp, mv } work OKay copying from local disk.
This only seems to happen when the source file is on an { NFS, CIFS/Samba } volume.
However, I also have this problem on my filesystem that I'm trying to integrate into GNU/Linux.

I still haven't understood why  :-//

I wrote a light version of these file-copy tools, much simpler than Coreutils/{ cp, mv }, but I would like a (toy) filesystem not to depend on custom "tools".

ejeffrey:
I think that shows up if you are copying multiple files into a directory and one of the source files is in the destination directory and is overwritten by a previously copied file.

Why cp skips the file in this case I'm not sure.  For this to come up with a posix compliant filesystem is almost certainly a mistake on the users part with carless use of wildcards or find expressions, so I guess reporting an error is fine, but it will cause problems on non-posix filesystems that can't keep the inode constant.

I understand why network filesystems cannot guarantee correct behavior under all conditions, but a local filesystem shouldnt have this problem.

DiTBho:
I still don't understand either. It looks weird at the moment.
But I know there are different ways to inplement a "file copy" tool

* open(fd_rd, fd_wr), loop(EOF(fd_rd)| is_ok) { read(fd_rd), write(fd_wr) }, close(fd_rd, fd_wr), with all the read and write buffers in userspace. Simplest way to achieve it
* mmap(), buffers work in kernel space
* via sendfile() system call, so everything is done in kernel space. In theory it should only work with sockets, but it seems it has become a superset and also works with file descriptors
* via copy_file_range() system call, so everything is done in kernel space. It gives better results, and it's the fastest way, only if the filesystem supports “copy-on-write”
* via clonefile() + indirect ioctl system call, It seems very convoluted to me, but they say that it can give better results, again only if the filesystem supports “copy-on-write” and other modern mechanisms
In the past, I only implemented the first (simplest) one, because on XINU I don't have any of the other possibilities offered by the Linux kernel.

DiTBho:
Funny and somehow useful alternatives to "cp"

--- Code: ---name         description                                          written in...
--------------------------------------------------------------------------------
coreutils    Default cp, mv                                       C
xcp          Extended cp                                          Rust
fcp          Significantly faster alternative to cp               Rust
gcp          Goffi's cp, a fancy file copier                      Python
pycp         cp and mv with a progressbar                         Python
rsync        fast incremental file transfer                       C
fuc          Fast UNIX cmds, provides alternatives for rm and cp  Rust
renameutils  Make copying files faster and less cumbersome        C

--- End code ---


* coreutils, see here
* xcp, see here
* fcp, see here
* gcp, see here
* pycp, see here
* rsync, see here
* fuc, see here
* renameutils, see here

Nominal Animal:

--- Quote from: DiTBho on May 30, 2024, 01:21:46 pm ---Why a file-copy does need to "fstat" files and compare the source dev/ino from the open file to the incoming, saved ones obtained via a previous call to stat?!?

--- End quote ---
Because TOCTOU: otherwise leaving a bait-and-switch race window open, exploitable by a local (unprivileged) attacker when a privileged user is copying an unprivileged file to a privileged location.

Fundamentally, there may be many more file names specified as command line parameters than can be kept open at the same time.  All source files need to be opened or stat()ed, at minimum to check if the arguments are files or directories, but also to avoid overwriting same files and detecting other problematic situations.  Thus, cp ends up stat()ing each source file twice: once beforehand, then again after opening it.  If the two differ, the source file was replaced between the two operations, and copying it could allow a security exploit.

A "bait-and-switch" attack relies on the target program first verifying the (unprivileged) source file, then the attacker changing it, before the target file uses (here, copies) it to a privileged directory.  This is a serious problem in general, and an endless source of exploitable time-of-check to time-of-use bugs.

Navigation

[0] Message Index

[#] Next page

There was an error while thanking
Thanking...
Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod