Placing a single file bits contiguously on a HD

[ricko_uk]

Hi,
just trying some experiments and am wondering if there is a way to guarantee that a single file can be guaranteed to be stored on a HD so that every bit of the file is physically next to the previous one without any interruption. The additional assumptions here are:
- the HD is the classic HDD/disk type NOT the SSD or any other memory type
- the HD is brand new or empty and freshly formatted
- preferably using FAT or exFAT

There are two cases:

1) if the file size is less than the sector size when formatted the HDD I assume the file is guaranteed to be stored in a single sector? Or can it still be split across multiple ones?

2) if the file is considerably larger than the sector size (for example 10 or 100 times the size of the sector size) how can it be guaranteed that every single bit is physically placed next to the previous one on the drive?

3) is there a way to specify and/or see the physical position of a byte or chunk of bytes on the drive itself? Is there any software that lets you see each byte and the physical position on the disk?

Thank you as always!

[retiredfeline]

Why do you even want to do this? You can't tell if the drive controller has encoded the block in such a way that the bits are enhanced with error detection/correction and stored in some scrambled order.

[ledtester]

1) if the file size is less than the sector size when formatted the HDD I assume the file is guaranteed to be stored in a single sector? Or can it still be split across multiple ones?

This is a safe assumption.

2) if the file is considerably larger than the sector size (for example 10 or 100 times the size of the sector size) how can it be guaranteed that every single bit is physically placed next to the previous one on the drive?

Be aware that the OS is balancing other concerns when it picks blocks for your file -- such as fragmentation. Also, hard drives may doing some remapping -- e.g. to avoid bad blocks.

You can use a low-level interface to write data directly to specific blocks on the drive, but then you'll also be responsible for the integrity of the file system directory blocks.

Programs which "defragmented" your disk by rearranging where files reside were a thing back in the 80s and 90s, but operating systems have gotten a lot better at file allocation since then.

Another thing that modern OS's will do is to pre-fetch data from a file so by the time your program requests the next block from a file it's already in memory.

[ledtester]

3) is there a way to specify and/or see the physical position of a byte or chunk of bytes on the drive itself? Is there any software that lets you see each byte and the physical position on the disk?

I'd look for "disk editor" / "drive editor" or "data recovery" program such as

- http://www.winhex.com/disk-editor.html
- http://www.runtime.org/diskexplorer.htm

Such programs are used for computer forensics -- e.g. to reconstruct deleted files and directories -- and provide a way to decode the information in the file system's directory blocks.

[T3sl4co1l]

In FAT specifically, contiguous sectors form clusters, so you are guaranteed up to that size at least.

Again, this is logical to the filesystem driver, and need not be true in the drive.  It will be so, in simple enough (usually old) devices, uh not that vintage HDDs are probably all that good of an idea to use, but you can certainly get Flash chips of this type.  (As for specific packages, I don't know; industrial CF or SD perhaps can?  Something about NAND vs. NOR Flash as well.)

Tim

[Manul]

A lot of things about disk geometry, data placement can be analyzed by what is called microbenchmarking. Here is a good read about such experiments.

https://blog.stuffedcow.net/2019/09/hard-disk-geometry-microbenchmarking/

[magic]

just trying some experiments and am wondering if there is a way to guarantee that a single file can be guaranteed to be stored on a HD so that every bit of the file is physically next to the previous one without any interruption

Nope. Firstly, each sector is followed by its corresponding ECC checksum. Secondly, I'm not even sure if it's guaranteed that data within one sector are encoded as the exact sequence of bits you wrote or perhaps transformed. It's known that at least some SSDs perform scrambling.

That being said, I have no idea what kind of experiment that you can reasonably perform could possibly reveal any difference here.

For throughput, it only matters that consecutive blocks of file are written to consecutive sectors. That is easily done on a freshly formatted FS and a reasonable OS will store long files sequentially, or at worst, in large sequential chunks. You will get >99% maximum throughput of your disk.

A single file can also be written onto disk directly, without any filesystem. Particularly easy on Unix.

[viperidae]

The only thing you can guarantee is the logical placement. The hard drive firmware determines physical location. It may be transparently reordering block and may not be writing consequitive blocks on the same platter. The blocks themselves could also be split across platters.
From an application software point of view, you have the filesystem to contend with as well. While most filesystems allow applications to preallocate the storage for an entire file, the sequence of the blocks is still not guaranteed.

[mariush]

NTFS file system may not even reserve a sector for a file, if the amount of data is too small. 

I don't know the amount from memory, it's something like less than 155 bytes if I remember correctly,  basically if the file size is less than this, there's a high chance the data will be in the file table where the metadata is ( file name, dates modified, created, last accessed, security descriptors/permissions, owner etc)

See https://web.archive.org/web/20210506084930/https://docs.microsoft.com/en-us/archive/blogs/askcore/the-four-stages-of-ntfs-file-growth   - the original link is still live, but the embedded pictures are gone, so linking to recent copy (6th of may) that still has the images.

You can also Windows API functions to preallocate disk space to a file, but I'm not sure it guarantees ONE continuous block of sectors ... I think it tries as best as possible to get big empty blocks.
See SetFilePointerEx() and  SetEndOfFile() and  the Set­File­Information­By­Handle functions ... here's a couple links:

How can I preallocate disk space for a file without it being reported as readable? : https://devblogs.microsoft.com/oldnewthing/20160714-00/?p=93875

First answer : https://stackoverflow.com/questions/7970333/how-do-you-pre-allocate-space-for-a-file-in-c-c-on-windows


FAT32 is probably simpler but I'm not sure you're guaranteed to get a continuous chunk for your file... With NTFS you could probably reserve it some way, not sure there's an API for it ... the only other way I can think of is to work like a defragmenting tool and get direct access to the drive and find the largest unused area to create your file. 

Yeah ... and mechanical drives CAN reallocate sectors, if they become too unreliable.  As for SSDs ... they're by design not continuous, because data is shuffled around to extend the life of the ssd.

[fordem]

just trying some experiments and am wondering if there is a way to guarantee that a single file can be guaranteed to be stored on a HD so that every bit of the file is physically next to the previous one without any interruption.

Let's define "every bit of the file".  Do you mean every piece of the file or do you mean every binary digit, as in eight bits make a byte.

[gabiz_ro]

You may try wincontig.

[ricko_uk]

Thank you all for the detailed infos and apps suggested!