EEVblog Electronics Community Forum

Electronics => Repair => Topic started by: Hugoneus on August 17, 2022, 11:36:09 pm

Title: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: Hugoneus on August 17, 2022, 11:36:09 pm
I have a hard drive from a 86142B, which is an optical spectrum analyzer. The instrument has a HDD with the firmware in it. The HDD does not have any partitions nor does it show up as anything meaningful under windows.

I tried Clonezilla (which is Unix-based) to duplicate it. But the software doesn't like the fact that the HDD has no partitions. It is as if they just wrote data to the sectors of the drive and used it like "memory" without any file structure.

Has anyone dealt with anything like this?
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: fzabkar on August 17, 2022, 11:44:47 pm
If the HDD is healthy, you can use the Tools -> Copy Sectors function of DMDE to clone the drive sector-by-sector. You can either clone it to another physical HDD, or you can create an image file.

https://dmde.com/ (https://dmde.com/)

DMDE is both a disc editor and a data recovery tool. It can recognise many file systems, and it can extract known file types from the image, if any exist.

If the HDD has bad sectors or bad heads, you can use HDDSuperClone (now open source). This tool understands how to work with failing HDDs.

http://hddsuperclone.com/ (http://hddsuperclone.com/)

In both cases the cloning process is OS-agnostic, ie the HDD is treated as a bunch of bits.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Monkeh on August 17, 2022, 11:47:23 pm
Fire up any Linux-based system and just use dd to copy the raw block device across. You can probably just use Clonezilla and switch to a terminal to do it.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Hugoneus on August 18, 2022, 12:01:20 am
I am going to try the suggestions now!
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: antenna on August 18, 2022, 12:58:55 am
Try Macrium Reflect
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: SoundTech-LG on August 18, 2022, 01:36:08 am
This has worked for anything I have thrown at it...

https://www.hddguru.com/software/HDD-Raw-Copy-Tool/ (https://www.hddguru.com/software/HDD-Raw-Copy-Tool/)

Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: apelly on August 18, 2022, 02:06:34 am
Has anyone dealt with anything like this?
This is a classic "hard-to-google" problem because there are a million charlatans trying to take advantage of the no-backup-numpties who are desperate to get back their almost complete first novel, thesis, or kids baby photos.

I wouldn't recommend buying anything. There is certainly a linux tool that will work.

As suggested: dd, is the obvious choice. One thing I would say about dd though; depending on the size of the HDD you're copying it can be worth while experimenting with the block size option. Doesn't effect the end result, but may reduce the time required by a bunch.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Hugoneus on August 18, 2022, 02:16:05 am
Thank you everyone! You guys are awesome. I managed to do it with DMDE and direct sector copy.

I also realized how horrible Google has become. None of the tools you suggested came up in my many searches. Except for Macrium Reflect which doesn't do sector-by-sector copy.

A dirty tactic that many sites use is to write a "fake" forum post about their own product and answer they own questions. They make the whole thing look like legitimate and of course Google indexes it and this makes 90% of search results useless.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Uunoctium on August 18, 2022, 03:00:57 am
tl;dr

You can try X-Ways Replica, an DOS based cloning tool. Came as give-away with a Winhex licence 20 yrs ago. One can choose "entire disk" and it works with un-partitioned hard disks since V0.94 (see history).
Only an DOS formatted start-disk is needed where replica.exe is present (48kB). Depending on disk size and host system a copy takes up to 2hrs, cause Replica copies exact bit-by-bit including unused and free space.
http://www.winhex.com/replica.html (http://www.winhex.com/replica.html)
(no longer distributed since'05, PM for further information)
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: andy3055 on August 18, 2022, 04:11:22 am
Taking note of this thread. Great ideas.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Ed.Kloonk on August 18, 2022, 05:39:34 am
Thank you everyone! You guys are awesome. I managed to do it with DMDE and direct sector copy.

I also realized how horrible Google has become. None of the tools you suggested came up in my many searches. Except for Macrium Reflect which doesn't do sector-by-sector copy.

A dirty tactic that many sites use is to write a "fake" forum post about their own product and answer they own questions. They make the whole thing look like legitimate and of course Google indexes it and this makes 90% of search results useless.

The problem with recommending all over the internet tools like dd is that they can be a double-edged sword.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Berni on August 18, 2022, 05:50:41 am
Yeah better be very VERY careful when using linux dd. It can wreck all of the data if the wrong parameters are put in.

My own solution is "EaseUS Todo Backup Free" it has a horrible name, but it can clone drives 1:1. Most of the reason i use it is that it can also resize partitions afterwards, so it comes pretty useful in moving the OS from a HDD to a SSD.

Another useful tip is that WinHex has the ability to open disk drives in raw form, letting you explore the contents of a drive as raw sectors and copy things around by hand. It can be used to do things like manually fix a damaged partition table or do forensic analysis on data left behind by the filesystem.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Ed.Kloonk on August 18, 2022, 06:08:53 am
I'd be interested to know just what the FS structure looks like, and why on earth the designers wouldn't use something off the shelf.

I've heard of some exotic PVR's where they corrupt the partition table so the ordinary windows user can't copy the sacred files on it.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Berni on August 18, 2022, 06:19:13 am
I'd be interested to know just what the FS structure looks like, and why on earth the designers wouldn't use something off the shelf.

I've heard of some exotic PVR's where they corrupt the partition table so the ordinary windows user can't copy the sacred files on it.

It could be that they just used a filesystem so obscure that none of the normal disk tools out there can recognize it.

There is a similar case of weird formats. Like in the HP 4145 boots from floppy but it uses a weird format that can't be read by a PC. Not even with special software, it needs a special floppy controller to do it. So as a result if you loose/corrupt the boot disk you have very little chance of finding a piece of hardware that can write a fresh boot floppy. Some of the HP dynamic signal analyzers also use weird formats, but some of them will at least read regular DOS formatted disks.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Nominal Animal on August 18, 2022, 06:31:33 am
I wouldn't recommend buying anything. There is certainly a linux tool that will work.

As suggested: dd, is the obvious choice. One thing I would say about dd though; depending on the size of the HDD you're copying it can be worth while experimenting with the block size option. Doesn't effect the end result, but may reduce the time required by a bunch.
I recommend starting with
    dd if=source-device of=target-device bs=2M iflag=fullblock conv=fdatasync
which reads and writes in two-megabyte (2,097,152-byte) chunks, and ensures the data hits the target device before it returns (so if the target device is slow, like USB Flash or similar, there will be a delay after it is seemingly done before it returns, because it waits for the operating system to tell it has all been written to the target device before returning).

To check what device you should use, most Linux distributions include the util-linux package, which provides a nice command for this:
    lsblk -at
The un-indented devices are the block devices (TYPE=disk), and the indented devices are the filesystems and partitions on them (TYPE=part or TYPE=crypt if encrypted) that are currently mounted.

If you want to check if /dev/foo is the one you think it is, then
    cat /sys/block/foo/device/transport
will tell you which bus it is connected with (PCIe, USB);
    cat /sys/block/foo/device/model
will tell you its model,
    cat /sys/block/foo/device/vendor
will tell you its vendor/manufacturer (if known), and
    cat /sys/block/foo/device/serial
will tell you its serial (if known).

If you are lazy like me, then you write a Bash script or alias that does
    for dev in /sys/block/*/device ; do d="${dev%/*}" ; printf '%s:\n Model: %s\n Vendor: %s\n Serial: %s\n Transport: %s\n\n' "/dev/${d##*/}" "$(cat "$dev/model")" "$(cat "$dev/vendor")" "$(cat "$dev/serial")" "$(cat "$dev/transport")" ; done 2>/dev/null
which makes it quick and easy to identify which is which.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: JohanH on August 18, 2022, 06:55:55 am
A vote for dd here as well. I've tried all kinds of crap backup tools for windows and none are good for professional use. The foolproof way is using dd. Now I always use a Linux machine for creating master copy hard disks/files (for production use in a factory).

Be sure to have enough disk space, otherwise you will get all sorts of issues.

A good way to ensure that the copy matches the disk is to verify checksum of source disk and target file.

After running the dd if=... of=... command, creating a file on the disk, look for the output where it said how many bytes it wrote, then use this number to calculate the checksum of the disk (change /dev/sdX to your actual disk):
sudo head -c <number of bytes> /dev/sdX | md5sum

Then compare to the file:
md5sum -b <your file>

The numbers should match. Now store the checksum in a text file so you can verify it at any time.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: apelly on August 18, 2022, 10:05:34 am
dd [...] can be a double-edged sword.

True. Most *nix command line tools will not ask if you are sure you want to shoot your grandma.

Personally, I'd rather trust myself to read my own script, than rely on some random internet windows software. YMMV.

You only
#dd if=/dev/random of=/dev/sda
a few times before you learn to read what you type before you press go. :)

Also... For anyone following... Don't copy pasta that... It's not a useful command.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Ed.Kloonk on August 18, 2022, 10:23:37 am
dd [...] can be a double-edged sword.

True. Most *nix command line tools will not ask if you are sure you want to shoot your grandma.

Personally, I'd rather trust myself to read my own script, than rely on some random internet windows software. YMMV.

You only
#dd if=/dev/random of=/dev/sda
a few times before you learn to read what you type before you press go. :)

Also... For anyone following... Don't copy pasta that... It's not a useful command.

I'm with you. The point I failed to make was that the OP received the right responses due to the way the question was asked.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Zenith on August 18, 2022, 10:45:32 am
I'd use dd.

However, unless you are used to it, it can certainly cause disasters very easily. I had one or two when I first tried it years ago.

So if you haven't used dd before, it would be sensible to become familiar with it using an old spare system, where it doesn't matter if you mess it up.

Can the HDD from the SA be jumpered to be read only? If so that would be a useful precaution.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Nominal Animal on August 18, 2022, 11:00:24 am
dd [...] can be a double-edged sword.

True. Most *nix command line tools will not ask if you are sure you want to shoot your grandma.
Exactly.  Which is why most of my advice related to dd use in #14 (https://www.eevblog.com/forum/repair/anyone-knows-how-to-clone-a-hdd-without-any-partitions/msg4364785/#msg4364785) was about how to make sure your source and target devices are the ones you think they are.

I never trust assumptions, not even if they are my own.  It's better to verify, and be sure, before you pull the trigger.  ;)
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: fzabkar on August 18, 2022, 05:34:49 pm
A vote for dd here as well. I've tried all kinds of crap backup tools for windows and none are good for professional use. The foolproof way is using dd. Now I always use a Linux machine for creating master copy hard disks/files (for production use in a factory).
FYI, DMDE runs on Linux ...

BTW, dd is only good for healthy storage devices. A better tool is ddrescue, and even better is HDDSuperClone. All are Linux software. DMDE does have configuration options to tune its error recovery behaviour, but it is not as good as the specialised cloning tools. If you have a really bad HDD that goes offline when it hits bad patches in the media, the pro version of HDDSuperClone (now open source) can operate a power relay to reset the drive. Otherwise you would need to baby-sit it.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Hugoneus on August 18, 2022, 05:37:54 pm
Yeah better be very VERY careful when using linux dd. It can wreck all of the data if the wrong parameters are put in.

My own solution is "EaseUS Todo Backup Free" it has a horrible name, but it can clone drives 1:1. Most of the reason i use it is that it can also resize partitions afterwards, so it comes pretty useful in moving the OS from a HDD to a SSD.

Another useful tip is that WinHex has the ability to open disk drives in raw form, letting you explore the contents of a drive as raw sectors and copy things around by hand. It can be used to do things like manually fix a damaged partition table or do forensic analysis on data left behind by the filesystem.

EaseUS does not work. As I said, there is no file structure, no partition, nothing at all. It is purely sector written data as if it was in some memory location.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: KE5FX on August 18, 2022, 05:44:32 pm
I haven't seen anything that Clonezilla can't handle, myself.  You can put it on a bootable USB drive, no need to install anything, run Linux, etc.

Avoid the commercial Windows tools like Macrium, they are all basically malware these days.  Intentionally designed to run at startup time and be as difficult as possible to uninstall.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Hugoneus on August 18, 2022, 05:46:00 pm
I haven't seen anything that Clonezilla can't handle, myself.  You can put it on a bootable USB drive, no need to install anything, run Linux, etc.
Avoid the commercial Windows tools like Macrium, they are all basically malware these days.  Intentionally designed to be difficult to uninstall.

None of the options of Clonezilla worked (I did not try the command line directly). Since the disk has no partition or initialization, it just says it can't work with this drive.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: JohanH on August 18, 2022, 05:59:27 pm
dd will create an 1:1 copy of the disk, even if it contains only zeros. If your goal is to only create a backup copy, this is enough. The file can then easily be written to a new disk, if desired.

ddrescue might be of some value if you wanted to analyze the data. I've only tried to use it a couple of times and it's kind of hard to use. You could try to use it to see if it finds some kind of structured data. I would copy the disk to a file first with dd, to not mess with the disk, then try to analyze the file with ddrescue or some other tool.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: fzabkar on August 18, 2022, 06:15:50 pm
ddrescue might be of some value if you wanted to analyze the data.
Ddrescue is purely a cloning tool. It doesn't analyse anything. There is a companion utility (ddrutility) which can use ddrescue's log file to determine which, if any, files are affected by bad sectors. This utility was written by the author of HDDSuperClone.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Zenith on August 18, 2022, 06:55:36 pm
Why do you want to do this - apart that from having a backup of an HDD is generally a good idea?

Do you have an suspicion that the HDD is acting up? There are numerous utilities for checking the health of HDDs. If it's playing up with bad sectors, you may about to begin a voyage of discovery.

You should be able to use dd to clone another disc or write a file to disc which can then be used with dd to produce a clone on another hdd.

As I said earlier, it's worth practicing on a throwaway system before you try to use dd in earnest.



Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Zenith on August 18, 2022, 06:57:51 pm
ddrescue might be of some value if you wanted to analyze the data.
Ddrescue is purely a cloning tool. It doesn't analyse anything. There is a companion utility (ddrutility) which can use ddrescue's log file to determine which, if any, files are affected by bad sectors. This utility was written by the author of HDDSuperClone.
This HDD doesn't seem to have a file system which is recognised by anything.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: fzabkar on August 18, 2022, 07:16:07 pm
ddrescue might be of some value if you wanted to analyze the data.
Ddrescue is purely a cloning tool. It doesn't analyse anything. There is a companion utility (ddrutility) which can use ddrescue's log file to determine which, if any, files are affected by bad sectors. This utility was written by the author of HDDSuperClone.
This HDD doesn't seem to have a file system which is recognised by anything.

Yes, I get that. I was addressing the claim that "ddrescue might be of some value if you wanted to analyze the data".

In fact, if the HDD stores the user's data, then by definition it must have some kind of file system, albeit proprietary. It would be interesting to examine it ...
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Zenith on August 18, 2022, 08:21:51 pm
Don't forget that modern HDDs offer a logical addressing space, but not necessarily one that may be used in an obviously rational way.

Were it my problem I'd have cloned it using dd to a cheap SSD, and also some naff HDD which came from a laptop or a chucked out set top box using converters.

But it's not my problem, so I help as best I can.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Gerhard_dk4xp on August 18, 2022, 10:17:40 pm
That's exactly what I did today when my vector network analyzer
did not boot and made funny noises that could have come from
the hard disk. In fact, they did not.

dd does not care about the contents of a disk, it just copies sector
by sector. I mounted the disk in the place of the disk that normally
houses the virtual machines on my Linux workstation and saved
all of the VNA disk (/dev/sdf)  to a 160G file, and then the first
and only partition (/dev/sdf1) to a different file.

I still had a 256G Samsung SSD, emptied it, made a new partition
table and copied the contents of the VNA partition file back to the SSD.
You can just as well copy the entire disk without looking at partitions.

The VNA is now somewhat snappier, the original disk and the files
are archived. I'll sleep better now.
The hardware bug was sth. with the power supply, preliminarilly fixed.

When you use dd, make 3 times sure that the partition you write to
is really the one you want. It will overwrite your boot partition if you
say so.  gparted tells you which partition is which in a nice graphical
way, but everything that can sweep a partition is dangerous.

Gerhard



Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: abdulbadii on August 19, 2022, 01:27:30 am
Imho many on Linux

gdisk
dd
hdparm
GSmartControl

Imho the proper place is on S/W forum site eg. stackexchange ones, not here
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Hugoneus on August 19, 2022, 01:18:50 pm
Sector by sector copy worked just fine.

Agilent has removed all software/firmware updates for this unit from the site. So if the HDD fails, that is it, the unit becomes unusable. I copied into an IDE SSD and the instrument is now safe!
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: Monkeh on August 19, 2022, 01:27:38 pm
Sector by sector copy worked just fine.

Agilent has removed all software/firmware updates for this unit from the site. So if the HDD fails, that is it, the unit becomes unusable. I copied into an IDE SSD and the instrument is now safe!

I recommend making images onto higher density storage - it's more convenient and reliable in the long term.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: coromonadalix on August 19, 2022, 02:27:45 pm
or at least, do  2  hhd backup(s)   

I did had one problem with one copy, the second one was okay,  the receiving disk or system had some problem  ...   lolll


I had one equipment who used an hdd in what we call "raw" mode i think,  it could not be read by windows or linux in a standard way, appeared as a "phantom" drive of some sort

It was accessible  by Winhex and somekind of a script was used  ...    hdd raw copy did the trick,  sure it was a long process,  but ended fine
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: IanB on August 19, 2022, 02:59:41 pm
Sector by sector copy worked just fine.

Agilent has removed all software/firmware updates for this unit from the site. So if the HDD fails, that is it, the unit becomes unusable. I copied into an IDE SSD and the instrument is now safe!

For archival purposes it might be safer to copy to a magnetic disk?
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: free_electron on August 19, 2022, 03:11:37 pm
on the risk of hacking this topic : since all the file system securities are basically handled by the file system driver , is there a driver that ignores the security ?
The file system says file xyz belongs to user yadda, or has certain bits turned on (like it he dos days , archive, read only , system , hidden etc )
This is all by convention and handled in the file system driver. swap out the driver for a version that ignores all this hoopla and you can read write anything you want.

i ran into a situation on an old computer of mine where i had two logins. i was admin on both. i wanted to delete one of them and no matter what it tried i could not delete one directory. you need admin permission. i am admin. access denied ... it was a google drive shadow folder. i finally found some obscure command line operation that could reset the flags so i could access the files.

having a file system driver that goes "screw all these security locks, i'm not respecting them" would solve such issues.

Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: alm on August 19, 2022, 03:58:46 pm
on the risk of hacking this topic : since all the file system securities are basically handled by the file system driver , is there a driver that ignores the security ?
The file system says file xyz belongs to user yadda, or has certain bits turned on (like it he dos days , archive, read only , system , hidden etc )
This is all by convention and handled in the file system driver. swap out the driver for a version that ignores all this hoopla and you can read write anything you want.

i ran into a situation on an old computer of mine where i had two logins. i was admin on both. i wanted to delete one of them and no matter what it tried i could not delete one directory. you need admin permission. i am admin. access denied ... it was a google drive shadow folder. i finally found some obscure command line operation that could reset the flags so i could access the files.

having a file system driver that goes "screw all these security locks, i'm not respecting them" would solve such issues.
What file system, and what host OS are you talking about? With a Linux host, as long as you have root, it will let you do anything to mounted file system. Also, when mounting an NTFS filesystem on Linux, it allows you to specify which Linux user should be the owner and what the Linux permissions will be: see the documentation for the mount command (https://manpages.ubuntu.com/manpages/xenial/man8/mount.8.html#mount%20options%20for%20ntfs).
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: free_electron on August 19, 2022, 05:20:16 pm
Quote
What file system, and what host OS are you talking about? With a Linux host, as long as you have root, it will let you do anything to mounted file system. Also, when mounting an NTFS filesystem on Linux, it allows you to specify which Linux user should be the owner and what the Linux permissions will be: see the documentation for the mount command (https://manpages.ubuntu.com/manpages/xenial/man8/mount.8.html#mount%20options%20for%20ntfs).
any file system any os.

Boot the computer from a customised minimal shell that ignores all the security and just lets you read/write anything. No password, No logins. As long as a file system is not encrypted this should exist.
Boot from a usb or cd into a tool that can read write ntfs , zfs, ext2 3 , or whatever file system and does not respect file security or user security. All that stuff is "locked" by the operating system normally running on that computer. The files themselves have no lock, in the sense that a harddisk will not prohibit you from reading writing anything. The locking mechanism is at operating system level. The OS verifies user identity and checks the file permissions before it allows read/write of a file. So if i have a customized OS that answers "YES" to any permission question i can go anywhere.

There should be a superuser OS that requires no login and does not check nor respect any lock bits or systems. Boot from such a disk , or attach a drive to such a system and it's all open. ideally the system should have a "read only" mode as well.

There used to be a tool called NTFS4DOS. You boot the computer from a floppy disk into a DOS installation and launch that driver. Go to the harddisk and you can read/write/copy anything and everything. The tool simply ignored the security mechanisms of NTFS.

i don't need it to be able to reset or alter security settings. It's not a password cracker or permission changer, you need physical access to the drive and the normal OS is not running on the computer. This shell just does not implement the security mechanisms and doesn't verify identity or permissions. File structures are known so we can traverse the drive and read. we skip the whole security verification and just read the file and copy it to external media. doesn't even need to be the same file system. the bootmedia can be something simple as FAT or FAT32. a system that has no security.  or we delete the file. i don't even need to be able to write to the drive under inspection (no overwriting of existing files or adding files). just be able to copy things from it elsewhere ( different media ) or delete things off it.

i frequently run into situations where i need to get something off an old machine or recover from a problem (infiniium or tek oscilloscopes) and have an arsenal of tools like windows password crack boot cd's and whatnot. but it would be easier to just have a boot image of a customised tool that just lets me read and copy files from a drive and ignores the user/file permissions. no need to crack anything. boot from the tool , grab the files needed (license key files, drivers etc) , format the disk and do a clean install , place the required files back. and you have a factory fresh machine with all the accumulated cruft gone.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: alm on August 19, 2022, 05:37:36 pm
Indeed, file system permissions provide no security outside the original operating system.

Any Linux distribution that can boot from a USB stick should be able to do what you ask. For example SystemRescue (https://www.system-rescue.org/).
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: IanB on August 19, 2022, 06:00:41 pm
any file system any os.

I'm not sure I understand the problem? Remove the drive from the original system, put in a drive caddy and plug it into another system, and you can read the data. The only way to protect drive data from this scenario is to encrypt the drive. Thieves and cybercriminals exploit this weakness all the time.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: free_electron on August 19, 2022, 06:48:10 pm
that doesn't work on a windows machine : can't ready foreign file systems and if the drive is ntfs it wants to enforce the permissions.
i'm looking for something simple to use . it can be linux based provided that i don't need to muck with command line tools.
it can run in 640x480 if it needs to, as long as i can use a mouse and have a file explorer with multiple windows and can do drag and drop copy operations . preferably has a read-only mode for the drive being worked on. starts in read-only mode and can be switched to editing mode. i don't want to mess up the drive i am working on. the goal is to take data off the drive, not edit the drive (apart from volume resizing.. one of the issues i run into is cloning a drive to a slightly smaller harddisk. This is becoming more and more a problem when cloning to SSD's. a 32Gig SSD is smaller than a 30Gig HDD ... i was trying to migrate an infiniium scope from a 30Gb IBM travelstar to 32Gbyte SSD. cloning fails. the SSD is just a hair too small.. had to switch to a 64Gbyte SSD ... i did not want to resize the master drive as it is pristine ( never booted so it is not locked to the hardware id's yet ). cloning those things is problematic as it has a system recovery partition that includes a norton ghost installation. most cloning programs don't know what to do with it. macrium can handle it. there is also the tektronix scopes with a vxworks partition next to windows 2000 which are problematic to clone.

- clone drive (including all partitions , boot managers etc)
- resize volume ( shave off a few gig of empty space)
- read random files and copy to different target drive
- source drive in read-only mode by default. needs deliberate mode switch to make drive writeable
- graphical user interface ( can be "text mode" like norton commander) that allows me to do things without needing to resort to cryptical command lines with option flags.

nice to have
- backup volume as is , warts and all. (skip read errors)
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: IanB on August 19, 2022, 06:54:51 pm
that doesn't work on a windows machine : can't ready foreign file systems and if the drive is ntfs it wants to enforce the permissions.

I guess I've never encountered that problem  :-//
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: J-R on August 19, 2022, 07:52:32 pm
When Windows mounts another system's NTFS file system, it does 'respect' the existing permissions, just like Linux, etc.  However, if you're logged in as an administrator on the OS, then you can simply seize ownership of the affected files/folders at which point you can change the permissions as necessary so you can access them.  The equivalent process in Linux might be 'sudo chown' and 'sudo chmod' for example.

In many cases, when simply trying to access the folder via an admin account, Windows Explorer will helpfully offer to do this automatically in one step.

Macrium is a good product if your world is Windows-centric.  You can mount backup images easily and it has a PE boot environment for easy recovery.  I've loaded the free version on dozens of machines over the years and it's been a good experience.  One favorite feature is that it can capture any necessary storage drivers from the Windows OS and place that on the USB recovery stick.  Makes offline cloning a lot easier during OS drive swaps, etc.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: Nominal Animal on August 19, 2022, 08:17:50 pm
When Windows mounts another system's NTFS file system, it does 'respect' the existing permissions, just like Linux, etc.  However, if you're logged in as an administrator on the OS, then you can simply seize ownership of the affected files/folders at which point you can change the permissions as necessary so you can access them.  The equivalent process in Linux might be 'sudo chown' and 'sudo chmod' for example.
Linux also allows one to mount the filesystem with an arbitrary mapping between NTFS users and groups and their Linux counterparts, with uid= and gid= as shorthand for mapping them all to one user and one group.  By default, the silent option is used on NTFS mounts, which causes the 'sudo chown' and 'sudo chmod' to silently do nothing on such mounts, unless an arbitrary mapping has been set.  Also, the actual default is to map all users and groups to the current user.  (If you do do 'sudo mount', the current user will be root, so then specifying that explicitly via '-o uid=$(id -un),gid=$(id -gn)' is needed.)

This means that if you just open an NTFS drive in Linux, 'sudo chown' and 'sudo chmod' are supposed to have no effect by default.

When you plug in a drive, and it gets mounted automatically, a default set of mount options is used.
Unfortunately, I do not know of any graphical tool that allows one to set the mount options.  There likely is one; I just don't know one. :(

I can only tell how to do it on the command line:
    sudo mount -o uid=$(id -un),gid=$(id -gn),umask=0,fmask=0,dmask=0,silent,ro volume directory
where volume is the device or partition containing the filesystem (under /dev/), and directory is where to mount to.
This mounts the NTFS volume in read-only mode, so that no changes are made to it at all.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: JohanH on August 19, 2022, 09:08:45 pm
i ran into a situation on an old computer of mine where i had two logins. i was admin on both. i wanted to delete one of them and no matter what it tried i could not delete one directory. you need admin permission. i am admin. access denied ... it was a google drive shadow folder. i finally found some obscure command line operation that could reset the flags so i could access the files.

If a file is set to immutable, you can't erase or modify it as superuser. But you can of course change the file attributes with chattr and make it non-immutable. Don't know if that was the case here, but it's something that could be overlooked sometimes. There are also ACL (Access Control Lists), where you can specify attributes and ownership for multiple groups and users (getfacl and setfacl commands). ACL are commonly used in multi-user environments, such as file servers.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: m k on August 19, 2022, 09:09:25 pm
Sector by sector copy worked just fine.

Agilent has removed all software/firmware updates for this unit from the site. So if the HDD fails, that is it, the unit becomes unusable. I copied into an IDE SSD and the instrument is now safe!

Back in the day when floppies and tapes were regular stuff I had some world's only backups.
Final set was in a bank vault in multiple copies, in hand backup was also in multiple copies.
Idea was that even if all copies were bad there were still a possible good one if all bad ones were combined.
Data format was also easy since it was source codes and so practically text.

Things have changed from those days but do not take "safe" too lightly.
Maybe one day your SD card is gone and only thing left is a peanut.
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: JohanH on August 19, 2022, 09:11:32 pm
Sector by sector copy worked just fine.

Agilent has removed all software/firmware updates for this unit from the site. So if the HDD fails, that is it, the unit becomes unusable. I copied into an IDE SSD and the instrument is now safe!

I would definitely also have copied it into a (gzipped) file and uploaded to a backup PC or the cloud. Hey, why not to archive.org and do the community a favor.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: free_electron on August 19, 2022, 09:40:18 pm
then you can simply seize ownership of the affected files/folders at which point you can change the permissions as necessary so you can access them.
And in that lies the problem. i do NOT want to modify the file system or permissions ! Just read the file without altering anything on the drive. it should leave no traces. the drive needs to be treated as read-only.

there are forensic tools that do exactly that. unfortunately very expensive and out of reach for my tinkering restoring test equipment
Title: Re: Anyone knows how to clone a HDD without any partitions?
Post by: edpalmer42 on August 19, 2022, 09:42:33 pm
Agilent has removed all software/firmware updates for this unit from the site. So if the HDD fails, that is it, the unit becomes unusable. I copied into an IDE SSD and the instrument is now safe!

Does the Wayback Machine have the downloads?  It's definitely hit-and-miss, but I've often used it to get drivers, documentation, etc. when the site or file has vanished.

Ed
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: IanB on August 19, 2022, 09:54:21 pm
And in that lies the problem. i do NOT want to modify the file system or permissions ! Just read the file without altering anything on the drive. it should leave no traces. the drive needs to be treated as read-only.

But the default Windows file permissions on NTFS allow full control to Administrators and the System account. So it would have to be a special case where you couldn't read the disk without modifying permissions.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: paul_g_787 on August 19, 2022, 09:54:49 pm
Clonezilla has an advanced option to do a bit for bit copy if I remember rightly. It literally copies 1s and 0s. I would guess it may even use dd under the hood.

I used this option some years ago to clone a dying PS3 120GB HDD to another disk without losing the encrypted FS and MBR.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: J-R on August 19, 2022, 11:37:33 pm
Clonezilla absolutely does have a dd imaging option, but at that point might as well just use dd from your favorite Linux live USB stick.


For NTFS, lots of imaging programs allow browsing a backup via the Windows Explorer.  I use Macrium for this quite often.  It has an image mount option that does basically ignore all NTFS permissions on the source and allows you full access of all the files in the backup image.

I don't think this helps if your source drive is not NTFS.

User directories are particularly common to have a different username set for the permissions, so when you place it in another computer you won't have access directly, even as an admin.
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: Nominal Animal on August 20, 2022, 06:37:18 am
i do NOT want to modify the file system or permissions ! Just read the file without altering anything on the drive. it should leave no traces. the drive needs to be treated as read-only.
Clone it first, and do your alterations and examinations on a read-write copy?

(Again, while mounting it read-only and giving you full access override can be done in a single command line command in Linux (the one in #44 (https://www.eevblog.com/forum/repair/anyone-knows-how-to-clone-a-hdd-without-any-partitions/msg4368541/#msg4368541)), I do not know of any proper UI tools that do it.  And I definitely recommend *against* having someone write a script or GUI for it.  This kind of tool has to be rock solid and reliable.)
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: free_electron on August 20, 2022, 12:56:36 pm
that's what i do now. i have a hardware drive cloner. (startech) plug in two drives (source and target) and hit one button. target needs to be larger or equal to source.
it takes a while though , but it copies anything. the cloner is ide so i need to use ide to sata adapters to go to ssd or modern drives but it works.
then work off the clone.

it would be nice to be able to ditch all that stuff. have a bootable usb stick with a drive imaging tool that can save the drive image , and a file explorer where i can quickly grab some stuff and copy it off. something i can drop on ventoy (ventoy is a special usb bootable that lets you launch iso files. simply write a collection of iso files to it and at boot you can select which one needs to launch. i keep all install medium for various os's on there. so an iso that contains what i need (cloner/imager/file access) would be great.

if i need to redeploy : access with file grabber , copy the two or three files i need , reboot from install iso , install , copy files back. no need to remove drives , hook up cables. simply boot from usb
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: Ed.Kloonk on August 20, 2022, 12:59:11 pm
it would be nice to be able to ditch all that stuff. have a bootable usb stick with a drive imaging tool that can save the drive image , and a file explorer where i can quickly grab some stuff and copy it off.

What's stopping you?
Title: Re: Anyone knows how to clone a HDD without any partitions? [DONE!]
Post by: free_electron on August 20, 2022, 04:32:05 pm
it would be nice to be able to ditch all that stuff. have a bootable usb stick with a drive imaging tool that can save the drive image , and a file explorer where i can quickly grab some stuff and copy it off.

What's stopping you?
what's stopping me is finding a ready made iso that can do that !