hdfmonkey - a Swiss Army Knife for HDF disk images
This is a spin-off from my recent experiments on ESXDOS. If you want to play with the DivIDE (or other Speccy hard disk interfaces) under emulation, you'll be working with HDF files, which are images of hard disks in much the same way that TZXs and DSKs are for tapes and disks. The problem is, HDF files are a bit of a black box, with no good way to get at the individual files contained within them... there are utilities around for copying them to and from physical disks, but you really don't want to be doing that every time you want to get at an individual file. (And it's particularly annoying if you're trying to develop software for something like ESXDOS under emulation...)
Which is where hdfmonkey comes in. If you have an HDF image of a FAT-formatted disk (which is what all good firmwares for Speccy hard disk interfaces will be supporting for the forseeable future), and you want to get wibble.txt off it, you can do this:
hdfmonkey get myimage.hdf wibble.txt ./wibble.txt
And if you have a wonderful new game as a .tap file that you want to copy to the /games/ directory of your disk so that you can load it in via ESXDOS, FATware or ResiDOS, you can do this:
hdfmonkey put myimage.hdf wonderful_new_game.tap /games/wonderful_new_game.tap
Likewise, there are commands for getting directory listings, creating directories, and deleting files/directories. In theory you can even format the disk with a freshly minted FAT filesystem, but neither ESXDOS nor FATware recognise it at the moment and I'm not sure why. One to be fixed in a later version, hopefully...
Source code (should be portable with no weird dependencies): ftp://ftp.untergrund.net/users/gasman/zx/hdfmonkey/hdfmonkey-0.1.tar.gz
Mac OS X binary: ftp://ftp.untergrund.net/users/gasman/zx/hdfmonkey/hdfmonkey-0.1-osx.zip
Development site: http://github.com/gasman/hdfmonkey
Something that would be a really neat addition (and probably not a huge amount of work to add at this point) would be a FUSE / MacFUSE extension so that you can mount the disk and work with it directly using ordinary file commands. That's not something I'll be immediately rushing to do though, because for now it does everything I want :-)
Which is where hdfmonkey comes in. If you have an HDF image of a FAT-formatted disk (which is what all good firmwares for Speccy hard disk interfaces will be supporting for the forseeable future), and you want to get wibble.txt off it, you can do this:
hdfmonkey get myimage.hdf wibble.txt ./wibble.txt
And if you have a wonderful new game as a .tap file that you want to copy to the /games/ directory of your disk so that you can load it in via ESXDOS, FATware or ResiDOS, you can do this:
hdfmonkey put myimage.hdf wonderful_new_game.tap /games/wonderful_new_game.tap
Likewise, there are commands for getting directory listings, creating directories, and deleting files/directories. In theory you can even format the disk with a freshly minted FAT filesystem, but neither ESXDOS nor FATware recognise it at the moment and I'm not sure why. One to be fixed in a later version, hopefully...
Source code (should be portable with no weird dependencies): ftp://ftp.untergrund.net/users/gasman/zx/hdfmonkey/hdfmonkey-0.1.tar.gz
Mac OS X binary: ftp://ftp.untergrund.net/users/gasman/zx/hdfmonkey/hdfmonkey-0.1-osx.zip
Development site: http://github.com/gasman/hdfmonkey
Something that would be a really neat addition (and probably not a huge amount of work to add at this point) would be a FUSE / MacFUSE extension so that you can mount the disk and work with it directly using ordinary file commands. That's not something I'll be immediately rushing to do though, because for now it does everything I want :-)
Post edited by gasman on
Comments
Source code: ftp://ftp.untergrund.net/users/gasman/zx/hdfmonkey/hdfmonkey-0.2.tar.gz
Mac OS X binary: ftp://ftp.untergrund.net/users/gasman/zx/hdfmonkey/hdfmonkey-0.2-osx.zip
Development site: is still http://github.com/gasman/hdfmonkey
Thanks to some sterling fault-finding work by Phoenix, this version fixes the problem with the 'format' command where ESXDOS and FATware wouldn't recognise the FAT partition. I've also added new commands 'create' (create a new FAT-formatted HDF or raw disk image of any size) and 'clone' (copy an existing real/virtual disk to HDF, like the raw2hdf program in fuse-utils - but this one goes in the other direction too), improved the 'put' command so that you can copy multiple files and directories at once, and added the ability to specify a volume label on format/create.
As an example, the following steps will create a 32Mb HDF image with volume label 'SPECCY' initialised with the ESXDOS system files (assuming you have them in ~/Development/esxdos). No more faffing around with actual CF card readers just to get something you can use in an emulator! :-)
Did you forget to format the hdf file?
hdfmonkey format speccy.hdf
By the way, I compiled it with mingw, I had to comment out some permission constants like S_*. Also Gmtime_r is not supported with mingw, and I gave a fixed date, so all files will be written as 10.10.2010 10:10:10. :)
Here is the windows version, warning, I didn't check it throughly:
http://arda.kisafilm.org/fish/hdfmonkey02_quickcompile.zip
Nope - formatting as FAT is done as part of the 'create' action (or at least, it should be...)
Thanks for the Windows port, much appreciated!
oh ok, then there may be a problem with windows port :D it doesn't work without formatting first.
This looks to be exactly what I was looking for...except I can't get it to work.
I'm using Windows 7 and the port provided by Arda.
I've done the following which creates a blank hdf that is visible in Fuse and ZXSpin
hdfmonkey create speccy.hdf 32M speccy
But, when I try to use the "put" command I get an error:
read() error. line: 23
Error opening file: Low-level disk error
Any help would be greatly appreciated...
Gareth
[EDIT] Here it is hdfmonkey-0.2 for windows, compiled with cygwin instead of mingw32. Seems to have better compatibility because "create" command format the file and fuse recognize it.
Thanks - Fuse and ZXSpin both recognise and can read it now. Put even seems to put a file on there but if I try to load them using FATWare I get the loading screen then a tape loading error. Very realistic but not quite what I wanted!!
Gareth
After create image HDF with other size e.g 16MB (with this same TAP file) is no errors and loading OK.
Maybe hdfmonkey working only with 16x multiple sizes e.g. 16, 32, 64 etc. ???
Use in case your 3.5 floppy breaks!
Last time I try make HDF image with Muppet.dvo + player.tap for use with Fuse emu. File size Muppet.dvo is 135MB so I need make HDF some more size to add file player.tap e.g 138MB and this not working :-(. After create HDF 140MB is OK and working.
That may be a particularly relevant answer in fact... :-)
Any image under 63Mb in size will be created as FAT12 (the type usually found on floppies) rather than FAT16. It seems that Fatware only officially supports FAT16 - apparently FAT12 does work to some extent, but that might just be a fluke. I'll have to do some experimenting to confirm whether this is indeed the problem - if so, it would probably make sense for hdfmonkey to have a command line switch to force the use of FAT16.
*ahem*
How very dare you!? ;)
oh wait, for the forseeable future - you're probably right there. :)
Incidentally my firmware will support FAT just not on an IBM format disk :)
- It should be a *lot* faster when copying large files now. (Copying outline.dvo to an .hdf image now takes 1.3 seconds on my Macbook, rather than 7 and a half minutes :-) ) For some reason, when I opened the image files, I previously had the "make all file access excruciatingly slow" (otherwise known as O_SYNC) flag set, and I have no idea why. If any Unix gurus want to pipe up and say "actually, removing O_SYNC set is a really bad idea", now is the time...
- There's a new command, as previously hinted here: which does a file-by-file copy of all the things on oldfile.hdf onto newfile.hdf, which should ensure that everything on newfile.hdf is unfragmented.
- It now handles read-only media correctly.
http://www.mediafire.com/?a4gcnliw1uuiq7s
and https://github.com/gasman/hdfmonkey
it's now possible to override the automatically-chosen format with a switch.
- There's a new command:
hdfmonkey create <--fat12> <--fat16> <--fat32> <imagefile> <size> [volumelabel]
Could please a build be created for Windows?
(with cygwin1.dll and cyggcc_s-1.dll)
Windows build:
http://www.mediafire.com/file/ah6emm5416n644n/hdfmonkey-win32-0.4.zip
The 04 is experimental.
Since so far the v03 also works very well, but still cygwin1.dll and cyggcc_s-1.dll needs, would it be nice if the time allowed it to compile the version 0.3 also once again to Cross, please?
Trying to read beyond mmc. Size: 33554432 Asked: 402759680. Disabling MMC
(see Screenshot).
No change when I use --fat16 switch when formatting the 32MB image.
When I use a 64MB image, there´s no error.
When I use ImDisk Virtual Disk Driver from ltr-data.se/opencode.html/#ImDisk to create and format the 32MB Image, there´s no error.
So I think it must be an issue with format from hdfmonkey
This option is experimental and error prone.
There seem to be a rename function in "FatFs - FAT file system module R0.07e" sourcecode at: https://github.com/gasman/hdfmonkey/blob/master/src/ff.c but neither HDFmonkey.EXE v0.3 nor v0.4 (Windows Version) use it? (Looking into v0.3 binary I see text "rename", but it don´t seem to be implemented?
Binary:
http://www.mediafire.com/file/t9272cx99c4bkxa/hdfmonkey-win32-0.4-wip20171230.zip
Source:
https://github.com/sbaldovi/hdfmonkey/commit/b7c83859aa6f83f067c535336d6d56af24d7a0fa
One thing which might be bettered is to rename message "Move (rename) failed: File not found" to "Move (rename) failed: File or Directory not found"