Difference between revisions of "Guides:MAME - chdman"
Latest revision as of 05:15, 10 February 2020
Compressed Hunks of Data Manager
A (C)ompressed (H)unks of (D)ata file or CHD for short is a standard format developed by the MAME project for storing hunks of data.
The format was originally developed to image later 200x arcade machines that had begun shipping with internal hard drives. Beyond that some arcade hardware systems also began shipping with interchangeable discs (e.g. NAOMI) and a format was also needed to capture these.
If you have been around from the late 90s or have grabbed dumps from around that time you will be aware of how many different disc image formats there are. Different formats took hold for short periods depending n their features set. Some example historical formats are mdf, nrg, img, iso and bin. Emulator authors then had the decision to add support for all these formats or provide a limited number of formats and force users to convert from one format to another during use. In some cases not all the data can be converted between formats.
CHD is expected to address this by generalizing the data storage format providing a solution for both of the above scenarios and more. The CHD format itself has undergone multiple iterations to add support for various media features since its release. The current chd hashes registered in MAME 0.217 are format version 5.
We can take a look at the supported image formats for a couple of systems by querying MAME with -listmedia. Let's look at a couple of consoles to start with that have cdrom drives
Sony PlayStation (USA)
$ ./mame64 psu -listmedia SYSTEM MEDIA NAME (brief) IMAGE FILE EXTENSIONS SUPPORTED ---------------- --------------------------- ------------------------------- psu memcard1 (memc1) .mc memcard2 (memc2) .mc quickload (quik) .cpe .exe .psf .psx cdrom (cdrm) .chd .cue .toc .nrg .gdi .iso .cdr
Sega Saturn (USA)
$ ./mame64 saturn -listmedia SYSTEM MEDIA NAME (brief) IMAGE FILE EXTENSIONS SUPPORTED ---------------- --------------------------- ------------------------------- saturn cdrom (cdrm) .chd .cue .toc .nrg .gdi .iso .cdr cartridge (cart) .bin
You'll notice neither of these systems support mdf/mds format. The mdf/mds format was developed for use with a very popular tool during the early 2000s called Alcohol 120%.
Another popular tool of the early 2000's CloneCD was the only tool for a very long time to properly support sub channel data dumping and became popular with groups dumping pc titles. The deafult format for CloneCD was ccd/img or ccd/img/sub when sub channel data was dumped. Again looking at our MAME output these images are not supported in either system above although dumps exist.
You will notice the nrg format is in the list of compatible file extensions, this was a proprietary format from the tool Nero Burning ROM, it is interesting that it is supported and we will need to find out why and update information here.
We can see that chd is supported on both systems, so if we can convert our discs to chd we will remain compatible with MAME and both these systems therein.
We mentioned previously chdman was created to be used as a tool to image hard disks originally. chdman can also create blank chd images for use with machines where a hdd may be managed by an Operating System. Lets take a look at the ct486, a 486SX/25 Mhz Personal Computer emulated in MAME.
Start by having a look at the output of -listmedia for this machine.
PC/AT 486 with CS4031 chipset
$ ./mame64 ct496 -listmedia SYSTEM MEDIA NAME (brief) IMAGE FILE EXTENSIONS SUPPORTED ---------------- --------------------------- ------------------------------- ct486 floppydisk1 (flop1) .dsk .ima .img .ufi .360 .d77 .d88 .1dd .dfi .hfe .imd .ipf .mfi .mfm .td0 .cqm .cqi floppydisk2 (flop2) .dsk .ima .img .ufi .360 .d77 .d88 .1dd .dfi .hfe .imd .ipf .mfi .mfm .td0 .cqm .cqi harddisk (hard) .chd .hd .hdv .2mg .hdi printout (prin) .prn
The harddisk option does have the ability to attach a chd file to it along with a number of other formats but we will focus on chd in this document.
Once the machine is started with the disk attached using -hard <chdfile.chd> the chd will be attached on the internal machine on ide0.
The tool itself is called chdman depending on how you installed it, it may be accessible by typing chdman into your terminal. I compile MAME to custom location so I will need to execute chdman from the current directory by including a ./ before the command. Substitute as necessary for your own system.
Firstly lets take a look at the usage summary for chdman
$ ./chdman -help chdman - MAME Compressed Hunks of Data (CHD) manager 0.217 (mame0217-611-ga7fec751ed) Usage: ./chdman info: displays information about a CHD ./chdman verify: verifies a CHD's integrity ./chdman createraw: create a raw CHD from the input file ./chdman createhd: create a hard disk CHD from the input file ./chdman createcd: create a CD CHD from the input file ./chdman createld: create a laserdisc CHD from the input file ./chdman extractraw: extract raw file from a CHD input file ./chdman extracthd: extract raw hard disk file from a CHD input file ./chdman extractcd: extract CD file from a CHD input file ./chdman extractld: extract laserdisc AVI from a CHD input file ./chdman copy: copy data from one CHD to another of the same type ./chdman addmeta: add metadata to the CHD ./chdman delmeta: remove metadata from the CHD ./chdman dumpmeta: dump metadata from the CHD to stdout or to a file ./chdman listtemplates: list hard disk templates For help with any command, run: ./chdman help <command>
The first thing you may note is the version number, this build of MAME was compiled around the time of the MAME 0.218 code freeze. Your version will differ.
chdman - MAME Compressed Hunks of Data (CHD) manager 0.217 (mame0217-611-ga7fec751ed)
After reviewing the version, you can take a look at the rest of the avialable options. At the bottom of all the options chdman tells us we can get further help on each of these commands.
For help with any command, run: ./chdman help <command>
So lets get help on the verify command to see how we would use it to confirm the chd image we may want to use is valid for this version of MAME.
Note: CHDs may change between versions of MAME, they may be redumped or recreated or copied to bring the original data up to a new version of the format.
$ ./chdman help verify chdman - MAME Compressed Hunks of Data (CHD) manager 0.217 (mame0217-611-ga7fec751ed) Usage: ./chdman verify [options], where valid options are: --input, -i <filename>: input file name (required) --inputparent, -ip <filename>: parent file name for input CHD
We wont cover parent CHDs (--inputparent) yet, but we will verify a standard CHD in the next test. You will need to obtain a CHD (upload a test chd here slacker) to use next
$ ./chdman verify --input <chdfile.chd> chdman - MAME Compressed Hunks of Data (CHD) manager 0.217 (mame0217-611-ga7fec751ed) Raw SHA1 verification successful! Overall SHA1 verification successful!
It is worth mentioning here that CHDs must be told to be --compressed when they are created. The default create action for chdman (ignoring the name) is to build an uncompressed chd file. These cannot be verified because the verification is of the compressed vs raw data. Uncompressed CHDs are only raw data. There are other property differences between compressed and uncompressed chds we will cover later.
If you attempt to run a verify on an uncompressed image, MAME will let you know there was nothing to do.
$ ./chdman verify --input <uncompressed.chd> chdman - MAME Compressed Hunks of Data (CHD) manager 0.217 (mame0217-611-ga7fec751ed) No verification to be done; CHD is uncompressed Fatal error occurred: 0
We can also query information about the CHD as it was stored as metadata during creation.
$ ./chdman info --input chdfile.chd chdman - MAME Compressed Hunks of Data (CHD) manager 0.217 (mame0217-611-ga7fec751ed) Input file: chdfile.chd File Version: 5 Logical size: 104,857,600 bytes Hunk Size: 4,096 bytes Total Hunks: 25,600 Unit Size: 512 bytes Total Units: 204,800 Compression: none CHD size: 102,574 bytes Metadata: Tag='GDDD' Index=0 Length=34 bytes CYLS:256,HEADS:16,SECS:50,BPS:512.
Here we can see the metadata stored with the CHD data, it lists information we can use for setting up a machine and info the state of the CHD.
For example let's say this chd was being attached to the ct486 machine in MAME. The line CYLS:256,HEADS:16,SECS:50,BPS:512 can be used to configure the HDD in the bios. There is no HDD autodetect mechanism in the bios on the ct486 machine, it must be manually configured.
Converting between versions is completed with the copy functionality. You do not extract to raw then createhd/cd etc generally, it should be fine but if you forget to pass the --compress parameter the resulting CHD will be invalid. The copy function maintains the original condition of the chd and copies it to a new file. You can understand how to use this function but requesting help on it.
$ ./chdman help copy chdman - MAME Compressed Hunks of Data (CHD) manager 0.217 (mame0217-611-ga7fec751ed) Usage: ./chdman copy [options], where valid options are: --output, -o <filename>: output file name (required) --outputparent, -op <filename>: parent file name for output CHD --force, -f: force overwriting an existing file --input, -i <filename>: input file name (required) --inputparent, -ip <filename>: parent file name for input CHD --inputstartbyte, -isb <offset>: starting byte offset within the input --inputstarthunk, -ish <offset>: starting hunk offset within the input --inputbytes, -ib <length>: effective length of input in bytes --inputhunks, -ih <length>: effective length of input in hunks --hunksize, -hs <bytes>: size of each hunk, in bytes --compression, -c <none|type1[,type2[,...]]>: which compression codecs to use (up to 4) --numprocessors, -np <processors>: limit the number of processors to use during compression
For Linux systems chdman ships with a man page, this includes a lot more information on the tool and should be reviewed as a starting point for any operation. You will have this available with the packaged version fropm your distro. If you compile it yourself you will need to generate and install the man pages which is outside the scope of this document.
$ man chdman CHDMAN(1) MAME Compressed Hunks of Data (CHD) manager CHDMAN(1) NAME chdman - MAME Compressed Hunks of Data (CHD) manager SYNOPSIS chdman option DESCRIPTION chdman is the (C)ompressed (H)unks of (D)ata (CHD) manager for MAME. It is a tool to manage CHD images.
You can also find the man page on self compiled version of MAME in the docs/man/ folder and read it with the man command
$ man docs/man/chdman.1