# checksum tricks and tipshints, secrets, behaviours, assumptions and more..

## Get the most from your hashing!

checksum represents a whole new way of working with hashes. This page aims to help you get the most out of the experience, wherever you're at..

## Absolute beginners..

The basics: checksum creates "hash files". A hash file is a simple, plain text file containing one or more file hashes, aka. "checksums". Hashes are small strings which uniquely represent the data that was hashed. e.g..

cf88430390b98416d1fb415baa494dce *08. Allow Your Mind To Wander.mp3

(Mike Mainieri - Journey Thru An Electric Tube [1968] - I have the vinyl)

If you want to know more about the algorithms that checksum uses to hash files (MD5, SHA1 and BLAKE2), see here.

Once these hashes have been created for a particular file or folder (or disk), you have a snapshot that can be used, at any time in the future, to verify that not one bit of data has changed. And I do mean a "bit"; even the slightest change in the data will, thanks to the avalanche effect, produce a wildly different hash, which is what makes these algorithms so good for data verification.

Most people will simply install checksum, and then use the Explorer context (right-click) menu to create and verify checksums, rarely needing any of the "extra" functionality that lurks beneath checksum's simple exterior. After all, checksum is designed to save you time, as well as aid peace of mind. This is how I mostly use it, too..

### Create checksums..

Right-click a file, the checksum option produces a hash file (aka. 'checksum file') with the same name as the file you clicked, except with a .hash extension (or .md5/.sha1, if you use those, instead). So a checksum of some-movie.avi would be created, named some-movie.hash (if you don't use the unified .hash extension, your file would instead be named some-movie.md5 or some-movie.sha1, depending on the algorithm used).

Right-click a folder, the Create checksums.. option will produce a hash file in that folder, containing checksums for all the files in the folder (and so on, inside any interior folders), named after the folder(s), again, with a .hash extension, e.g. somefolder.hash

### Verify checksums..

Click (left-click) a hash file (or right-click and choose Verify this checksum file..), checksum immediately verifies all the hashes (.hash/.md5/.sha1) contained within.

Right-click a folder, the Verify checksums.. option instructs checksum to scan the directory and immediately verify any hash files contained within.

That's about it, and this simple usage is fine for most situation. But occasionally we need more..

## checksum launch modifiers..

When you launch checksum, you can modify its default behaviour in two important ways.

The first modifier is the <SHIFT> key. Hold it down when checksum launches, and you pop-up the one-shot options dialog, which enables you to change lots of other things about what checksum does next. This works with both create and verify tasks, from explorer menus or drag-and-drop. Here's what the one-shot create dialog looks like..

In there, as you can see, you can set all sorts of things. Hover over any control to get a Help ToolTip (you might need to repeat that to read the entire tip!). You can also drag files and folders directly onto that dialog, if you want to alter the path setting without typing. Same for the verify options.

The file mask: input is, by default, *.*, which means "All files", "*" being a wildcard, which matches any number of any characters. You can have multiple types, too, separated by commas. For example, if you wanted to hash only PNG files, you would use *.png; if you wanted to hash only text files beginning with "2008", you could use 2008*.txt, and so on.

If you click the drop-down button to the right of the input, you can access your pre-defined file groups, ready-for-use (you can easily add to/edit these in your checksum.ini)..

NOTE: Normally one drops folders into checksum's create options (path input). If you drop a file onto the create options, the path is inserted into the path: input, and the file name is added to the mask input - it is also inserted into the drop-down lost in case you need to get back to it.

If you drop multiple files into the create options, checksum will create a custom file mask from your selection. For example, if you dropped these three files; "hasher.jpg", ".txt" and "security.pdf", checksum would create the mask: "*.jpg,*.txt,*.pdf".

Here is what the one-shot verify options dialog looks like..

The second modifier is the <Ctrl> key. Hold it down when checksum launches and you force checksum into verify mode, that is to say, no matter what type of file it was, you instruct checksum to treat it as a hash file, and verify it. This works with drag-and-drop too, onto checksum itself, or a shortcut to checksum. checksum's default drag-and-drop action is to create hashes.

Amongst other things, this is useful for verifying folders in portable mode, simply Ctrl+drag-and-drop the folder directly onto checksum (or a shortcut to it), and all its hashes will be immediately verified.

Hit the modifier key as soon as checksum launches, in other words, hit the <SHIFT>/<Ctrl> key right after you choose the Explorer menu item, or before you let go of a drag & drop, and so on. Hold the key down until checksum appears a moment later.

## batch processing..

### hashDROPA batch-processing front-end for checksum..

Because checksum can be controlled by command-line switches, it's possible to create all sorts of interesting front-ends for it. The first of these to come to my attention, is a neat wee application called "hashDROP", which enables you to run big batches of jobs through checksum, using a single set of customizable command-line switches.

As developer seVen explains on the hashDROP page..

hashDROP is a front-end for checksum which enables you to queue a bunch of jobs (files/folders) and then pass them all through checksum with your own custom switches in one batch process.

On seVen's desktop, at least, it looks something like this..

### Batch RunnerRun multiple programs in a batch..

I originally designed Batch Runner to run a big batch of tests on checksum before release, but it has since proven useful for other tasks, so I spruced it up a bit, made it available.

If you want to run loads of hashing jobs using the same switches, hashDROP is probably more useful to you. But if you want to run lots of checksum jobs with different switches, or as part of a larger batch of jobs involving other programs, then check out Batch Runner.

Batches can be saved, selected from a drop-down, run from the command-line, even from inside other batches, so it's handy for repetitive scheduled tasks, or application test suites, as well as general batch duties. At least on my desktop, it looks like this..

## Automatic Music playlists..

Perhaps checksum's second most common extra usage is making music playlists. After you have ripped an album, you will most likely want a playlist along with your checksums, so why not do both at once? checksum can.

Right-click a folder and SHIFT-Select the checksum option (which pops up the one-shot options dialog), check either the Winamp playlists (.m3u/.m3u8) or shoutcast playlists (.pls) option, and then do it now! You're done.

By default, checksum will also recurse (dig) into other folders inside the root (top) folder. Now you've got music playlists that you can click to play the whole album in your media player.

Note that checksum will thoughtfully switch your file masks to your current music group when you select a playlists option, reckoning that you'll probably only want to actually hash the music files, not associated images, info files and such, but it's easy enough to switch it back to *.* (hash all files) if you need that. The rationale behind this being that it's what most people want, so the majority get the simpler, two-click task.

If you do this sort of thing a lot, check out the next section, for how to put this functionality directly into your Explorer context menu, and skip the dialog altogether..

On the subject of music files, you may encounter a lot of these, and fancy creating a custom explorer right-click command along the lines of "checksum all music files", or something like that. No problem; you can simply create a new command in the registry, add the "m" switch add your file masks, right?

But what if you change your file masks? Perhaps add a new music file type? Do you have to go and change your registry again? NO! checksum has it covered. Instead of specifying individual file masks, use your group name in the command, e.g. m(music) and checksum applies all the file masks from that group automatically, so your concept command is always up-to-date with your latest preferences.

Here's an example registry command that would do exactly that. Copy and paste into an empty plain text file, save as something.reg, and merge it into your registry. If you installed checksum in a different location, edit the path to checksum before you merge it into the registry (not forgetting to escape all path backslashes - in other words, double them)..

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\01b.checksum music]
@="Checksum &MUSIC files.."

[HKEY_CLASSES_ROOT\Directory\shell\01b.checksum music\command]
@="\"C:\\Program Files\\corz\\checksum\\checksum.exe\" crm(music) \"%1\""


NOTE! If you add a "3" to the switches [i.e. make them c3rm(music)] you'll get a media player album playlist files created automatically along with the checksum files. Groovy! Here's one I prepared earlier.

### Setting new default Explorer context actions..

You can also change checksum's default Explorer commands, as well as add new commands, without going anywhere near the registry. Simply edit the installer's setup.ini file, [keys] section. For example, to always bring up the one-shot options dialog when creating checksums on folders and drives, you would add an "o" to those two commands..

HKCR\Directory\shell\01.|name|\command="|InstalledApp|" cor "%1" HKCR\Drive\shell\01.|name|\command="|InstalledApp|" cor "%1" Then run checksum's installer (setup.exe), and install/reinstall checksum with the new options. From then on, any time you select the "Create checksums.." Explorer context menu item, you will get the one-shot options dialog. If you would prefer to synchronize hashes under all circumstances, add a y, and so on.

## Creating checksums "quietly"..

If you want to script or schedule your hashing tasks, you will probably want checksum to run without any dialogs, notifications and so forth. If so, add the Quiet switch.. q

When the q options is used alone, if checksum encounters existing hash files, it continues as if you had clicked "No to All", in the "checksum file exists!" dialog, so no existing files are altered in any way. This is the safest default.

If you would prefer checksum to act as if you had clicked "Yes to all", instead, use q+, and any existing checksums will be overwritten.

If you want synchronization, add a y switch (it can be anywhere in the switches, so long as it's in there somewhere, but qy is just fine)

Quiet operation also works for verification, failures are logged, as usual. Like most of checksum's command-line switches, these behaviours can be set permanently, in your checksum.ini.

## Working with Cross-Platform hashes..

checksum has a number of features designed to make your cross-platform, inter-networking life a bit easier.

You don't have to do anything special to verify hash files created on Linux, UNIX, Solaris, Mac, or indeed any other major computing platform; checksum can handle these out-of-the-box.

If you need to create hash files for use on other platform, perhaps with some particular system file verification tool, checksum has a few preferences which might help..

You will perhaps appreciate checksum's plain text ini file (checksum.ini) containing all the permanent preferences. Inside there you can set not only which Line Feeds checksum uses in its files (Windows, UNIX, or Mac), but also enable UTF-8 files, single-file "root" hashing, generic hash file naming, UNIX file paths, and more. Lob checksum.ini into your favourite text editor and have a scroll.

There are a number of ways to run checksum. One handy way, especially if you are running checksum in portable mode without Explorer menus, is to keep a shortcut to checksum in the SendTo menu.

Simply put; any regular file or folder sent to checksum will be immediately hashed. Send a checksum file (.hash, .md5, .sha1, plus whatever UNIX hash files you have set), and it will be immediately verified. If you want extra options, hold down the <SHIFT> key, as usual.

If you want to send a non-checksum file, but have checksum treat it as a checksum file, hold down the <Ctrl> key during launch, to force checksum into verify mode (either just after you activate the SendTo item, or perhaps easier; hold down <SHIFT> AND <Ctrl> together while you click, to bring up the one-shot verify options). This is also handy for verifying folder hashes.

## How to accurately compare two folders or disks.

This is an easy one. First, create a "root" hash in the root of the first (source) folder/disk, then copy the .hash file over to the second (target) folder and click it.

That's it!

For situations where you don't need a permanent record of the hashes, you can fully automate the process of comparing two folders with simple checksum.

## How to accurately compare two CDs, DVDs, etc.(even when they don't have hash files on them)..

When hashing read-only media, obviously we cant store the hash files on the disk itself. However, thanks to checksum's range of intelligent read-only fall-back strategies, you can make light work of comparing read-only disks with super-accurate MD5 or SHA1 hashes, even if those disk were burned without  hashes.

All we need to do, is ask checksum to create a "root" hash file of the original disk, using the "Absolute paths" option. This will produce a hash file containing hashes for the entire disk, with full, absolute paths, e.g..

531a3ce6b631bb0048508d872fb1d72f *D:\Sweet.rar 558e40b6996e8a35db668011394cb390 *D:\Backups\Efficient.cab 832e98561d3fe5464b45ce67d4007c11 *D:\Sales Reports\April.zip

There are a few ways to achieve this. For one-off jobs, you can simply add k1 to your usual command-line switches. For example, to create a recursive root hash file of a disk, with absolute paths, you would use crk1.

Another way, is to set (and forget) checksum's fallback_level preference to 2, inside checksum.ini..

fallback_level=2

With fallback_level=2, when checksum is asked to create hashes of a read-only volume, it will fall-back to creating a single "root" hash with absolute paths, inside your fall-back location (also configurable), which is exactly what we need!

Then in the future, to verify the original disk, or copies of the disk; you simply insert it, and click the hash file.

You can store the .hash file anywhere you like; so long as the disk is always at D:\, or whatever drive letter you used to create the .hash file originally, it will continue to function perfectly.

If you want to know more about checksum's read-only fall-back strategies, see here.

## Or accurately compare a burned disk to its original .iso hashes..

If you have a .hash file of the original .iso file, in theory, a future rip of the disk to ISO format, should produce an .iso file with the exact same checksum as the original. My burner is getting old, but I needed to know, and so tested the theory.

I Torrented an .iso file of a DVD (Linux Distro) - the hashes were published onsite, checksum verified these were correct. I burned the disk to a blank DVD-R, and then deleted the original .iso file. Everything is now on the disk only. The .hash file is still on my desktop.

Then I used the ever-wonderful ImgBurn, to read the DVD to a temporary .iso file on my desktop.

Fortunately, the .iso file, and the original .iso file had the same name, so I didn't need to edit the .hash file in any way. Then the moment of truth. I click the .hash file, and checksum spins into action, verifying. A few seconds later... Beep-Beep! No Errors! It's a perfect match!

I can't speak for other software, but with ImgBurn at least, a burned disk can produce an .iso file with a hash bit-perfectly identical to that of the original .iso file used to create the disk, and can be relied upon for data verification. Good to know.

## checksum as an installer watcher..

Because checksum can so accurately inform you of changes in files, it can function as an excellent ad-hoc installer watcher. All you do is create a root checksum for the area you would like to watch. Run the installer. And afterwards, verify the checksum. If anything has changed, checksum will let you know about it, with the option to log the list to funky XHTML or plain text.

Similarly, checksum can be utilized in any situation where you need to know about changed files. You can even use it to compare registry files, one exported before, the other after an install or other process. If the hashes match, there's no need to look further.

## checksum's custom command-line switches..

Click & Go! is the usual way to operate checksum; but checksum also contains a lot of special functionality, accessed by the use of "switches"; meaningful letter combinations which instruct checksum to alter its operation in some way.

If you have some unusual task to accomplish, the one-shot options dialog enables you to manipulate the most common switches with simple checkbox controls. You can see the current switches in a readout, updating dynamically as you check and uncheck each option. But this output is also an input, where you can manipulate the switches directly, if you wish. If that is the case, you will probably find the following reference useful.

You may also find this section useful if you are constructing a full checksum command-line for some reason, maybe a Batch Runner command or batch script, or custom front-end for checksum, or altering your explorer context menu, or creating a scheduled task, or some Übertask for your Run command (Win+R) or something else. In each case, switches are placed before the file/folder path, for example; the full command-line to verify a checksum file might look like this..

C:\Program Files\corz\checksum\checksum.exe v C:\path\to\file.hash

Here are all the currently available switches:

c
Create checksums.
v
Verify checksums.
r
Recurse (only for directories).
y
Synchronize (add any new file hashes to existing checksum files).
i
During creation: create individual hash files (one per file).

During verification: performs hash search for individual file (see examples, below).

s
Create SHA1 checksums (default is to create MD5 checksums).
2
Create BLAKE2 checksums (default is to create MD5 checksums).
u
UPPERECASE hashes (default is lowercase).
m

During Creation: File masks. Put these in brackets after the m. e.g..  m(*.avi,*.rm)
Note: You can use your file groups here, e.g. m(music)

During verification: Check Only Modified Files.

Perform operations only on files with a modified timestamp more recent than their recorded timestamp. This is generally used along with the "w" switch, to update the hash and timestamp of a file or set of files you have mindfully changed, whilst skipping bit-checking all other files, potentially saving a lot of disk access and massive amounts of time, especially over network links.

Note: This feature is currently marked as "experimental".

j
Custom hash name (think: "John"!). Put this in brackets after the j. e.g..  j(my-hashes)
d

During Creation: Output Directory. Put this in brackets after the d. e.g..  d(C:\hashes)
NOTE: Make this the last bracketed switch on your command-line, i.e. m(..)j(..)d(..).

During verification: Override Root Directory. *ßeta Only

Using the d switch, you can specify a new root directory for relative hash files, enabling them to be checked outside their original location.

For example, if you created a relative .hash file for files in "D:\my files" and put the hashes in a folder, "e:\my hashes" using a command-line something like..

  crd("e:\my hashes") "D:\my files"

You can now verify this .hash file ("e:\my hashes\my files.hash") IN-PLACE using similar syntax:

  vrd("D:\my files") "e:\my hashes"

e
Add file extensions to checksum file name (for individual file hashes)..
1
Create one-file "root" checksums, like Linux CD's often have.
3
Create .m3u/.m3u8 playlists for all music files encountered (only for folder hashing)..
p
Create .pls playlists for all music files encountered (only for folder hashing)..
q
Quiet operation, no dialogs (for scripting checksum - see help for other options)..
h
Hide checksum files (equivalent to 'attrib +h').
o
One-shot Options. Brings up a dialog where you can select extra options for a job.
(to pop up the options at run-time, hold down the <SHIFT> key at launch)
b
Beeps. Enable audio alerts (PC speaker beeps or WAV files).
t
ToolTip. Enable the progress ToolTip windoid.
n
No Pause. Normally checksum pauses on completion so you can see the status. This disables it.
(note: you can also set the length of the pause, in your prefs)
k
Absolute Paths. Record the absolute path inside the (root) checksum file.
Use this only if you are ABSOLUTELY sure the drive letter isn't going to change in the future..
f
Log to a file
(if there are failures, checksum always gives you the option to log them to a file)
g
Go to errors.
If a log was created; e.g. there were errors; open the log folder on task completion.
l
Log everything.
(the default is to only log failures, if any).
w
Update changed hashes. (think: reWrite)
(during verification, hashes and timestamps for "CHANGED" files can be updated in your .hash file).

USE WITH CAUTION ON VOLUMES YOU KNOW TO BE GOOD!

x
During creation: used to specify ignored directories, using standard file masks.
e.g. checksum.exe cr1x(foo*,*bar,baz*qux) "D:\MyDir"

During verification: delete missing hashes.
(hashes for "MISSING" files are removed from your .hash file).

a
Only verify these checksum files.
(followed by algorithm letter: am for MD5, as for SHA1, a2 for BLAKE2 - see example below).
z
Shutdown when done.
Handy for long operations on desktop systems. A dialog will appear for 60 seconds, enabling you to abort the process, if required

The 'a', 'f', 'g', 'l' and 'w' switches take effect when verifying hashes.

The '1', '2', '3', 'e', 'h', 'j', 'k', 'm', 'p', 's', 'u', and 'y' switches take effect when creating hashes.

The 'd', 'i' and 'x' switches have different functions for creation and verification.

In other words..

global switches = b, n, o, q, r, t, z.
creation switches = 1, 2, 3, c, d, e, h, i, j, k, m, p, s, u, x, y.
verify switches = a(m/s/2), d, f, g, i, l, v, w, x.

Switches can be combined, like this..

… checksum.exe v "C:\my long path\to\files.md5"
[ note 'long' path (with spaces) enclosed in "quotes" ]
… checksum.exe crim(movies) c:\downloads
[ create individual checksums for all my movie files - note use of group name ]
… checksum.exe vas c:\archives
[ check all *.sha1 files in the path, not *.md5 files ]
… checksum.exe c3rm(music) p:\audio
[ recursive music file checksum creation, with automatic playlists ]
… checksum.exe cr1m(*.zip) d:\
[ create a "root" checksum for all zip files on drive D: ]
… checksum.exe vi d:\path\to\some\video file.avi
[ search for matching entry for "d:\path\to\some\video file.avi" and verify that one file ]
… checksum.exe crkq1m(movies)j(video-hashes)d(@desktop) v:\
[ quietly create a "root" checksum (named "video-hashes.hash") for all movie files on drive V: and place it on the desktop ]

note: Although it won't appear in the options dialog, the custom name, "video-hashes", will still be set when you begin the job.

### notes:

• The order of the switches isn't important, though the "m" switch must always be immediately followed by the file masks (in brackets), e.g. m(*.avi) (same with d(dir), j(name) and x(file*mask) switches), and the "a" switch must  be the first letter of a two letter combination, e.g. am

The d() (output directory/root directoy) switch must be the last (bracketed) switch on your command-line. If you are using custom hash file name and/or custom file groups, put those first, e.g..

"c:\path to\checksum.exe" cr1qnm(movies)j(my-hashes)d("c:\some (dir) here") "D:\My Movies"

..which would quietly create a root hash file for all the movie files in D:\My Movies, and put those hashes in "c:\some (dir) here\my-hashes.hash". Those tricky braces inside the example path are why it goes last.

• You don't need to specify the m(music) group switch to create playlists, only the 3. A command like checksum.exe rc3 "P:\audio" would create checksums recursively for all  files in the path p:\audio, whilst creating playlists for only  the music file types. Nifty, huh?

• Most of these switches also have a preference inside checksum.ini. If that preference is enabled, you can disable it temporarily by prepending the switch with a - (minus) character, e.g. to disable the Progress ToolTip, use -t

• Any of these switches can be easily added permanently to your Explorer right-click (context) menus. For instance, you may like to always use the one-shot options, without having to hold the SHIFT key every time. So simply add an o, and it will be so. See here for details of how to permanently alter checksum's Explorer context menu commands.

And remember, if there's some specific behaviour that you want set permanently, you can do that, and a lot more, inside checksum.ini..

## checksum.iniworking with checksum's UNIX-style preference file..

checksum has a lot of available options. Here is a page that will help you get the most out of them.

## I do requests!

If there's something you would like to accomplish with checksum, but don't think think checksum can; feel free to request a feature, below..

## Request a feature!

Skippy - 21.01.15 4:10 pm

Another request:

Can you add the possibility when you use the checksum launch modifiers with add/remove some options to put this hash/verify modification do a a txt or bat files on the root folder from checksum. So you can use it again on a easy way.

thanks

Skippy

You want checksum to copy its current command-line to a text file? Sure that's fairly trivial. Coming up. ;o) Cor

Diamond - 25.01.15 4:40 pm

Hi Corz...
Hope this isn't a stupid question...but how can I minimize simple checksum to tray on exit (pressing close 'X' button)? Is there a way to accomplish this w/the .ini file, and if so, how?
I'm using 4t tray minimizer -freeware- at present but I keep forgetting to hit 'minimize' and I keep closing simple checksum accidentally (very annoying!)

Pressing close (or hitting Esc) will always exit simple checksum. You can minimize it to the tray by clicking its tray icon.

Or you might prefer to set the transparency higher and leave it on the desktop somewhere.

;o) Cor

Gryph - 02.02.15 12:46 am

Hello Corz,

I'd like to exclude from hashing all system folders and .svn folders, and exclude . Also I'd like get both sha1 and md5 checksums for all files to be hashed in a single run. What is the easiest way to do so?

At the moment I use two runs (for two hashes) and command line string
cqyrb1snkx(".s*n","progra*s","*ecycle*","cache","*pplicati*ata","*ygwin*","*ython*")d(k:\infopack)

It works, but... )

Alternatively I tried feeding checksum with list of files I need a hash for (filenames exported from locate32), but it takes too long: for each file checksum process is started and terminated.

Thanks,
Gryph

There is currently no facility to add multiple hashing algorithms in a single run, though I will consider this.

When I find myself doing this sort of thing with any app, especially if there are large command-line parameters, I turn to Batch Runner.

;o) Cor

Paul - 13.02.15 4:11 am

Hey there!

I have the same request as Nacho. I'd like to be able to specifically state and compare a hash file to a directory, but I don't want the hash file to reside inside the directory. I can see how I can state which directory the hash gets written to, but I'm confused at how to actually verify or compare afterwards without copying it into the folder.

If you use absolute paths, you can simply click it wherever it is.

Also, I'd like to be able to run the create flag and consolidate nested folders hashes into the top level hash. Instead of having a hash file per nested folder, I'd like to have just one has file.

Paul

checksum has always been able to do this. See the "root" hashing option ("1" on the command-line) ;o) Cor

Keith Douglas - 27.04.15 12:08 pm

I love this tool.

How about being able to checksum just the image-portion of a JPEG? That way if, for example, I update the keywords, the JPEG checksum will still read as OK.

I was thinking of doing something like having ExifTool strip all metadata, then doing the checksum.

exiftool -all= image.jpg
<do checksum on resulting file>

Similarly for video file types that support metadata -- although that subject seems to be much more murky.

Thanks!

It's something I've been thinking about for a long time. Specifically, with MP3 files (so I can change the ID3 tags without re-hashing). But even with jpegs there is EXIF, IPTC, Comments and more. The entire subject is murky!

And quite a big job. Part of the work (hashing only part of a file) has been done with the recent embedded hash facility, but there is still a lot to be done. It's unlikely I'll get around to this any time soon unless some large corporation purchases a large license and wants it badly!

;o) Cor

Keith Douglas - 28.04.15 11:58 pm

... follow on to the JPG request above ...

I've made a python script to open and checksum the actual jpg image (so no metadata). I'd like to re-use your ".hash" container.

Would you add a ".ini" switch to ignore unknown checksum types? For example, if I specify hash-type "md5_jpg" in the hash file, your tool would ignore it rather than reporting a changed/corrupt file. I'd like to continue to use your tool for all but JPG files.

(And also, unknown checksum types should not be overwritten)

The hash types in the comment (date/time) line are, in fact, not used. One day they may be. So currently, it's not trivial to skip hashes based on what these lines contain. I will look into adding a mechanism for this. ;o) Cor

<edit>Incorporated into v1.7.1.*</edit>

Guy Gordon - 16.06.15 9:41 am

Great tool, thanks. Have a question about the synchronization option. If I've got a folder full of a huge music collection and I want to synchronize the hashes once every week (for example) to a single root .hash file, and I want it to update the hash for any changed files (from things like tagging & art changes, etc) and remove any from the .hash for files that are no longer in the collection, what is the process to do that? There seems to be options in the verify for updating changed, but I can't seem to get a create process that would pick up on new stuff to also be able to update changed and remove missing. I'm doing cloud backup of the .hash files so I can always go back to a previous version if I want to. I'm just trying to keep the root .hash file in true sync with what's currently (and up to date) in the collection.

Thanks!

You need two commands for this.

First, a verify command (which you can configure to remove hashes for missing files, update changed hashes and such), and then a create command to add any new hashes.

Note: you can put both commands into a single scheduled task.

;o) Cor

Guy Gordon - 22.06.15 5:57 pm

Ah, was wondering if a separate verify step was needed. But it seems to be verifying all files when I try, not just the changed/added ones. Is there a switch I'm not noticing that only checks files with a newer date/time stamp, and not the rest that haven't changed? I'm checking 20+ TB of stuff, and the "vrywxq1" switches I'm testing with takes a long time, and slow down to read all the files during the verify step on all the files that haven't changed.

But how do you know they haven't changed? There may be corruption, and unless you check them all, you won't know about it. Checking only "new" files is a recipe for disaster.

This sounds like the sort of task you would want to schedule for when you are asleep.

Having said that, I think I know where you are coming from; you want checksum to perform the "special" functions that can only be done during a verify operation without performing an actual verify operation.

But the problem is, checksum can't know if a file is changed, unless it checks it!

By the way, the "y" switch is only for checksum creation. More details here.

;o) Cor

Guy Gordon - 27.06.15 11:30 pm

I'm using Checksum as a "just to be safe" fallback. I'm checking against 15 drives in an unRAID array, with more than 20 TB across a GB network link. It takes days to run a full check, and is a lot of needless drive activity when those drives could be spun down, particularly with directory caching not spinning up drives unless actual file access is needed (iow, more than just name, date and usual directory info). unRAID is already doing full monthly health and parity checks. I'm just wanting to use Checksum as an extra level of safety, just in case the unthinkable happens and I suffer more than one drive death at the same time. In the case of unRAID, the other drives would still be readable and contiguous file data would still be on those, what Checksum would do for me is on the drives that failed (in an unlikely multi-drive failure situation) I would be able to verify against the hashes and see what on those drives is still OK. The other ones that didn't fail could also be checked, of course, and added back into a parity protect array again.

Two put it more simply, it's really not worth me using Checksum if I have to read more than 20TB of stuff across a network for days every time I want to simply update the hashes for changed/added files (which I'd probably do on a weekly basis). I simply want a mirror of hash info for what's there, as of the last time I ran Checksum. I don't recall how long it took for the first time I ran it across everything, but it was at least a couple days. It doesn't take too long to do the create pass to add in any new files, and of course only needs to spin up drives for when it does have new files to add which it needs to read & hash. If, however, I can't trust the hashes on files that were already there in the past to be up to date to any edits, tag changes, etc, then it's simply not worth the effort and resources of maintaining.

You stated that Checksum can't know if a file changed unless it checks it. Then what's the date/time stamp in the hash for? If it sees the file is newer than the stamp in the hash, then it updates the hash. At least that's what I'm trying to get it to do.

If Checksum can't do this, do you happen to know if there's a different app out there that can? I haven't gone hunting to see as yet. Was hoping this one could, as I really like everything else about it so far.

Apologies for the slightly delayed reply. The whole Google Malware fiasco was more than a bit consuming.

Back on topic.. If all you want to do is add hashes for new files, checksum can rip through your 20TB array in no time, adding hashes only for files that doesn't already exist in your .hash file(s).

Updating changed files is where it gets tricky. As it stands, when creating hashes, if checksum discovers a file that already has an entry in the corresponding .hash file, it moves immediately onto the next file.

During verification, checksum has the option to update changed files, as you know. Ideally, this is something that would only be done only after a full verification of the volume(s), and subsequent manual inspection of the resultant log file.

The trouble with simply enabling some flag and having checksum update all changed files on a scheduled basis for entire volumes is that that it only takes one single pass to update the hash of a recently CORRUPTED file, which also has a changed modification time. And WHAM! Now you have a false checksum, file corruption goes unnoticed, accumulates, until it's too late. This is, of course, exactly the sort of situation checksum was designed to prevent!

It's only after a full verification of the files that you have the information (a comprehensive log) required to make that sort of decision for each file. If all the changed files in the log were mindfully changed, you can go ahead and rewrite hashes and timestamps for those files.

Having this happen automatically, or on a schedule, is a recipe for disaster. And adding a flag to skip unmodified files encourages schedules, mindless hashing, defeating the whole purpose of hashing.

Having said all that, I do see the utility of having checksum update the hashes of only changed files for many users and actually slipped this exact functionality into the latest release (1.7.0.1), which I put up yesterday (slightly ahead of schedule thanks to the GoogleDebackle).

Most unusually for a general (non-beta) release this functionality hasn't been tested much and there is a known issue with verification inside Daylight Saving Time (BST) adding an hour (or perhaps other amounts) to the calculated modification times of files hashed outside DST (Somewhere deep in the Windows API!). If you don't use the "m" switch during verification, there is no impact to the rest of the code-base, so I left it in as an "experimental feature". Have fun! And do let me know of any issues.

Although it inevitably takes time, a full verification is always advisable before switching on any of checksum's auto-update flags. But if you know your drive and what you are doing, knock yourself out!

Please do enable logging and do examine those LOGS!

;o) Cor

archedraft - 07.07.15 2:21 am

some code..
archedraft - 12.09.14 8:45 pm

Cor,

I just wanted to add to Jumperalex's posts that having an updated corz linux cli would be fantastic. I currently use my Windows box with checksum but I am sure it would check much faster if I could perform checksums directly from my NAS box. Either way, keep up the great work! I'll continue to send as many friends your way as I can.

-archedraft

Friends are good, thank you! Customers, too - you can never have too many of those!

checksum on Linux is only sleeping. There will be more ASAP!

;o) Cor

Just wanted to check in and see if checksum on Linux has awaken!

Any updates to the Linux version will be noted on the checksum for Linux page. ;o) Cor

Nacho - 08.07.15 6:21 pm

Hi! Any updates on the ability to specify a custom target path to verify hashes (see my last post Nacho - 06.10.14 11:34 pm)?
Also Paul asked about it (Paul - 13.02.15 4:11 am). You answered "If you use absolute paths, you can simply click it wherever it is.", but that's not an option if you want batch processing and you are trying to verify a hash file with relative paths from another place, for instance from a backup on USB drive where drive letters change.

I also would like to suggest the ability to use relative paths while creating and verifying hashes:

checksum cr1 "..\myfolder"
checksum v "..\myfolder"


That way you could run portable checksum.exe from a removable device no matter the assigned drive letter is.

Thanks again for your time and effort

All updates are posted in the usual place!

As for relative operation. checksum can already do this. Ensure you correctly set the working directory (or run from the portable location) and you should be good to go.

;o) Cor

### (if you find the code difficult to decipher, click it for a new one!)

Enter the 5-digit code this text sounds like :

lower-case queue, Upper-Case Aye, sicks, lower-case you, Upper-Case Vee

Welcome to corz.org!

If something isn't working, I'm probably improving it, try again in a minute. If it's still not working, please mail me!