HASHSUM.BAT v1.8 - emulate md5sum, shasum, and the like
Posted: 05 Dec 2016 23:47
The Windows CERTUTIL command has an option to compute file hashes using any of the following algorithms: MD2, MD4, MD5, SHA1, SHA256, SHA384, SHA512.
But it is not very convenient for preparing or checking manifests (digests) that are compatible with md5sum, shasum, or the like. So I wrote HASHSUM.BAT to package up CERTUTIL in a way that is compatible with md5sum, etc.
There were a few minor issues to overcome:
When preparing a manifest, it defaults to SHA256, but you can change the algorithm with the /A option.
When checking a manifest with the /C option, you have three options for specifying the algorithm, with higher precedence at the top.
Examples of usage follow the code in this post.
HASHSUM.BAT version 1.8
Here is an example showing the generation of MD5 digests for all exe files under my c:\test folder, including subdirectories.
I can capture the output as a manifest using simple redirection. Say I saved the file as exe_files.md5, then I can simply use the following to verify all files in the manifest:
I can use a pipe to quickly check a file if I happen to know the hash value
I can also compute the hash of any text by piping the content into HASHSUM
The new /U option in version 1.8 can be used to enable support for Unicode in filenames. This may be needed if your filesystem has file names that are not supported by the active code page. However, at time of this release, this option has not been adequately tested - please report any problems with the /U option to this thread
Dave Benham
Code: Select all
C:\test>certutil -hashfile test.bat md5
MD5 hash of file test.bat:
89 68 0c 14 1d 7a 1f 00 a2 43 79 c6 b1 ac fd b9
CertUtil: -hashfile command completed successfully.
There were a few minor issues to overcome:
- CERTUTIL delimits hex pairs with spaces, whereas md5sum etc. uses a continuous stream of hex digits.
- CERTUTIL gives an error if the file is empty (size 0). I had to hardcode the various hashes for zero length strings.
- I needed a way to capture relative paths when using the /S option. I opted to use FORFILES, which is very convenient, but slow.
- I needed a way to capture stdin to a temp file, without any modification. I initially used FINDSTR, but that appends \r\n if the last line does not end with \n. So I converted to using JScript, thus HASHSUM is no longer pure batch.
When preparing a manifest, it defaults to SHA256, but you can change the algorithm with the /A option.
When checking a manifest with the /C option, you have three options for specifying the algorithm, with higher precedence at the top.
- Explicitly specify the algorithm with the /A option
- Derive the algorithm from the extension of the manifest
- Automatically detect the algorithm based on the length of the first hash in the manifest
Examples of usage follow the code in this post.
HASHSUM.BAT version 1.8
Code: Select all
@if (@X)==(@Y) @end /* Harmless hybrid line that begins a JScript comment
@goto :Batch
::::
::::HASHSUM.BAT history
::::
:::: v1.8 2021-01-19 - Improved detection of current code page as per work at
:::: https://www.dostips.com/forum/viewtopic.php?f=3&t=8533
:::: v1.7 2021-01-19 - Added /U option to support Unicode in file names
:::: v1.6 2019-02-26 - Modify /F and /FR to support non-ASCII characters
:::: v1.5 2018-02-18 - Added /H, /F, /FR and /NH options.
:::: v1.4 2016-12-26 - Convert /A value to upper case because some Windows
:::: versions are case sensitive. Also improve JScript file
:::: read performance by reading 1000000 bytes instead of 1.
:::: v1.3 2016-12-17 - Bug fixes: Eliminate unwanted \r\n from temp file by
:::: reading stdin with JScript instead of FINDSTR.
:::: Fix help to ignore history.
:::: v1.2 2016-12-07 - Bug fixes: Exclude FORFILES directories and
:::: correct setlocal/endlocal management in :getOptions
:::: v1.1 2016-12-06 - New /V option, and minor bug fixes.
:::: v1.0 2016-12-05 - Original release
:::
:::HASHSUM [/Option [Value]]... [File]...
:::
::: Print or check file hashes using any of the following standard
::: hash algorithms: MD5, SHA1, SHA256, SHA384, or SHA512.
:::
::: HASHSUM always does a binary read - \r\n is never converted to \n.
:::
::: In the absence of /C, HASHSUM computes the hash for each File, and writes
::: a manifest of the results. Each line of output consists of the hash value,
::: followed by a space and an asterisk, followed by the File name. The default
::: hash alogrithm is sha256. File may include wildcards, but must not contain
::: any path information.
:::
::: If File is not given, then read from standard input and write the hash
::: value only, without the trailing space, asterisk, or file name.
:::
::: Options:
:::
::: /? - Prints this help information to standard output.
:::
::: /?? - Prints paged help using MORE.
:::
::: /V - Prints the HASHSUM.BAT version.
:::
::: /H - Prints the HASHSUM.BAT history.
:::
::: /U - Unicode mode: Experimental setting that attempts to support
::: Unicode in filenames. Please report any problems to the DosTips
::: forum at https://www.dostips.com/forum/viewtopic.php?f=3&t=7592
:::
::: /A Algorithm
:::
::: Specifies one of the following hash algorithms:
::: MD5, SHA1, SHA256, SHA384, SHA512
:::
::: /P RootPath
:::
::: Specifies the root path for operations.
::: The default is the current directory.
:::
::: /S - Recurse into all Subdirectories. The relative path from the root
::: is included in the file name output.
::: This option is ignored if used with /C.
:::
::: /I - Include the RootPath in the file name output.
::: This option is ignored if used with /C.
:::
::: /T - Writes a space before each file name, rather than an
::: asterisk. However, files are still read in binary mode.
::: This option is ignored if used with /C.
:::
::: /C - Read hash values and file names from File (the manifest), and verify
::: that local files match. File may include path information with /C.
:::
::: If File is not given, then read hash and file names from standard
::: input. Each line of input must have a hash, followed by two spaces,
::: or a space and an asterisk, followed by a file name.
:::
::: If /A is not specified, then the algorithm is determined by the
::: File extension. If the extension is not a valid algorithm, then
::: the algorithm is derived based on the length of the first hash
::: within File.
:::
::: Returns ERRORLEVEL 1 if any manifest File is not found or is invalid,
::: or if any local file is missing or does not match the hash value in
::: the manifest. If all files are found and match, then returns 0.
:::
::: /F FileName
:::
::: When using /C, only check lines within the manifest that contain the
::: string FileName. The search ignores case.
:::
::: /FR FileRegEx
:::
::: When using /C, only check lines within the manifest that match the
::: FINDSTR regular expression FileRegEx. The search ignores case.
:::
::: /NH - (No Headers) Suppresses listing of manifest name(s) when using /C.
:::
::: /NE - (No Errors) Suppresses error messages when using /C.
:::
::: /NM - (No Matches) Suppresses listing of matching files when using /C.
:::
::: /NS - (No Summary) Suppresses summary information when using /C.
:::
::: /Q - (Quiet) Suppresses all output when using /C.
:::
:::HASHSUM.BAT version 1.8 was written by Dave Benham
:::maintained at https://www.dostips.com/forum/viewtopic.php?f=3&t=7592
============= :Batch portion ===========
@echo off
setlocal disableDelayedExpansion
:: Define options
set "options= /A:"" /C: /I: /P:"" /S: /T: /?: /??: /NH: /NE: /NM: /NS: /Q: /V: /H: /U: /F:"" /FR:"" "
:: Set default option values
for %%O in (%options%) do for /f "tokens=1,* delims=:" %%A in ("%%O") do set "%%A=%%~B"
set "/?="
set "/??="
:getOptions
if not "%~1"=="" (
set "test=%~1"
setlocal enableDelayedExpansion
if "!test:~0,1!" neq "/" endlocal & goto :endOptions
set "test=!options:*%~1:=! "
if "!test!"=="!options! " (
endlocal
>&2 echo Invalid option %~1
exit /b 1
) else if "!test:~0,1!"==" " (
endlocal
set "%~1=1"
) else (
endlocal
set "%~1=%~2"
shift /1
)
shift /1
goto :getOptions
)
:endOptions
:: Display paged help
if defined /?? (
(for /f "delims=: tokens=*" %%A in ('findstr "^:::[^:] ^:::$" "%~f0"') do @echo(%%A)|more /e
exit /b 0
) 2>nul
:: Display help
if defined /? (
for /f "delims=: tokens=*" %%A in ('findstr "^:::[^:] ^:::$" "%~f0"') do echo(%%A
exit /b 0
)
:: Display version
if defined /V (
for /f "delims=: tokens=*" %%A in ('findstr /ric:"^:::hashsum.bat version" "%~f0"') do echo(%%A
exit /b 0
)
:: Display history
if defined /H (
for /f "delims=: tokens=*" %%A in ('findstr "^::::" "%~f0"') do echo(%%A
exit /b 0
)
:: Unicode filename support
set "find=find"
if defined /U (
for /f "tokens=*" %%A in ('chcp') do for %%B in (%%A) do set "chcp=%%~nB"
set "find=findstr /l"
>nul chcp 65001
)
:: If no file specified, then read stdin and write to a temp file
set "tempFile="
if "%~1" equ "" set "tempFile=%~nx0.%time::=_%.%random%.tmp"
if defined tempFile cscript //nologo //E:JScript "%~f0" "%temp%\%tempFile%"
if defined /P cd /d "%/P%" || exit /b 1
if defined /C goto :check
:generate
if defined tempFile cd /d "%temp%"
if not defined /A set "/A=sha256"
if defined /S set "/S=/s"
if defined /T (set "/T= ") else set "/T=*"
call :defineEmpty
if not defined /P goto :generateLoop
if not defined /I goto :generateLoop
if "%/P:~-1%" equ "\" (set "/I=%/P:\=/%") else set "/I=%/P:\=/%/"
set "rtn=0"
:generateLoop
(
for /f "delims=" %%F in (
'forfiles %/s% /m "%tempFile%%~1" /c "cmd /c if @isdir==FALSE echo @relpath" 2^>nul'
) do for /f "delims=" %%A in (
'certutil.exe -hashfile %%F %/A% ^| %find% /v ":" ^|^| if %%~zF gtr 0 (echo X^) else echo %empty%'
) do (
set "file=%%~F"
set "hash=%%A"
setlocal enableDelayedExpansion
set "file=!file:~2!"
if defined tempFile (
if !hash! equ X (
set "rtn=1"
echo ERROR
) else echo !hash: =!
) else (
if !hash! equ X (
set "rtn=1"
echo ERROR: !/I!!file!
) else echo !hash: =! !/T!!/I!!file:\=/!
)
endlocal
)
) || (
set "rtn=1"
echo MISSING: %/T%%1
)
shift /1
if "%~1" neq "" goto :generateLoop
if defined tempFile del "%tempFile%"
if defined /U >nul chcp %chcp%
exit /b %rtn%
:check
if defined /Q for %%V in (/NE /NM /NS /NH) do set "%%V=1"
if defined /F if defined /FR (
>&2 echo ERROR: /F and /FR cannot be combined
exit /b 1
)
set "searchTemp="
if defined /F (
set "searchTemp=%temp%\%~nx0.%time::=_%.%random%.search.tmp"
setlocal enableDelayedExpansion
(echo(!/F!) > "!%searchTemp!"
endlocal
set "file=" & set "freg=rem" & set "norm=rem"
) else if defined /FR (
set "searchTemp=%temp%\%~nx0.%time::=_%.%random%.search.tmp"
setlocal enableDelayedExpansion
(echo(!/FR!) > "!%searchTemp!"
endlocal
set "file=rem" & set "freg=" & set "norm=rem"
) else (
set "file=rem" & set "freg=rem" & set "norm="
)
set /a manifestCnt=missingManifestCnt=invalidCnt=missingCnt=failCnt=okCnt=0
:checkLoop
set "alogorithm=%/A%"
if defined tempFile set "tempFile=%temp%\%tempFile%"
for %%F in ("%tempFile%%~1") do call :checkFile "%%~F"
if defined tempFile del "%tempFile%"
shift /1
if "%~1" neq "" goto :checkLoop
if defined searchTemp del "%searchTemp%"
if not defined /NS (
echo ========== SUMMARY ==========
echo Total manifests = %manifestCnt%
echo Matched files = %okCnt%
echo(
if %missingManifestCnt% gtr 0 echo Missing manifests = %missingManifestCnt%
if %invalidCnt% gtr 0 echo Invalid manifests = %invalidCnt%
if %missingCnt% gtr 0 echo Missing files = %missingCnt%
if %failCnt% gtr 0 echo Failed files = %failCnt%
)
if defined /U >nul chcp %chcp%
set /a "1/(missingManifestCnt+invalidCnt+missingCnt+failCnt)" 2>nul && (
echo(
exit /b 1
)
exit /b 0
:checkFile
set /a manifestCnt+=1
if not defined /NH if defined tempfile (echo ---------- ^<stdin^> ----------) else echo ---------- %1 ----------
if not defined algorithm set "/A="
if not defined /A echo *.md5*.sha1*.sha256*.sha384*.sha512*|%find% /i "*%~x1*" >nul && for /f "delims=." %%A in ("%~x1") do set "/A=%%A"
findstr /virc:"^[0123456789abcdef][0123456789abcdef]* [ *][^ *?|<>]" %1 >nul 2>nul && (
if not defined /NE if defined tempFile (echo *INVALID: ^<stdin^>) else echo *INVALID: %1
set /a invalidCnt+=1
exit /b
)
(
%norm% for /f "usebackq tokens=1* delims=* " %%A in (%1) do (
%file% for /f "tokens=1* delims=* " %%A in ('type %1 ^| findstr /ilg:"%searchTemp%"') do (
%freg% for /f "tokens=1* delims=* " %%A in ('type %1 ^| findstr /irg:"%searchTemp%"') do (
set "hash0=%%A"
set "fileName=%%B"
if defined /A (call :defineEmpty) else call :determineFormat
setlocal enableDelayedExpansion
set "fileName=!fileName:/=\!"
for /f "tokens=1* delims=" %%C in (
'certutil.exe -hashfile "!fileName!" !/A! ^| %find% /v ":" ^|^| if exist "!fileName!" (echo !empty!^) else echo X'
) do set "hash=%%C"
if /i "!hash0!" equ "!hash: =!" (
if not defined /NM echo OK: !fileName!
endlocal
set /a okCnt+=1
) else if !hash! equ X (
if not defined /NE echo *MISSING: !fileName!
endlocal
set /a missingCnt+=1
) else (
if not defined /NE echo *FAILED: !fileName!
endlocal
set /a failCnt+=1
)
)
) 2>nul || if not defined /F if not defined /FR (
if not defined /NE echo *MISSING: %1
set /a missingManifestCnt+=1
)
exit /b
:determineFormat
if "%hash0:~127%" neq "" (
set "/A=SHA512"
) else if "%hash0:~95%" neq "" (
set "/A=SHA384"
) else if "%hash0:~63%" neq "" (
set "/A=SHA256"
) else if "%hash0:~39%" neq "" (
set "/A=SHA1"
) else set "/A=MD5"
:defineEmpty
if /i "%/A%"=="md5" (
set "empty=d41d8cd98f00b204e9800998ecf8427e"
set "/A=MD5"
) else if /i "%/A%"=="sha1" (
set "empty=da39a3ee5e6b4b0d3255bfef95601890afd80709"
set "/A=SHA1"
) else if /i "%/A%"=="sha256" (
set "empty=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
set "/A=SHA256"
) else if /i "%/A%"=="sha384" (
set "empty=38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b"
set "/A=SHA384"
) else if /i "%/A%"=="sha512" (
set "empty=cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e"
set "/A=SHA512"
) else (
echo ERROR: Invalid /A algorithm>&2
(goto) 2>nul&exit /b 1
)
exit /b
************* JScript portion **********/
var fso = new ActiveXObject("Scripting.FileSystemObject");
var out = fso.OpenTextFile(WScript.Arguments(0),2,true);
var chr;
while( !WScript.StdIn.AtEndOfStream ) {
chr=WScript.StdIn.Read(1000000);
out.Write(chr);
}
Code: Select all
C:\test>hashsum /s /a md5 *.exe
8d443f2e93a3f0b67f442e4f1d5a4d6d *md5.exe
eb574b236133e60c989c6f472f07827b *md5sum.exe
9f036e40755daaa5a5fc141d3cbfbb99 *nGetPid.exe
36622f75d59bd1988bfd18a259ae82ef *Aacini Tools/Ascii.exe
f26e6cdc8f8bba5df1634bb631f6306f *Aacini Tools/Colorshow.exe
512b675126f22da6d50916b98222c5d7 *Aacini Tools/Cursorpos.exe
bda35b908b81f7dfad35beafd1582279 *Aacini Tools/Cursorsize.exe
4cf48fe5dca5638d0ddfd3429faf6abf *Aacini Tools/Flushinputbuffer.exe
d81a9a79847aa075bebb8ec054bd13e7 *Aacini Tools/Getkey.exe
b269ecf2bed68cef8d623c877c3b93d6 *Aacini Tools/Show.exe
ddbece33177224a8a2f44600cc71e76b *Aacini Tools/Stddate.exe
0d929b261d0981d3c46c04418af2ec6d *Aacini Tools/Stdtime.exe
34df06129a094d773c14092eb151a8e0 *Aacini Tools/Strlen.exe
c0a7f18cb075d39ab2d0978dd7a261dc *Aacini Tools/Textcolor.exe
3ff1f054f90a708e0fdeb8b4c81db24e *Aacini Tools/Window.exe
538d34e6b28951cb412761e634805f5b *Aacini Tools/bin/Hexchar.exe
dfc42151c290ae5bbbc448d211405489 *ansisys/AnsiSys.exe
6795af42d625abe4bd529798aba07bfd *bf/BFI.exe
5f02fcfe6262dfb852b5ce904f14e20d *bf/**censored**.exe
0e93af2d97030b34334b8f3200f5ca44 *calc/ColorChar.exe
512b675126f22da6d50916b98222c5d7 *calc/CursorPos.exe
82d93fd60f4686c82034f8a70dc0b269 *calc/GetInput.exe
59f74ed847f7b7fac82b809896b12f67 *choice/choice.exe
50c52220cd00b642ff4e5b85abd28557 *curl/curl.exe
33900b6c02c16fcf630b468a36156496 *fonts/hbFontEdit/FONTEDIT.EXE
c6e160e565a0b5764cf70ed258b9ffd9 *fonts/MittensKnight/FontEdit.exe
a89f984fbb6820cd505971f5f6d0ca89 *fonts/RasterFontEditor/RasterFontEditor.exe
e49097bb9659520ae8cb7cdddab1f5fd *pixelfnt_dev/PIXELFNT.EXE
69b8044bb90d89705289df7e10a5152a *SendMessage/SendMessage.exe
Code: Select all
C:\test>hashsum /c exe_files.md5
---------- "exe_files.md5" ----------
OK: md5.exe
OK: md5sum.exe
OK: nGetPid.exe
OK: Aacini Tools\Ascii.exe
OK: Aacini Tools\Colorshow.exe
OK: Aacini Tools\Cursorpos.exe
OK: Aacini Tools\Cursorsize.exe
OK: Aacini Tools\Flushinputbuffer.exe
OK: Aacini Tools\Getkey.exe
OK: Aacini Tools\Show.exe
OK: Aacini Tools\Stddate.exe
OK: Aacini Tools\Stdtime.exe
OK: Aacini Tools\Strlen.exe
OK: Aacini Tools\Textcolor.exe
OK: Aacini Tools\Window.exe
OK: Aacini Tools\bin\Hexchar.exe
OK: ansisys\AnsiSys.exe
OK: bf\BFI.exe
OK: bf\**censored**.exe
OK: calc\ColorChar.exe
OK: calc\CursorPos.exe
OK: calc\GetInput.exe
OK: choice\choice.exe
OK: curl\curl.exe
OK: fonts\hbFontEdit\FONTEDIT.EXE
OK: fonts\MittensKnight\FontEdit.exe
OK: fonts\RasterFontEditor\RasterFontEditor.exe
OK: pixelfnt_dev\PIXELFNT.EXE
OK: SendMessage\SendMessage.exe
========== SUMMARY ==========
Total manifests = 1
Matched files = 29
Code: Select all
C:\test>echo 9f036e40755daaa5a5fc141d3cbfbb99 *nGetPid.exe | hashsum /c
---------- <stdin> ----------
OK: nGetPid.exe
========== SUMMARY ==========
Total manifests = 1
Matched files = 1
Code: Select all
C:\test>echo Hello world|hashsum /a sha1
f0e19ccba8cd50a4288c368eeb655c524fedcf5f
Dave Benham
