CRC32

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
einstein1969
Expert
Posts: 960
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

CRC32

#1 Post by einstein1969 » 02 Dec 2013 15:11

Hi, I have implemented a simple version of CRC32.

I have token the code from this (look at "Using direct calculation")

The test is made on string "123456789" that has result 0xCBF43926


UPDATE:

  • 18/6/2014 Complete rewritten.
  • 19/6/2014 fixex bug of files with ! in their name


version 0.1c: (older version follow)

Code: Select all

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
:: CRC32 version 0.1c by einstein1969.
:: ----------------------------------------------------------------
::
:: Thanks to: penpen, Magialisk, jeb, dbenham, foxidrive
::
:: 19/06/2014 ver. 0.1c
::            - Tuning performance. Doubled the speed.
::            - Fixed filename with ! bug.
::            - Other code optimization.
::   
:: TODO: Async Mode with pipe and SET/P read.
::
:: Rif: http://www.dostips.com/forum/viewtopic.php?p=30336#p30336
::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

@echo off & setlocal DisableDelayedExpansion

:: Show Syntax
if "%~1"=="" (
  echo(
  echo CRC32 ^<string^|filename^> [^<variable^>]
  echo       Calculates the CRC32 of the input string or file.
  echo       Optionally stores it in "variable".
  echo(
  echo Example:
  echo CRC32 "123456789"
  Echo CRC32:0xCBF43926
  exit /b
)

set "inFile=%~1"

setlocal EnableDelayedExpansion

:: String MODE
IF not exist !inFile! (
   set SM=1
   <nul set /p ".=%inFile%" > "%temp%\crc32.tmp"
   call "%~f0" "%temp%\crc32.tmp" %~2
   del "%temp%\crc32.tmp"
   :: Pass the result back over the second endlocal barrier
   if Not "%~2"=="" for %%a in (!crc32!) DO (endlocal & endlocal & set %~2=%%a)
   exit /b
)

:: create dummy file (of @=40Hex) of dim %~Z1 for compare with FC. Jeb's technique.
set "$t=%temp%\crc32_$$$.tmp"
set "$t2=%temp%\crc32_$$$2.tmp"
type nul > "!$t2!"
<nul > "!$t!" set /p ".=@"
set "ds=%~z1"
for /l %%n in (1,1,32) do (
  set /a "r=ds %% 2, ds/=2"
  if !r! equ 1 type "!$t!" >> "!$t2!"
  if !ds! gtr 0 type "!$t!" >> "!$t!"
)
set ds=%~z1

:: Compare and write to tmp file
if !ds! gtr 50000 echo Wait...
>"!$t!" fc /B "!inFile!" "!$t2!"

:: Prepare progress bar
set "L0=_" & set "L1=°" & set "L2=±" & set "L3=²" & set "L4=Û"
for /L %%l in (0,1,4) do for /L %%c in (0,1,49) do set L%%l=!L%%l!!L%%l:~0,1!

:: prepare for fast execute
setlocal
(
for /F "Tokens=1 delims==" %%v in ('set') do set "%%v="
for /f %%a in ('copy /Z "%~f0" nul') do set "_CR=%%a"

:: Init b=CRC
set /a b=0xFFFFFFFF

>"%temp%\time_t0.tmp" echo %time%

:: Scan tmp file
set /a I=1
for /F "usebackq skip=1 tokens=1,2 delims=: " %%b in ("%$t%") do (

  :: Fill gaps
  for /L %%B in (!I!, 1, 0x%%b) do (
    set /a "a=(b ^ 0x40) & 0xff"
    for /L %%i in (0,1,7) do set /a "a=((a>>1) & 0x7FFFFFFF) ^ (0xEDB88320 * (a & 1))"
    set /a "b = ((b >> 8)&0x00FFFFFF) ^ a"
  )

  :: CRC core calculation
  set /a "a=((((b ^ 0x%%c) & 0xff)>>1) & 0x7FFFFFFF) ^ (0xEDB88320 * ((b ^ 0x%%c) & 1)), a=((a>>1) & 0x7FFFFFFF) ^ (0xEDB88320 * (a & 1)), a=((a>>1)&0x7FFFFFFF) ^ (0xEDB88320 * (a & 1)), a=((a>>1) & 0x7FFFFFFF) ^ (0xEDB88320 * (a & 1)), a=((a>>1)&0x7FFFFFFF) ^ (0xEDB88320 * (a & 1)), a=((a>>1) & 0x7FFFFFFF) ^ (0xEDB88320 * (a & 1)), a=((a>>1)&0x7FFFFFFF) ^ (0xEDB88320 * (a & 1)), a=((a>>1) & 0x7FFFFFFF) ^ (0xEDB88320 * (a & 1)), b = ((b >> 8)&0x00FFFFFF) ^ a, I=0x%%b+2"

  :: Progress bar.
  if "!i:~-4!"=="0000" call :progress %ds% "%temp%\time_t0.tmp" %L0% %L1% %L2% %L3% %L4%
)

:: Final Bytes/gap
for /L %%B in (!I!, 1, %~z1) do (
  set /a "a=(b ^ 0x40) & 0xff"
  for /L %%i in (0,1,7) do set /a "a=((a>>1) & 0x7FFFFFFF) ^ (0xEDB88320 * (a & 1))"
  set /a "b = ((b >> 8)&0x00FFFFFF) ^ a"
)

:: Progress bar. Final
if %ds% gtr 10000 (set I=%~z1 & call :progress %ds% "%temp%\time_t0.tmp" %L0% %L1% %L2% %L3% %L4%)

)
endlocal & set b=%b%

:: Final operations
set /a "CRC32=b ^ 0xffffffff"

:: Convert to Hexdecimal
call cmd /c exit /b !CRC32!
set crc32=!=ExitCode!

:: Print out result.
echo(
echo CRC32:!CRC32!

:: If requested return into variable.
IF NOT "%~2"=="" IF !SM! equ 1 (endlocal & endlocal & set CRC32=%CRC32%) else (endlocal & endlocal & set %~2=%CRC32%)
exit /b

:progress
(   set "t1=!time!" & set/p "t0=" <"%~2"
    for /F "tokens=1-8 delims=:.," %%a in ("!t0: =0!:!t1: =0!") do set /a "a=(((1%%e-1%%a)*60)+1%%f-1%%b)*6000+1%%g%%h-1%%c%%d, a+=(a>>31) & 8640000"
    set /a "max=%1, parz=max/4, k=i/parz, n=k+1, rapp=(i-k*parz)*50/parz, d=50-rapp, vel=I*100/a, pt=I*100/max"
    set L0=%3&set L1=%4&set L2=%5&set L3=%6&set L4=%7&set L5=%7
    for %%k in (!k!) do set LL0=!L%%k!
    for %%n in (!n!) do set LL1=!L%%n!
    for %%r in (!rapp!) do for %%d in (!d!) do <nul set /p "=^<!LL1:~0,%%r!!LL0:~0,%%d!^> !pt!%% - !vel!B/s  !_CR!"
    for /F "Tokens=1 delims==" %%v in ('set') do if /I not "%%v"=="I" if /I not "%%v"=="b" if /I not "%%v"=="_CR" set "%%v="
exit /b )


version 0.1a:

Code: Select all

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
:: CRC32 version 0.1a by einstein1969.
:: ----------------------------------------------------------------
::
:: Thanks to: penpen, Magialisk, jeb, dbenham
::
:: Rif: http://www.dostips.com/forum/viewtopic.php?p=30336#p30336
::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

@echo off & setlocal EnableDelayedExpansion

:: Show Syntax
if "%~1"=="" (
  echo(
  echo CRC32 ^<string^|filename^> [^<variable^>]
  echo       Calculates the CRC32 of the input string or file.
  echo       Optionally stores it in "variable".
  echo(
  echo Example:
  echo CRC32 "123456789"
  rem result expected is 0xCBF43926
  rem CALL "%~f0" "123456789"
  exit /b
)

set "inFile=%~1"

:: String MODE
IF not exist !inFile! (
   set SM=1
   <nul set /p ".=%inFile%" > "%temp%\crc32.tmp"
   CALL "%~f0" "%temp%\crc32.tmp" %~2
   del "%temp%\crc32.tmp"
   :: Pass the result back over the second ENDLOCAL barrier
   IF NOT "%~2"=="" FOR %%a IN (!crc32!) DO ENDLOCAL& set %~2=%%a
   exit /b
)

:: Init CRC
set /a  CRC32=0xFFFFFFFF
set /a Poly=0xEDB88320

:: create dummy file (of @=40Hex) of dim %~Z1 for compare with FC. Jeb's technique.
type nul > "%temp%\crc32_dummy.tmp"
<nul > "%temp%\crc32_$$$.tmp" set /p ".=@"
set "ds=%~z1"
for /l %%n in (1,1,32) do (
  set /a "r=ds %% 2, ds/=2"
  if !r! equ 1 type "%temp%\crc32_$$$.tmp" >> "%temp%\crc32_dummy.tmp"
  if !ds! gtr 0 type "%temp%\crc32_$$$.tmp" >> "%temp%\crc32_$$$.tmp"
)
del "%temp%\crc32_$$$.tmp"
set ds=
set r=


:: File big. Do Progress bar
rem if %~z1 gtr 10240 (
  for /f %%a in ('copy /Z "%~f0" nul') do set "CR=%%a"
  set "L0=_" & set "L1=°" & set "L2=±" & set "L3=²" & set "L4=Û"
  for /L %%l in (0,1,4) do for /L %%c in (1,1,39) do set L%%l=!L%%l!!L%%l:~0,1!
  set /a max=%~z1, parz=max/4, max=parz*4
rem )

:: Compare and write to tmp file
>"%temp%\crc32_$$$.tmp" fc /B "%inFile%" "%temp%\crc32_dummy.tmp"

:: Scan tmp file
set /a I=1
set t0=!Time!
for /F "usebackq skip=1 tokens=1,2 delims=: " %%b in ("%temp%\crc32_$$$.tmp") do (
  for /L %%B in (!i!, 1, 0x%%~b) do (
    set /a "a=(CRC32 ^ 0x40) & 0xff"
    for /L %%i in (0,1,7) do (
      set /a "t=a & 1"
      if !t! equ 1 (set /a "a=((a>>1)&0x7FFFFFFF)^Poly") else set /a "a=((a>>1)&0x7FFFFFFF)"
    )
    set /a "CRC32 = ((CRC32 >> 8)&0x00FFFFFF) ^ a"
  )

  set /a "a=(CRC32 ^ 0x%%c) & 0xff, a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1))"
  set /a "a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1)), a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1))"
  set /a "a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1)), a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1))"
  set /a "a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1)), a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1))"
  set /a "a=(((a>>1)&0x7FFFFFFF)^Poly) * (a & 1) + ((a>>1)&0x7FFFFFFF) * (1-(a & 1)), CRC32 = ((CRC32 >> 8)&0x00FFFFFF) ^ a, I=0x%%b+2"

  :: Progress bar.
  if "!i:~-3!"=="000" (
    for /F "tokens=1-8 delims=:.," %%a in ("!t0: =0!:!time: =0!") do set /a "a=(((1%%e-1%%a)*60)+1%%f-1%%b)*6000+1%%g%%h-1%%c%%d, a+=(a>>31) & 8640000"
    set /a "k=i/parz, n=k+1, rapp=(i-k*parz)*40/parz, d=40-rapp, vel=I*100/a"
    for %%k in (!k!) do set LL0=!L%%k!
    for %%n in (!n!) do set LL1=!L%%n!
    for %%r in (!rapp!) do for %%d in (!d!) do <nul set /p "=^<!LL1:~0,%%r!!LL0:~0,%%d!^> !i!/!Max! - !vel!Bytes/s !CR!"
  )
)

:: Final Bytes
for /L %%B in (!I!, 1, %~z1) do (
    set /a "a=(CRC32 ^ 0x40) & 0xff"
    for /L %%i in (0,1,7) do (
   set /a "t=a & 1"
   if !t! equ 1 (set /a "a=((a>>1)&0x7FFFFFFF)^Poly") else set /a "a=((a>>1)&0x7FFFFFFF)"
    )
    set /a "CRC32 = ((CRC32 >> 8)&0x00FFFFFF) ^ a"
)

:: Final operations
set t=
set a=
set /a "CRC32 = CRC32 ^ 0xffffffff"

:: Convert to Hexdecimal
call cmd /c exit /b !CRC32!
set crc32=!=ExitCode!

:: Print out result.
echo(
echo CRC32:!CRC32!


IF NOT "%~2"=="" IF !SM! equ 1 (ENDLOCAL& set CRC32=%CRC32%) else (ENDLOCAL& set %~2=%CRC32%)

exit /b


Einstein1969
Last edited by einstein1969 on 19 Jun 2014 08:36, edited 4 times in total.

penpen
Expert
Posts: 2009
Joined: 23 Jun 2013 06:15
Location: Germany

Re: CRC32 Problem with this code

#2 Post by penpen » 02 Dec 2013 18:45

You have just missed 0x prior to hex values, when setting up b:

Code: Select all

:: ...
:loop1
  set b=0x%strhex:~0,2%
:: ...

penpen

einstein1969
Expert
Posts: 960
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

Re: CRC32 Problem with this code

#3 Post by einstein1969 » 03 Dec 2013 04:57

penpen wrote:You have just missed 0x prior to hex values, when setting up b:

Code: Select all

:: ...
:loop1
  set b=0x%strhex:~0,2%
:: ...

penpen


many thanks I was going crazy! :D


Einstein1969

einstein1969
Expert
Posts: 960
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

Re: CRC32 Problem with this code

#4 Post by einstein1969 » 18 Jun 2014 15:11

Hi, I have complete rewritten the code.

Now is a complete batch.

Hope anyone test this for find error.

einstein1969

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: CRC32

#5 Post by foxidrive » 19 Jun 2014 01:45

Scaling seems a bit wrong - it also took over 30 seconds to display anything on the console.

Image

It seems that filenames with ! are not supported.

The CRC32 are all correct on the files i tried.

This display is the end of calculation and the filesize figures don't match, FWIW.
It took around 30 minutes if you're interested.

Code: Select all

<███████████████████████████████████████▓> 3178000/3178372 - 1843Bytes/s
CRC32:E8317B69

einstein1969
Expert
Posts: 960
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

Re: CRC32

#6 Post by einstein1969 » 19 Jun 2014 08:42

foxidrive wrote:Scaling seems a bit wrong - it also took over 30 seconds to display anything on the console.

Image

It seems that filenames with ! are not supported.

The CRC32 are all correct on the files i tried.

This display is the end of calculation and the filesize figures don't match, FWIW.
It took around 30 minutes if you're interested.

Code: Select all

<███████████████████████████████████████▓> 3178000/3178372 - 1843Bytes/s
CRC32:E8317B69



Thank you very much foxidrive.

- I have probed to fix the ! problem. I'm not good at this. I hope it's ok.
- I have optimized the performance. Now is faster.
- When wait for create a big file now visualize a "wait..."
- I have fixed the final progress bar.

The new and the old version is on first Post. I hope having fixed all.

einstein1969

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: CRC32

#7 Post by foxidrive » 19 Jun 2014 12:57

Yes, it solves all the things I mentioned in this test, and is a lot faster.

Code: Select all

Wait...
<██████████████████████████████████████████████████> 100% - 6094B/s
CRC32:E8317B69
0 days 00:09:26.844

Bam137
Posts: 1
Joined: 08 Jul 2020 01:13

Re: CRC32

#8 Post by Bam137 » 08 Jul 2020 01:29

Hi, I’m Bam, i’m so sorry that i’m not quite expert on this. i just want to ask, is it this coding created into the “.bat” file? I have task to do to get the File CRC from multiples PDF files in a folder & added the CRC numbers to the their respective PDF filename. Perhaps it can be done using a Batch File (.bat). I need helps 🙏🏼.

Eg: Input Folder : FILENAME_SAMPLES.pdf
Output Folder : FILENAME_SAMPLES_CRCnumbers.pdf

Appreciate your helps
Thanks
Bam

Post Reply