Create nul and all ascii characters with only batch

[carlos]

@aGerman thanks. I ended and post my version. I write it using a cabinet file with two files inside (the second file was for change the cabinet file size), because from the begin I think that the 1a from cabinet size would be a problem.

edit:
code removed because better improvements were done.

[carlos]

@aGerman: I test both codes on xp, and your code not Works on xp spanish, it says: FINDSTR: the line 1 is too large. My code works on xp because I cut off the cabinet in the 1a character and then adding it to end of file. Please, you can fix your code? it would be great.

[dbenham]

Not sure if aGerman's code can be fixed.

FINDSTR is limited to 8191 bytes per line when reading piped or redirected data. There is no line length limit if FINDSTR opens the file directly.

If the line of interest is always less then 8192 bytes, then you can ignore the error. But if the line of interest might exceed 8192 bytes then the code would have to be modified to not use a pipe into FINDSTR.


Dave Benham

[aGerman]

OK, I think it should be possible with an additional copy. Please try again.

Code: Select all

@echo off &setlocal EnableDelayedExpansion
REM This code creates 256 files containing one single Byte each from 0x00 until 0xFF.
REM Teamwork of carlos, penpen and aGerman
REM Tested under XP, Win7, Win8
2>nul md "characters"
pushd "characters"

>"t.tmp" <nul set /p "=a"
for /l %%i in (1 1 140) do >>t.tmp <nul set /p "=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
>nul copy /y nul + nul /a "sub.chr" /a

set "strMap=0123456789ABCDEF"
for /l %%i in (0 1 255) do (
  set "byte=%%i" &set "strHex="
  for /l %%j in (1 1 2) do (
    set /a "x = byte & 15, byte >>= 4"
    for %%k in (!x!) do set "strHex=!strMap:~%%k,1!!strHex!"
  )
  >>t.tmp <nul set /p "=a"
  >nul makecab /d compress=off "t.tmp" "temp.tmp"  
  >nul copy /y "temp.tmp" /a + "sub.chr" /b "!strHex!.chr" /b
  type "!strHex!.chr" | >"temp.tmp" (pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&findstr "^")
  >nul copy /y "temp.tmp" /a "!strHex!.chr" /b
)

>nul move /y "sub.chr" "1A.chr"
del "t.tmp" "temp.tmp"
popd

Regards
aGerman

[carlos]

@aGerman: now it Works on win xp, also win 8. Please add a note to the old code for avoid get incompatible code.
I would add a link to your code on my post code.
thanks for the code aGerman, also to penpen for give the idea.

[aGerman]

Thanks for testing. I linked to the new code as you suggested.

Regards
aGerman

[carlos]

thanks aGerman. I'm thinking that is great this code, because with this we can create any binary data using only batch.
In the past the way for do this was unknowed.

Is impressive for me that because makecab is present from Windows 2000, this means that create all the ascii characters was possible from 13 years ago, but only in this days the technique born and also the code for do it and with so few lines.

I'm happy with this code.

[carlos]

@aGerman. I write a little optimization (write to t.tmp in each loop removed).
This uses the directive reservepercabinetsize that is a reserved space in the cabinet for a application purpose (i learn this when I study the cabinet format), I think that the directive was not available, but yes. Tested ok on xp and win8.

Optimized using directive reservepercabinetsize:

Code: Select all

@echo off &setlocal EnableDelayedExpansion
REM This code creates 256 files containing one single Byte each from 0x00 until 0xFF.
REM Teamwork of carlos, penpen and aGerman
REM Optimized using directive reservepercabinetsize
REM Tested under XP, Win7, Win8
2>nul md "characters"
pushd "characters"

>"t.tmp" type nul
>nul copy /y nul + nul /a "sub.chr" /a

set "p=6586"
set "strMap=0123456789ABCDEF"
for /l %%i in (0 1 255) do (
  set "byte=%%i" &set "strHex="
  for /l %%j in (1 1 2) do (
    set /a "x = byte & 15, byte >>= 4"
    for %%k in (!x!) do set "strHex=!strMap:~%%k,1!!strHex!"
  )
  >nul makecab /d compress=off /d reservepercabinetsize=!p! "t.tmp" "temp.tmp"
  >nul copy /y "temp.tmp" /a + "sub.chr" /b "!strHex!.chr" /b
  type "!strHex!.chr" | >"temp.tmp" (pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&findstr "^")
  >nul copy /y "temp.tmp" /a "!strHex!.chr" /b
  set /a "p+=1"
)

>nul move /y "sub.chr" "1A.chr"
del "t.tmp" "temp.tmp"
popd

[foxidrive]

The forfiles that I downloaded in XP days doesn't have the same syntax as Vista and later version so I added a kludge for XP.

Gotta make sure you get the one that 2003 server uses. Don't use the one for NT/2000 which uses the hyphen syntax. I posted a link to download it a few months ago.

Thanks for your reply Squashman.

I have an XP VM and the version in that must be the old one - I wonder how widespread that version is?
While I was testing I looked for at Microsoft for a download link and the search didn't return any fits for forfiles or forfiles.exe which is interesting.

It's academic anyway because I no longer use XP, and with April and the end of support for XP coming, it may not be too long before the people with XP upgrade.

[foxidrive]

this means that create all the ascii characters was possible from 13 years ago, but only in this days the technique born and also the code for do it and with so few lines.

Back in MSDOS and Win9x days Qbasic was installed and you could write any binary code. You can do it these days with VBS too.

Is there some advantage to doing it with makecab?

[npocmaka_]

Is there some advantage to doing it with makecab?

WSh can be forbidden by the administrator (but mshta is almost always an option)...

[Sponge Belly]

Hello All!

Wow! What a fascinating thread. Allow me to tie up a few loose ends.

I thought Penpen’s final version of CreateNul.cmd was definitive… and then I saw Carlos’s optimisation of my snippet. I’m going to be controversial and call it a draw!

Squashman, I think this is the post on forfiles you mentioned.

Dave Benham discovered a way to capture Form Feed in a variable back in February, 2012.

And here's an mshta solution for capturing a tab in an environment variable:

Code: Select all

for /f "delims=" %%t in ('mshta ^"javascript:close(new ^
ActiveXObject('Scripting.FileSystemObject'^).^
GetStandardStream(1^).Write(^"\x09^"^)^)^"') do set tab=%%t
echo(words%tab%separated%tab%by%tab%tabs

Lastly, could someone please explain this makecab voodoo solution? I don’t understand it. Did you somehow manage to reverse-engineer a file that, when unpacked by makecab, creates 256 one-byte files?

- SB

[aGerman]

Optimized using directive reservepercabinetsize

Great improvement

Is there some advantage to doing it with makecab?

Not at all. It's just another challange to use any possibility, each bug, and every undefined behavior of the Windows console tools to achieve results that were said to be impossible. If I would need to write a robust code with a good performance I would certainly write it in a language that supports these things without any tinkering.

Lastly, could someone please explain this makecab voodoo solution?

If you want to understand it you should open a cab file in a HEX editor. The code above makes use of the fact that the size of a cab file is always saved in the file header.
Imagine you would like to create a TAB character:

Code: Select all

>"test.txt" type nul
>nul makecab /d compress=off /d reservepercabinetsize=6590 "test.txt" "test.cab"

These two lines of code create an empty file that will be packed in a cab file with reserved space. The resulting size of the cab file is 6665 bytes.
The hexadecimal equivalent of 6665 is 1A09. If you open the cab file in a HEX editor you will see the following:

Code: Select all

4D53 4346 0000 0000 091A 0000 0000 0000 ...

Two HEX digits represent one byte. The 9th and 10th bytes are 091A which is the file size (1A09) in little endian order. The ASCII representations of these bytes are TAB (0x09) and SUB (0x1A).
Now you need techniques to extract the TAB.
- COPY /A copies a file in ASCII manner. A peculiarity of COPY /A is that it stops reading/copying the file at the first found SUB character.
- TYPE "file" | (PAUSE>NUL &FINDSTR "^") eats a single byte from the beginning of a file (but appends a linebreak that we have to remove later). The more PAUSE>NUL you concatenate the more bytes are stripped. We need to remove 8 bytes for reaching the byte of interrest in the cab file.

As you can see if you increase or decrease the size of the cab file you are able to extract each possible byte.

Regards
aGerman

[carlos]

Reanalyzing the cabinet format I found a really good improvement that will avoid the creation of big cabinet file in each loop.

aGerman: please check this:

Code: Select all

makecab /d compress=off /d reserveperfoldersize=0 /d reserveperdatablocksize=26 t.tmp t.cab

because in the cabinet format this part:

u1 cbCFFolder; /* (optional) size of per-folder reserved area */
u1 cbCFData; /* (optional) size of per-datablock reserved area */

are a only 1 byte. This means that using reserveperdatablocksize=26 I putting directly the 1A chracter after the character specified in reserveperfoldersize
reserveperfoldersize=0 and reserveperdatablocksize=26 would generate 001A
and:
reserveperfoldersize=1 and reserveperdatablocksize=26 would generate 011A

Then we can get the ascii character in the offset 38.

[penpen]

Sorry, yesterday i had no time , you've done a great work!
But nevertheless i tried to optimize the code (only a little bit: i hope you like it although it should be tested another time):

Code: Select all

@echo off &setlocal EnableDelayedExpansion
REM This code creates :
REM - 256 files containing one single Byte each from 0x00 until 0xFF, and
REM - a file table.dat that contains [0x20][0x01 ... 0xFF] for easy access.
REM Teamwork of carlos, penpen and aGerman
REM Optimized using directive reservepercabinetsize
REM Tested Win7 home (64)

2>nul md "characters"
pushd "characters"

set "p=6586"
>"t.tmp" type nul
>nul copy /y nul + nul /a "1A.chr" /a
for %%h in (0 1 2 3 4 5 6 7 8 9 A B C D E F) do for %%l in (0 1 2 3 4 5 6 7 8 9 A B C D E F) do (
   if NOT "%%~h%%~l" == "1A" (
      >nul makecab /d compress=off /d reservepercabinetsize=!p! "t.tmp" "temp.tmp"
      >nul copy /y "temp.tmp" /a + "1A.chr" /b "%%~h%%~l.chr" /b
      type "%%~h%%~l.chr" | >"temp.tmp" ((for /L %%b in (1,1,8) do pause)>nul & findstr "^")
      >nul copy /y "temp.tmp" /a "%%~h%%~l.chr" /b
   )
   set /a "p+=1"
)
del "t.tmp" "temp.tmp"

>nul copy "20.chr" "table.dat"
for %%l in (1 2 3 4 5 6 7 8 9 A B C D E F) do (>nul copy "table.dat" /b + "0%%~l.chr" /b + "table.dat" /b)
for %%h in (1 2 3 4 5 6 7 8 9 A B C D E F) do for %%l in (0 1 2 3 4 5 6 7 8 9 A B C D E F) do (>nul copy "table.dat" /b + "%%~h%%~l.chr" /b + "table.dat" /b)

popd

In addition i have added the table.dat file, that content is [0x20, 0x01, ...0xFF], so you can read all bytes (except the null, which i've replaced by a space char: 0x20) using:

Code: Select all

set "table="
set /P "table=" < table.dat

The codes 0x01 to 0x1F were problematic under my XP x64; haven't tested it anywhere else.
Actually i'm using a win7 home 64 version here.

penpen

Edit: Shortened the cration of the table.dat file.