Create nul and all ascii characters with only batch

[dbenham]

Awesome stuff And great teamwork

I never dreamed there would be such an elegant hack that would work even on native XP. Lots of clever techniques all packed into one solution.


Dave Benham

[carlos]

@dbenham.
Is posible adapt charlib for create all the ascii characters in 256 files.
Nul is ready.
What are the most problematic characters that are corrupted on posted?

If we create the 256 characters in a file, we can create binary files using only batch.

[dbenham]

@Carlos - Yes, I was thinking about that.

The existing CHARLIB.BAT was intended to work with variables, and of course that will not work with null bytes. So when dealing with null bytes it could only be used to write to a file.

Most byte characters could be written using SET /P, but there is the leading equal/quote/white space issue. Those problem characters would require jeb's temp file <Ctrl-Z> technique.

One thing I don't like about CHARLIB.BAT is many of the characters are hard coded within the file. In particular, I don't like the <tab> character hard coded. My text editor is frequently configured to convert tabs into spaces, and also tabs often do not post well to bulletin boards. Unfortunately, no one figured out a universal way to generate <tab> with batch. Also, any of the extended ASCII characters (byte code greater than 127) can be a problem because you have to worry about the active code pages.

Because of those issues, CHARLIB.BAT must basically be treated as a binary file, not typical for batch scripts

Given that CHARLIB.BAT is already binary, I don't see any reason why we couldn't also embed the null byte as well.

As long as all lines are terminiated by <carriage return><line feed>, then simply append a single null byte after the last line. The null byte does not interfere with the operation of the script. FINDSTR can be used to write out the null byte using:

Code: Select all

findstr /v $ "%~f0" >outputFile

I think I saw the above technique on StackOverflow when following the links associated with the recent DosTips activity dealing with batch binaries.


Dave Benham

[foxidrive]

no one figured out a universal way to generate <tab> with batch.

This works in Vista/Win7/Win 8.1

The forfiles that I downloaded in XP days doesn't have the same syntax as Vista and later version so I added a kludge for XP.

Code: Select all

@echo off
set "t=~.bat"
type nul>"%temp%\%t%"
set "space= "
ver | find /i "XP" >nul && set "space="
forfiles -m%space%"%t%" -p%space%"%temp%" -c%space%"cmd /c set /p =set tab=0x09<nul">"%temp%\%t%"
call "%temp%\%t%"
del "%temp%\%t%"
echo "%tab%"
pause

[carlos]

I have this stuff code. It create the files for: 0, 8, 10, 13, 26, 27 ascii characters:
I working for create more characters.

Code: Select all

@Echo Off
Set "foo=%Temp%\foo.foo"



Call :0
Call :8
Call :10
Call :13
Call :26
Call :27

Goto :Eof


:0
::This routine create the file 0.chr that only have the nul character
::Tested on windows 8, winxp 32 home, win xp 32 prof, windows 7 32bit
::Authors: carlos, aGerman, penpen
Cmd /U /C Set /P "=a" <Nul > 0.chr
Copy /Y 0.chr+Nul 0.chr >Nul
Type 0.chr |(Pause>Nul &Findstr "^") > "%foo%"
Copy /Y "%foo%" /A 0.chr /B >Nul
Goto :Eof

:8
For /F %%# In (
'"Prompt $H &For %%_ In (_) Do Rem"') Do Set /P "=%%#" <Nul >8.chr
Goto :Eof

:10
Echo(|(Pause >Nul &Findstr "^" >10.chr)
Goto :Eof

:13
Call :26
For /F %%# In (26.chr) Do Set "sub=%%#"
Echo(>"%foo%"
For /F %%# In ('Copy /Z "%foo%" Nul') Do Echo(%%#%sub%>"%foo%"
Copy /Y "%foo%" /A 13.chr /B >Nul
Goto :Eof

:26
Copy /Y Nul+Nul /A 26.chr /A >Nul
Goto :Eof

:27
For /F %%# In (
'"Prompt $E &For %%_ In (_) Do Rem"') Do Set /P "=%%#" <Nul >27.chr
Goto :Eof

[penpen]

Actually i'm posting by my mobile phone so i can't test it, but couldn't we use the makecab.exe:
I think i remember it contains its own filesize somewhere in the header (in little endian).
So creating a non compressed cab with the right filesize (and name) should serve us the needed hex codes.
A filesize of 0x1AFF should serve us a header entry of "FF1A" so we could do the same trick as above to extract this hex FFchar.
So we need to create cab filesizes of 0x1A01 to 0x1AFF.

penpen

Edit: If i'm wrong with the cab filesize, then the filesize of the (un)compressed must be present somewhere in the cab, same trick, but not as near to the start of the file.

[foxidrive]

The same technique I posted earlier will also work for an FF character.

It may work for most 256 characters

Code: Select all

@echo off
set "t=~.bat"
type nul>"%temp%\%t%"
set "space= "
ver | find /i "XP" >nul && set "space="
forfiles -m%space%"%t%" -p%space%"%temp%" -c%space%"cmd /c set /p =set FF=0xff<nul">"%temp%\%t%"
call "%temp%\%t%"
del "%temp%\%t%"
echo "%FF%"
pause

[aGerman]

I think i remember it contains its own filesize somewhere in the header (in little endian).

Good idea. It can be found at the 9th and 10th Byte. The rest is a bit tricky though. Also the file name belongs to the header and influences the size of the cab file.

Regards
aGerman

EDIT:
tab character:

Code: Select all

@echo off &setlocal
>"t.tmp" type nul
for /l %%i in (1 1 169) do >>t.tmp <nul set /p "=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
>nul makecab /d compress=off "t.tmp" "t.chr"
type "t.chr" | >"t.tmp" (pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&findstr "^")
>nul copy /y "t.tmp" /a "t.chr" /b
del "t.tmp"

As soon as you change the name t.tmp to a different length you also have to change the number of characters written to t.tmp.

EDIT2: Changed t.chr to t.tmp because I confused myself

[Squashman]

The forfiles that I downloaded in XP days doesn't have the same syntax as Vista and later version so I added a kludge for XP.

Gotta make sure you get the one that 2003 server uses. Don't use the one for NT/2000 which uses the hyphen syntax. I posted a link to download it a few months ago.

[carlos]

Actually i'm posting by my mobile phone so i can't test it, but couldn't we use the makecab.exe:
I think i remember it contains its own filesize somewhere in the header (in little endian).
So creating a non compressed cab with the right filesize (and name) should serve us the needed hex codes.
A filesize of 0x1AFF should serve us a header entry of "FF1A" so we could do the same trick as above to extract this hex FFchar.
So we need to create cab filesizes of 0x1A01 to 0x1AFF.

penpen

Edit: If i'm wrong with the cab filesize, then the filesize of the (un)compressed must be present somewhere in the cab, same trick, but not as near to the start of the file.

Nice idea penpen.
Without talk about the cabinet format:
If you do this:

Code: Select all

Makecab /D Compress=OFF file.dat file.cab

int this case the file.cab should have the uncompressed size in offset 44: the size as you say: Little endian 4 bytes.
For get the 1A at right we need a file as base of 6656 bytes: this is 0x1A00 and in Little endian: 00 1A

In simple idea:
create a file with 6656 bytes.
Skip 44 bytes and now we have the ...

But there are a litte problem. And we need two strategies.
Because in cabinet also file offset 8 have 4 bytes with the size of the cabinet.
The minimal cabinet size that generate:

Code: Select all

Makecab /D Compress=OFF a file.cab

with a empty file with a length filename of 1, ex: a
is 62 bytes.
then if we create a cabinet of file with 6656 bytes with a length filename of 1, the cabinet size would be 6718 bytes, in Little endian 3E 1A
and we have this 1A in offset 9 stopping our work.

Also, cutoff the cabinet in the 1A is needed for avoid a findstr error message "too large line".

I working on this.

[miskox]

I have been following this thread from the beginning. I am speechless! You have so many ideas and knowledge.

And now to the point:
- aGerman's solution to get TAB does not work for me (XP PRO)
- here is my code (modified original Carlo's work):

Code: Select all

@Echo Off

REM This will create a file with the size of 6665 bytes.
>a.a (echo AAA)
for /L %%f in (1,1,2220) do >>a.a (echo A)

Set "p=Pause>Nul"
Set "q=%p%&%p%&%p%&%p%&%p%&%p%&%p%&%p%&%p%&%p%&%p%"
Set "s=%q%%q%%q%%q%&%p%&%p%&%p%"

Makecab /D Compress=OFF a.a a.cab >Nul
Type a.cab |(%s%&Findstr "^") >wnul.tmp

Copy /Y wnul.tmp /A tab.chr /B >Nul
Del a.cab a.a wnul.tmp

Rem tab.chr has the TAB character

Saso

[aGerman]

- aGerman's solution to get TAB does not work for me (XP PRO)

I'm wondering why Could you tell me what values the 9th and 10th Bytes in t.chr are if you write a pause after the makecab line?

- here is my code (modified original Carlo's work):

Works for me even if it seems to be buggy. The %q%'s are not concatenated via &. The same in carlos' initial post. Maybe someone could explain what I miss here.

Regards
aGerman

[aGerman]

then if we create a cabinet of file with 6656 bytes with a length filename of 1, the cabinet size would be 6718 bytes, in Little endian 3E 1A
and we have this 1A in offset 9 stopping our work.

Also, cutoff the cabinet in the 1A is needed for avoid a findstr error message "too large line".

Hmm, won't that work for you:
EDIT Doesn't work under XP, please use http://www.dostips.com/forum/viewtopic.php?p=32169#p32169

Code: Select all

@echo off &setlocal EnableDelayedExpansion
md "characters"
pushd "characters"

>"t.tmp" <nul set /p "=a"
for /l %%i in (1 1 140) do >>t.tmp <nul set /p "=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

set "strMap=0123456789ABCDEF"
for /l %%i in (0 1 255) do (
  set "byte=%%i" &set "strHex="
  for /l %%j in (1 1 2) do (
    set /a "x = byte & 15, byte >>= 4"
    for %%k in (!x!) do set "strHex=!strMap:~%%k,1!!strHex!"
  )
  >>t.tmp <nul set /p "=a"
  >nul makecab /d compress=off "t.tmp" "!strHex!.chr"
  type "!strHex!.chr" | >"temp.tmp" (pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&pause>nul&findstr "^")
  >nul copy /y "temp.tmp" /a "!strHex!.chr" /b
)

REM carlos' bug fix for the SUB character
Copy /Y Nul+Nul /A 1A.chr /A >Nul

del "t.tmp" "temp.tmp"
popd
pause

Of course the SUB itself can't be created that way (and I did'n look into each file if the right byte was written) but it was running without any error.

Regards
aGerman

EDIT: SUB fixed.
EDIT2: Hex file names instead of decimal.

[carlos]

Nice code aGerman.
Please add this fix for the sub chr:

Code: Select all

Rem Fix for 26.chr
Copy /Y Nul+Nul /A 026.chr /A >Nul

I would check if all the .chr generated with yout code are right.
I write a code using a cabinet with two files inside, but currently my code have problem with the 178 character.

Edit: All the characters generated with aGerman code are right, except 026.chr (it need the fix). I check it with a c program.

Now, we can write binary data using batch.
The hexdump of dbenham output in hexadecimal.
aGerman, please you can create the character on hexadecimal notation with a width of 2, example: 0F.chr

Now, is only time for write a tool that write binary using only batch.

This is a great advance in the batch.

[aGerman]

Fixed Nice teamwork indeed

Since the tab creation didn't work for Saso I'm really curious to know if that code would work under XP.

Regards
aGerman

aGerman, please you can create the character on hexadecimal notation with a width of 2, example: 0F.chr

Yes I can. I'll edit the code above as soon as I finished it.
EDIT: OK, done