Create nul and all ascii characters with only batch

[foxidrive]

in the past when I post some code to alt.msdos.batch.nt it add a space to the end of every line of the code

I presume you posted using Google.

Google *really* mangles Usenet posts - but some Usenet clients (Thunderbird) also mangles some text.

Most Usenet clients work well.



Good work on creating the hex 00 there, Sponge Belly.

[miskox]

I hereby nominate Carlos for promotion to Expert. Any seconds?

+1.

Saso

[carlos]

I develop a way for increase the speed of creation of the all 255 .chr files.
This code uses function :gen255 that use asynchronus macro for generate the .chr files.
Because, the task are asynchronus, the code uses a loop for wait until the .chr file have 1 byte (that is when it is ready for use).

Code: Select all

@echo off

:gen255
::This code creates one single byte. Parameter: int
::This is encoded in a macro way for be called with cmd /c
::Teamwork of carlos, penpen, aGerman, dbenham
::Tested under Win8
set "genchr=if not %%#==26 (type nul >%%#.tmp"
set "genchr=%genchr% &makecab /d compress=off"
set "genchr=%genchr% /d reserveperdatablocksize=26"
set "genchr=%genchr% /d reserveperfoldersize=%%#"
set "genchr=%genchr% %%#.tmp %%#.chr >nul"
set "genchr=%genchr% &type %%#.chr"
set "genchr=%genchr% |((for /l %%_ in (1 1 38) do pause)>nul"
set "genchr=%genchr% &findstr ^^^^ >%%#.tmp)"
set "genchr=%genchr% &copy /y %%#.tmp /a %%#.chr /b >nul"
set "genchr=%genchr% &del %%#.tmp) else"
set "genchr=%genchr% (copy /y nul + nul /a 26.chr /a >nul)"
::end of genchr macro::
del /f /q /a *.chr >nul 2>&1
for /l %%# in (0,1,255) do start "" /b /i cmd /d /c "%genchr%"
set "check=0"
:check255
for /l %%# in (%check%,1,255) do for %%_ in (
%%#.chr) do if not "1"=="%%~z_" (
ping -l 0 -n 1 -w 100 1.0.0.0 >nul
set "check=%%#"
goto :check255)
goto :eof

[dbenham]

Nice work Carlos.

I took your idea and optimized it some more. I was able to basically double the speed yet again

1) There is no point in creating more processes than you have processors (or cores)

2) Better to launch the last process synchronously. Then you don't have to begin checking if done until the last one completes. Each process is doing basically the same amount of work, so they should all end about the same time. So I don't bother putting a delay in the wait loop since I am confident it won't be running for long.

The above points don't really improve speed, but they do minimize the number of processes that the OS has to keep track of.

3) Most of the printable ASCII characters can be generated much more quickly using CMD /C EXIT n to set %=exitCodeAscii%. This saves a lot of time.

Code: Select all

@echo off
setlocal

:genAllChr
::This code creates 256 1 byte files, one for each possible byte value.
::This is encoded in a macro way to be called asynchronously with start cmd /c
::Teamwork of carlos, penpen, aGerman, dbenham
::Tested under Win7 and XP
set ^"genchr=(^
for /l %%N in (%%A !cnt! 255) do (^
  if %%N equ 26 (^
    copy /y nul + nul /a 26.chr /a ^>nul^
  ) else (if %%N geq 35 if %%N leq 126 if %%N neq 61 (^
    cmd /c exit %%N^&^
    ^<nul set /p "=!=exitCodeAscii!" ^>%%N.chr^
  ))^&^
  if not exist %%N.chr (^
    makecab /d compress=off /d reserveperdatablocksize=26 /d reserveperfoldersize=%%N %%A.tmp %%N.chr ^>nul^&^
    type %%N.chr ^| ((for /l %%n in (1 1 38) do pause)^>nul^&findstr "^^" ^>%%N.temp)^&^
    ^>nul copy /y %%N.temp /a %%N.chr /b^&^
    del %%N.temp^
  )^
))^&^
del %%A.tmp^"
del /f /q /a *.chr *.tmp >nul 2>&1
set /a cnt=number_of_processors
if %cnt% lss 1 set cnt=1
if %cnt% gtr 256 set cnt=256
set /a "end=cnt-1"
for /l %%A in (0 1 %end%) do (
  type nul >%%A.tmp
  if %%A equ %end% (
    start "" /b /wait cmd /q /v:on /c "%genchr%"
  ) else (
    start "" /b cmd /q /v:on /c "%genchr%"
  )
)
:genAllChr.check
if exist *.tmp goto :genAllChr.check
exit /b

Here are some timings I got comparing your original code and my modified version.

Code: Select all

                          Carlos' original       My modifid version
                        -------------------     --------------------
Win7 64bit               7.93 7.47 7.12          3.77 3.72 3.79
4 processors (cores)     Avg = 7.51              Avg = 3.76

XP virtual machine       57.73 57.92 57.50       35.71 35.93 36.13
1 processor              Avg = 57.72             Avg = 35.92

Dave Benham

[carlos]

dbenham: thanks for the tips. I not know why, your posted code generates really faster the .chr files, but the script not end.
I will take some of your ideas using ExitCodeAscii and post a new code.

[dbenham]

your posted code generates really faster the .chr files, but the script not end.

That is really weird. It works perfectly for me, and the code seems so straight forward. I'm eager to hear what the problem is.


Dave Benham

[foxidrive]

Dave's code ends here ok.

I got these timings:

1.01
1.11
1.11

[carlos]

Dave, thinking about you say all process should end the work at same point: I remember write that function because without it i get some messages like: is not posible access to the file. try rebuild a binary file with the generated.

Edit: i found the cause in your code the waiting forever. this portions:

Code: Select all

:genAllChr.check
if exist *.tmp goto :genAllChr.check

why? I don't now why: In my current test folder i have strange hidden folders:

Code: Select all

09-02-2014  07:09    <DIR>          0529b7c51f51499b97790158a5c048e0$dpx$.tmp
09-02-2014  07:09    <DIR>          0f9a6cef33234a339777cc8c7a6ac410$dpx$.tmp
09-02-2014  22:29    <DIR>          51028f328af346c7877d778b7cd1a38e$dpx$.tmp
04-02-2014  23:59    <DIR>          7d7efd315de74dc195a48db87689949e$dpx$.tmp
09-02-2014  07:09    <DIR>          fe4c2a5674f04b898a40277759970f75$dpx$.tmp

deleted it, the code ends ok.

[carlos]

About my code, I implemente your trick with =exitcodeascii, and also for check that all .chr files are ready I removed the goto loop system, using a function that uses for /l.

please, check time with this: (in my system it takes round 6 seconds):

Code: Select all

@echo off

:gen255
::generates 0-255 .chr files that have 1 byte
::Teamwork of carlos, penpen, aGerman, dbenham
::Tested under Win8

set "func1=type nul >%%#.tmp"
set "func1=%func1% &makecab /d compress=off"
set "func1=%func1% /d reserveperdatablocksize=26"
set "func1=%func1% /d reserveperfoldersize=%%#"
set "func1=%func1% %%#.tmp %%#.chr >nul"
set "func1=%func1% &type %%#.chr"
set "func1=%func1% |((for /l %%_ in (1 1 38) do pause)>nul"
set "func1=%func1% &findstr ^^^^ >%%#.tmp)"
set "func1=%func1% &copy /y %%#.tmp /a %%#.chr /b >nul"
set "func1=%func1% &del %%#.tmp"
set "func2=cmd /c exit /b %%# &cmd /a /c set /p"
set "func2=%func2% =^^%%=exitcodeascii%%<nul >%%#.chr"
set "func3=for /l %%- in (0) do if "1"=="%%~z_" exit /b"

del /f /q /a *.chr >nul 2>&1
for /l %%# in (0,1,25) do start "" /b /i cmd /d /c "%func1%"
copy /y nul + nul /a 26.chr /a >nul
for /l %%# in (27,1,34) do start "" /b /i cmd /d /c "%func1%"
for /l %%# in (35,1,60) do start "" /b /i cmd /d /c "%func2%"
for %%# in (61) do %func1%
for /l %%# in (62,1,126) do start "" /b /i cmd /d /c "%func2%"
for /l %%# in (127,1,255) do start "" /b /i cmd /d /c "%func1%"

for /l %%# in (0,1,255) do for %%_ in (
%%#.chr) do cmd /q /d /c "%func3%"

goto :eof

[foxidrive]

Time here for Carlos' version

2.16
2.33
2.31

[dbenham]

Hidden .tmp files would do it.

I took Carlos' idea and explicitly check for the existence of each .tmp file that my code creates using a FOR /L loop. I no longer del *.tmp at the top. I also removed the START /B /WAIT from the last process - simpler to use CMD /C directly.

Code: Select all

@echo off
setlocal disableDelayedExpansion

:genAllChr
::This code creates 256 1 byte files, one for each possible byte value.
::This is encoded in a macro way to be called asynchronously with start cmd /c
::Teamwork of carlos, penpen, aGerman, dbenham
::Tested under Win7 and XP
set ^"genchr=(^
for /l %%N in (%%A !cnt! 255) do (^
  if %%N equ 26 (^
    copy /y nul + nul /a 26.chr /a ^>nul^
  ) else (if %%N geq 35 if %%N leq 126 if %%N neq 61 (^
    cmd /c exit %%N^&^
    ^<nul set /p "=!=exitCodeAscii!" ^>%%N.chr^
  ))^&^
  if not exist %%N.chr (^
    makecab /d compress=off /d reserveperdatablocksize=26 /d reserveperfoldersize=%%N %%A.tmp %%N.chr ^>nul^&^
    type %%N.chr ^| ((for /l %%n in (1 1 38) do pause)^>nul^&findstr "^^" ^>%%N.temp)^&^
    ^>nul copy /y %%N.temp /a %%N.chr /b^&^
    del %%N.temp^
  )^
))^&^
del %%A.tmp^"
del /f /q /a *.chr >nul 2>&1
if "%~1" neq "" (set /a cnt=%~1) else set /a cnt=number_of_processors
if %cnt% lss 1 set cnt=1
if %cnt% gtr 256 set cnt=256
set /a "end=cnt-1"
for /l %%A in (0 1 %end%) do (
  type nul >%%A.tmp
  if %%A equ %end% (
    cmd /q /v:on /c "%genchr%"
  ) else (
    start "" /b cmd /q /v:on /c "%genchr%"
  )
)
:genAllChr.check
for /l %%N in (0 1 %end%) do if exist %%N.tmp goto :genAllChr.check
exit /b

I don't see any difference in timing from before.

In production code I would probably create a temporary directory in the %TEMP% folder to hold all the working files. That folder would be guaranteed to be empty at the start. Then I don't even need to delete *.CHR, and I can revert back to a simple IF EXIST *.TMP at the end.


Dave Benham

[Squashman]

Dave's code ends here ok.

I got these timings:

1.01
1.11
1.11

I think someone just wanted to brag that he has a faster computer than all of us.

[foxidrive]

I think someone just wanted to brag that he has a faster computer than all of us.

*laughs*

I bought it second hand 20 months ago - $600 well spent.

[einstein1969]

Hidden .tmp files would do it.

I took Carlos' idea and explicitly check for the existence of each .tmp file that my code creates using a FOR /L loop. I no longer del *.tmp at the top. I also removed the START /B /WAIT from the last process - simpler to use CMD /C directly.

Code: Select all

@echo off
setlocal disableDelayedExpansion

:genAllChr
::This code creates 256 1 byte files, one for each possible byte value.
::This is encoded in a macro way to be called asynchronously with start cmd /c
::Teamwork of carlos, penpen, aGerman, dbenham
::Tested under Win7 and XP
set ^"genchr=(^
for /l %%N in (%%A !cnt! 255) do (^
  if %%N equ 26 (^
    copy /y nul + nul /a 26.chr /a ^>nul^
  ) else (if %%N geq 35 if %%N leq 126 if %%N neq 61 (^
    cmd /c exit %%N^&^
    ^<nul set /p "=!=exitCodeAscii!" ^>%%N.chr^
  ))^&^
  if not exist %%N.chr (^
    makecab /d compress=off /d reserveperdatablocksize=26 /d reserveperfoldersize=%%N %%A.tmp %%N.chr ^>nul^&^
    type %%N.chr ^| ((for /l %%n in (1 1 38) do pause)^>nul^&findstr "^^" ^>%%N.temp)^&^
    ^>nul copy /y %%N.temp /a %%N.chr /b^&^
    del %%N.temp^
  )^
))^&^
del %%A.tmp^"
del /f /q /a *.chr >nul 2>&1
if "%~1" neq "" (set /a cnt=%~1) else set /a cnt=number_of_processors
if %cnt% lss 1 set cnt=1
if %cnt% gtr 256 set cnt=256
set /a "end=cnt-1"
for /l %%A in (0 1 %end%) do (
  type nul >%%A.tmp
  if %%A equ %end% (
    cmd /q /v:on /c "%genchr%"
  ) else (
    start "" /b cmd /q /v:on /c "%genchr%"
  )
)
:genAllChr.check
for /l %%N in (0 1 %end%) do if exist %%N.tmp goto :genAllChr.check
exit /b

I don't see any difference in timing from before.

In production code I would probably create a temporary directory in the %TEMP% folder to hold all the working files. That folder would be guaranteed to be empty at the start. Then I don't even need to delete *.CHR, and I can revert back to a simple IF EXIST *.TMP at the end.


Dave Benham

Hi Dave,

I do not have enough experience to include this piece of code in the macro. On my single processor system this reduces by a further 20% of the execution time. It should be possible to adapt it to run on Multicores.

@echo off
set t0=%time%
del /f /q /a *.chr >nul 2>&1
setlocal EnableDelayedExpansion
set "charlist=#$%%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~"
for /L %%N in (0,1,91) do if %%N neq 26 <nul set /p "=!charlist:~%%N,1!" >%%N.chr
endlocal
setlocal disableDelayedExpansion
:genAllChr
::This code creates 256 1 byte files, one for each possible byte value.
::This is encoded in a macro way to be called asynchronously with start cmd /c
::Teamwork of carlos, penpen, aGerman, dbenham
::Tested under Win7 and XP
set ^"genchr=(^
for /l %%N in (%%A !cnt! 255) do (^
if %%N equ 26 (^
copy /y nul + nul /a 26.chr /a ^>nul^
) else (if not exist %%N.chr if %%N geq 35 if %%N leq 126 if %%N neq 61 (^
cmd /c exit %%N^&^
^<nul set /p "=!=exitCodeAscii!" ^>%%N.chr^
))^&^
if not exist %%N.chr (^
makecab /d compress=off /d reserveperdatablocksize=26 /d reserveperfoldersize=%%N %%A.tmp %%N.chr ^>nul^&^
type %%N.chr ^| ((for /l %%n in (1 1 38) do pause)^>nul^&findstr "^^" ^>%%N.temp)^&^
^>nul copy /y %%N.temp /a %%N.chr /b^&^
del %%N.temp^
)^
))^&^
del %%A.tmp^"
if "%~1" neq "" (set /a cnt=%~1) else set /a cnt=number_of_processors
if %cnt% lss 1 set cnt=1
if %cnt% gtr 256 set cnt=256
set /a "end=cnt-1"
for /l %%A in (0 1 %end%) do (
type nul >%%A.tmp
if %%A equ %end% (
cmd /q /v:on /c "%genchr%"
) else (
start "" /b cmd /q /v:on /c "%genchr%"
)
)
echo %t0% - %time%
:genAllChr.check
for /l %%N in (0 1 %end%) do if exist %%N.tmp goto :genAllChr.check
echo %time%
exit /b

EDIT: There is an error on For. Must add +35 in the %%N.chr

einstein1969

[carlos]

I wrote many variants today, but this is maybe the better for me:

In the macro I copied the if else logic from dbenham:

Code: Select all

@echo off

:gen255
::generates 0-255 .chr files that have 1 byte
::Teamwork of carlos, penpen, aGerman, dbenham
::Tested under Win8

set "genchr=if %%# equ 26 ("
set "genchr=%genchr% copy /y nul + nul /a 26.chr /a >nul ) else ("
set "genchr=%genchr% if %%# geq 35 if %%# leq 126 if %%# neq 61 ("
set "genchr=%genchr% cmd /a /d /q /c exit /b %%# &cmd /a /d /q /c set /p"
set "genchr=%genchr% "=%%=exitcodeascii%%" <nul >%%#.chr"
set "genchr=%genchr% )) &if not exist %%#.chr ("
set "genchr=%genchr% type nul >%%#.tmp"
set "genchr=%genchr% &makecab /d compress=off"
set "genchr=%genchr% /d reserveperdatablocksize=26"
set "genchr=%genchr% /d reserveperfoldersize=%%#"
set "genchr=%genchr% %%#.tmp %%#.chr >nul"
set "genchr=%genchr% &type %%#.chr"
set "genchr=%genchr% |((for /l %%_ in (1 1 38) do pause)>nul"
set "genchr=%genchr% &findstr ^^^^ >%%#.tmp)"
set "genchr=%genchr% &copy /y %%#.tmp /a %%#.chr /b >nul"
set "genchr=%genchr% &del %%#.tmp)"

for /l %%# in (0,1,255) do (
if exist %%#.chr del /f /q /a %%#.chr >nul
if exist %%#.tmp del /f /q /a %%#.tmp >nul
)

for /l %%# in (0,1,255) do start "" /b /i cmd /a /d /q /c "%genchr%"

:gen255chk
for /l %%# in (0 1 255) do for %%_ in (
%%#.chr) do if not "1"=="%%~z_" goto :gen255chk

goto :eof