problem in set /p with large data in pipe

[carlos]

Hello. In windows 8, I have a little problem.
I'm working in a program that output text many seconds and I send this output to a batch that read that text with set /p in a loop.
The problem is after the data is many bytes. Maybe some buffer. I not know. Set /P fails.

For reproduce this I produce a 145KB file called klog.txt :
that have the text word NONE + CRLF repeated.

produce_klogtxt.cmd

Code: Select all

@Echo Off
SetLocal EnableExtensions
Call :Rebuild
If ErrorLevel 1 Echo Rebuild failed.
Goto :Eof

:Rebuild
Rem Script made using BHX 5.6 { consolesoft.com/p/bhx }
SetLocal EnableExtensions EnableDelayedExpansion
Set "bin=KLOG.CAB"
Set /A "size=441"
For %%# In (
"klog.txt"
"!bin!" "!bin!.da" "!bin!.tmp"
) Do If Exist "%%#" (Del /A /F /Q "%%#" >Nul 2>&1
If ErrorLevel 1 Exit /B 1 )
Set "fsrc=%~f0"
Findstr /B /N ":+res:!bin!:" "!fsrc!" >"!bin!.tmp"
(Set /P "inioff="
Set /P "endoff=") <"!bin!.tmp"
For /F "delims=:" %%# In ("!inioff!") Do Set "inioff=%%#"
For /F "delims=:" %%# In ("!endoff!") Do Set "endoff=%%#"
Set ".=ado="adodb.stream""
Set ".=!.! :set a=createobject(ado) :a.type=1 :a.open"
Set ".=!.! :set u=createobject(ado) :u.type=2 :u.open"
Set ".=!.! :set fs=createobject("scripting.filesystemobject")"
Set ".=!.! :set s=fs.opentextfile("!fsrc!",1,0,0)"
Set ".=!.! :e="0123456789abcdefghijklmnopqrstuvwxyzABCDEF"
Set ".=!.!GHIJKLMNOPQRSTUVWXYZ.-:+=^^`/*?&<>()[]{}~,$#"
Set ".=!.!" :max=!size! :wri=0 :n=array(0,0,0,0,0)"
Set ".=!.! :for i=1 to !inioff! step 1 :s.readline :next"
Set ".=!.! :do while i<!endoff! :d=replace(s.readline," ","")"
Set ".=!.! :for j=1 to len(d) step 5 :num85=mid(d,j,5)"
Set ".=!.! :v=0 :for k=1 to len(num85) step 1"
Set ".=!.! :v=v*85+instr(1,e,mid(num85,k,1))-1 :next"
Set ".=!.! :n(1)=Fix(v/16777216) :v=v-n(1)*16777216"
Set ".=!.! :n(2)=Fix(v/65536) :v=v-n(2)*65536"
Set ".=!.! :n(3)=Fix(v/256) :n(4)=v-n(3)*256"
Set ".=!.! :for m=1 to 4 step 1 :if (wri < max) then"
Set ".=!.! :u.writetext chrb(n(m)) :wri=wri+1 :end if :next"
Set ".=!.! :next :i=i+1 :loop"
Set ".=!.! :u.position=2 :u.copyto a :u.close :set u=nothing"
Set ".=!.! :a.savetofile "!bin!",2 :a.close :set a=nothing"
Set ".=!.! :s.close :set s=nothing :set fs=nothing"
Echo !.!>"!bin!.da"
Set "ret=1"
Cscript.exe /B /E:vbs "!bin!.da" >Nul
For %%# In ("!bin!") Do If "%%~z#"=="!size!" Set "ret=0"
If "0"=="!ret!" Expand.exe -r "!bin!" -F:* . >Nul
If ErrorLevel 1 Set "ret=1"
Del /A /F "!bin!" "!bin!.da" "!bin!.tmp" >Nul
Exit /B !ret!

:+res:KLOG.CAB:
o&BzG00000XD8Wg00000ec2UI000000,5A50rr9100000mf0q/1POM6[iE9Q
00000005lOo+vv9yIW{4e{8tW0f]FBi.xwLFi##,-4.Y300&XhP9erf.[Ndz
=(QCzX[{cfzqhHo*Qp$NWR:^czqhHo*Qp$NWR:^czqhHo*Qp$NWR:^czqhHo
*Qp$N~}#<qqDr(*004Kpoyp7H0rr91.>qaX(MtpUfA/e>nS>t<K,ARZbUsdA
nS>t<K,ARZbUsdAnS>t<K,ARZbUsdAnS>t<K,ARZbUsf^3tbEHmf0srlRD*[
f+19.0k>EVn=vzm5GR8kFH0*IbUsdAnS>t<K,ARZbUsdAnS>t<K,ARZbUsdA
nS>t<K,ARZbUsdAnS>vvCa[I&pdkc.Fi##,^7]XZ0009A.>1M`mev4F2A*u7
K,ARZbUsdAnS>t<K,ARZbUsdAnS>t<K,ARZbUsdAnS>t<K,ARZbUsdA/8Wrh
cz&N0[iGAm)A^0S000AeU.>yiUx0tq*~]&z14`U3nS>t<K,ARZbUsdAnP8^Q
aw4.wnS>vLj]+M.
:+res:KLOG.CAB:

This is the cmd that read the commands an fail:

in.cmd

Code: Select all

Set "k=NONE"
:loop
Set /P "k="
If Not "NONE"=="%k%" Rem
Goto :loop

Then, after run this in the command line:

Code: Select all

type klog.txt | in.cmd

Set /p fails:

Code: Select all

C:\dev>If Not "NONE" == "NONE" Rem

C:\dev>Goto :loop

C:\dev>Set /P "k="

C:\dev>If Not "NONE" == "E" Rem

C:\dev>Goto :loop

C:\dev>Set /P "k="

C:\dev>If Not "NONE" == "NE" Rem

C:\dev>Goto :loop

C:\dev>Set /P "k="
The syntax of the command is incorrect.

C:\dev>If Not "NONE"=="
NONE" Rem

It not fails if klog.txt have few lines.

Please, some help for found a solution for this problem.

[jeb]

Hi carlos,

I suppose it's the standard problem of set /p with pipes.
With a pipe, one set /p reads up to 1023 bytes, independent of crlf
The problem of this behaviour is, that all data after the first linefeed will be discarded and also the linefeed.

You could switch to redirected input, as there the set /p reads up to 1023 bytes BUT stops at the first linefeed.

[carlos]

Thanks Jeb. That should be the problem.

¿How I can do this?

I have a program that output text out.exe

I want read that outputs with set /p in a cmd file in a loop, and output some commands for send to other program.

Example:

out.exe | mybat.cmd | other.exe

Currently, I have problems with the set /p when the data is from the pipe.

Example: out.exe prints SUN, mybat.cmd read that text with set /p and if it is "SUN" echo "SUNDAY", that is readed by other.exe

I want mybat.cmd as intermediary between out.exe and other.exe

¿How I can accomplish this with redirections?}

[Aacini]

You may do this:

Code: Select all

copy nul pipeFile.txt
out.exe >> pipefile.txt | mybat.cmd < pipefile.txt | other.exe

... and the way to read the data in mybat.cmd is this:

Code: Select all

rem Read data from out.exe
:read
set "input="
:wait
   set /P "input="
if not defined input goto wait

rem Process the data and pass it to other.exe
set "output=%input% + other data"
echo %output%
goto read

This is the method I used in the last version of Tetris game. A small detail in this method is that the Batch code must identify a certain data as signal to end; otherwise, it will stay in the input loop after the out.exe program have terminated...

Antonio

[penpen]

A small detail in this method is that the Batch code must identify a certain data as signal to end; otherwise, it will stay in the input loop after the out.exe program have terminated...

You could externalize this signal:
Create a file "terminate", after out.exe has finished and
check the existance before using "set/p" in "mybat.cmd":

Code: Select all

@echo off
copy nul pipeFile.txt
>nul (
   del terminate
   ping.exe "www.google.de" >> "pipefile.txt"
   copy nul terminate
   echo(
) | "mybat.cmd" < pipefile.txt | find.exe /V ""

Code: Select all

@echo off
rem Read data from out.exe
:read
set "input="

:wait
if NOT exist "terminate" (
   set "input="
   set /P "input="
) else goto :terminate
if defined input (
   echo(processed: "%input%"
)
goto :wait

:terminate
del terminate
>con echo Bye.

This approach also has an issue:
If you ever have to kill the "leftmost pipe process" (depends on your out.exe), then you may have to create the file "terminate" manually ("terminate.bat"; or you may implement "mybat "terminate""):

Code: Select all

@copy nul terminate

penpen

Edit: Corrected a flaw in code.
Edit2: Corrected a flaw in text.

[carlos]

Aacini. Thanks.
@penpen. I will test that concept.