forking commands

[Matice]

hello,

is there a quick trick that can fork a tasks to run concurrently hence speeding up the process instead of running in a single long loop.

for example,

i have a command i want to execute on several hundred boxes,
if i would simply use

for /f "tokens=*" %%r in (myhosts.txt) do (
echo test command on %%r >> test.txt
)

the above would take ages to run sequentially in loop,

is there a trick to fork the command based on the variables ?
might the following work?
===
for /f "tokens=*" %%r in (myhosts.txt) do (
call task.cmd %%r
)
===

task.cmd
===
echo test command on %1 >> test.txt
==
this way i could have 300 concurrent executions cutting down on time taken to perform the task in sequence,

any ideas will this work or are there better means?

regards
Matice

[dizze]

More or less yes.

Something like the following would work:

==Multi.bat==
@echo off
for /f "tokens=1 skip=3 delims=\ " %%a in ('net view') do (start d:\temp\foo.bat %%a)
==End Multi.bat==

==foo.bat==
echo this is %1 >%1.txt
exit
==end foo.bat==

Note:

1) Use start rather than call.
2) Specify the full path to the other batch file
3) Use %1 %2 etc in the other batch file to reference the variables passed from the first batch file
4) put 'exit' at the end of the batch file or you will have 300 open cmd windows at the end
5) you may want to consider not running 300 of them at once, as it will probably cripple your box.

[Matice]

yep start works fine, now im facing another issue,
as each command retrieves data and echos it to a common file, the data gets entered in random order because the data retrieved returns on different time intervals.. now in need to gifure out a "write lock" mechanism so that the data is entered to the common file correctly :/

any ideas?

More or less yes.

Something like the following would work:

==Multi.bat==
@echo off
for /f "tokens=1 skip=3 delims=\ " %%a in ('net view') do (start d:\temp\foo.bat %%a)
==End Multi.bat==

==foo.bat==
echo this is %1 >%1.txt
exit
==end foo.bat==

Note:

1) Use start rather than call.
2) Specify the full path to the other batch file
3) Use %1 %2 etc in the other batch file to reference the variables passed from the first batch file
4) put 'exit' at the end of the batch file or you will have 300 open cmd windows at the end
5) you may want to consider not running 300 of them at once, as it will probably cripple your box.

[dizze]

I would use seperate files for the report on each server, then type them all into one once they have all finished executing. To use the example:


==Multi.bat==
@echo off
del results.txt
for /f "tokens=1 skip=3 delims=\ " %%a in ('net view') do (start d:\temp\foo.bat %%a)
type *.output >>results.txt
==End Multi.bat==

==foo.bat==
echo this is %1 >%1.output
exit
==end foo.bat==


Dizz-E

[Matice]

well one thing that could be done is to create files for output that are sequential and unique to every host and at the end join them together hence;

just need to figure how to create an encrimented variable for unique file name..

I would use seperate files for the report on each server, then type them all into one once they have all finished executing. To use the example:


==Multi.bat==
@echo off
del results.txt
for /f "tokens=1 skip=3 delims=\ " %%a in ('net view') do (start d:\temp\foo.bat %%a)
type *.output >>results.txt
==End Multi.bat==

==foo.bat==
echo this is %1 >%1.output
exit
==end foo.bat==


Dizz-E

[Matice]

this is how far i got,

as you can see, im passing variables to start in a loop, but for some reason, the variables are not being received, any ideas`?

ive been banging my head with this now for a while, i hope someone can enlighten me.

regards
Matice

forktest.cmd
====

@echo off
cls
echo scanning hosts ..

set snmp1=mypublic
set snmp2=myprivate

set mysearch ="firefox"

set logfile=fork_log.txt
set output=final_results.txt

echo %mysearch% > %output%
echo %output% > %logfile%

REM start console to follow up progress ..
start c:\mypath\tail -f %logfile%

echo Searching for %mysearch%
echo .
echo forking tasks ..

set FILEID=0
for /f "tokens=*" %%f in (myhosts.txt) do (
set FILEID=%FILEID% + 1
echo forking tasks for %%f
start c:\mypath\tasktest.cmd %%f %mysearch% %snmp1% %snmp2% %FILEID%.txt %logfile%
)

set lastlogfile=%FILEID%
echo generating log ..

for /L %%i in (1,1,%lastlogfile%) do cat %%i.txt >> %output%

echo cleaning up ..
for /L %%i in (1,1,%lastlogfile%) do del %%i.txt

echo done!
====

tasktest.cmd

====

echo performing search criteria of %2 on %1 with the following snmp strings %3 and %4 >> %6

echo %1 >> %5
echo %2 %3 %4 %5 %6 >> %5
echo =============== >> %5
@echo off
echo loging process results of %1 into %5
echo Task for %1 completed! >> %6
echo . >> %6
echo . >> %6
echo Done!
exit
====

[Matice]

problem solved,

i had an excess space mark and used set /a FILEID+=1

instead of set /a fileid=%fileid%+1

thanks a lot for this!

this is how far i got,

as you can see, im passing variables to start in a loop, but for some reason, the variables are not being received, any ideas`?

ive been banging my head with this now for a while, i hope someone can enlighten me.

regards
Matice

forktest.cmd
====

@echo off
cls
echo scanning hosts ..

set snmp1=mypublic
set snmp2=myprivate

set mysearch ="firefox"

set logfile=fork_log.txt
set output=final_results.txt

echo %mysearch% > %output%
echo %output% > %logfile%

REM start console to follow up progress ..
start c:\mypath\tail -f %logfile%

echo Searching for %mysearch%
echo .
echo forking tasks ..

set FILEID=0
for /f "tokens=*" %%f in (myhosts.txt) do (
set FILEID=%FILEID% + 1
echo forking tasks for %%f
start c:\mypath\tasktest.cmd %%f %mysearch% %snmp1% %snmp2% %FILEID%.txt %logfile%
)

set lastlogfile=%FILEID%
echo generating log ..

for /L %%i in (1,1,%lastlogfile%) do cat %%i.txt >> %output%

echo cleaning up ..
for /L %%i in (1,1,%lastlogfile%) do del %%i.txt

echo done!
====

tasktest.cmd

====

echo performing search criteria of %2 on %1 with the following snmp strings %3 and %4 >> %6

echo %1 >> %5
echo %2 %3 %4 %5 %6 >> %5
echo =============== >> %5
@echo off
echo loging process results of %1 into %5
echo Task for %1 completed! >> %6
echo . >> %6
echo . >> %6
echo Done!
exit
====