Access denied in concurrent access

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
einstein1969
Expert
Posts: 961
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

Re: Access denied in concurrent access

#16 Post by einstein1969 » 06 Apr 2016 05:27

Thanks jeb. I have probed on monocore and there is no errors.

The complex version is one master and 2 or more slave. This is very slow but work.

The only doubt is "if not exist flag*.tmp" in the master.cmd :?:

on multicore launch with:
start /affinity 1 master.bat

master.cmd

Code: Select all

@echo off
setlocal EnableDelayedExpansion
title Master
start /affinity 1 "Slave 1" "%comspec%" /c "slave1.cmd"
start /affinity 1 "Slave 2" "%comspec%" /c "slave2.cmd"
>nul ping -n 2 127.0.0.1
set count=0
if exist flag1.tmp del flag1.tmp
if exist flag2.tmp del flag2.tmp
for /l %%. in (1,1,100000) do (
  if !count! lss 30 if not exist flag*.tmp (
    set /a count+=1
    >job1.bat echo set /a job=!count!*2
    >job2.bat echo set /a job=!count!*3
    > flag1.tmp break
    > flag2.tmp break
    set count
  )
)
exit/b


slave1.cmd

Code: Select all

@echo off
setlocal EnableDelayedExpansion
for /l %%. in () do if exist "flag1.tmp" (
  call "job1.bat"
  echo Do job 1 ... : "!job!"
  set job=
  del flag1.tmp
)


slave2.cmd

Code: Select all

@echo off
setlocal EnableDelayedExpansion
for /l %%. in () do if exist "flag2.tmp" (
  call "job2.bat"
  echo Do job 2 ... : "!job!"
  set job=
  del flag2.tmp
)


I will try on multicore with and without affinity parameter.

einstein1969

jeb
Expert
Posts: 1055
Joined: 30 Aug 2007 08:05
Location: Germany, Bochum

Re: Access denied in concurrent access

#17 Post by jeb » 07 Apr 2016 06:46

Sorry,
but the flag-file solution fails too.

I tested it on another PC and it fails sometimes in the line "del flag.tmp".
Even if I run both batches on the same cpu core.

But even if it would work without problems, it's incredible slow :!:

I recommend the use of pipes to communicate between batch threads.
It has the big advantage, that the threads can go into sleep mode without consuming cpu time.

master3.bat

Code: Select all

@echo off
setlocal EnableDelayedExpansion
if "%~1" NEQ "" goto %~1
%~f0 :internal | slave3.bat
exit /b

:internal
set start=%time%
set "hash=#"
for /L %%n in ( 1, 1 13) do set "hash=!hash:~-4000!!hash:~-4000!"
set "long=!hash!"
REM set "long=!hash:~-1021!A!hash:~-1022!a!hash:~-1022!A!hash:~-1022!a!hash:~-1022!A" for better debug

for /L %%. in (1,1,10) do (
    set /a count+=1

    set "line=!count!!long!"
    set "line=!line:~0,4092!" 4092=4*1023, this works but the next send will be blocked until at least the slave read from the pipe

    <nul set /p ".=!line!"

    echo MasterSend: !count! > CON
)
set end=!time!
(
echo ---- Finish -----
echo !start!
echo !end!
) > con


slave3.bat

Code: Select all

@echo off
setlocal EnableDelayedExpansion
set count=0
ping -n 2 localhost > nul
for /l %%. in () do (
    set "partial="
    rem pause
    rem ping -n 2 localhost > nul
    set /p partial=
    if defined partial (
        rem echo !time! Recv: !partial!
        set "job=!job:A=!"
      set "job=!partial:#=!"
       
        if defined job (
            set /a count +=1
            echo SlaveRecv:..!job!
        )
    ) ELSE (
        echo SlaveCount:!count!
        exit
    )
)


This is really fast and it could be even faster.

It uses the fact, that a set /p in the pipe consumer (slave3.bat) waits for input as long as the pipe exists, else it reads and empty string.

And the pipe producer waits when the pipe is full until the consumer takes the messages from the pipe.

It's a bit strange that the consumer reads up to 1023 bytes per set /p.
But the pipe full size is at 4096 bytes (1024*4).

I ignored this, the master sends for each job a message with a size of 3=1023=4092 Bytes, when it tries to send the next message before the first message is consumed it will be blocked.

There is only the problem of the missing back channel from the slave to the master.
This can be solved with files :( or with DOSKEY macros.

I used doskey macros in the next sample

masterDoskey.bat

Code: Select all

@echo off
setlocal EnableDelayedExpansion
if "%~1" NEQ "" goto %~1
%~f0 :internal | slaveDoskey.bat
exit /b

:internal
set start=%time%
set "hash=#"
for /L %%n in ( 1, 1 13) do set "hash=!hash:~-4000!!hash:~-4000!"
set "hash=!hash:~0,4092!"
set "long=!hash!"

doskey slave2master=
REM set "long=!hash:~-1021!A!hash:~-1022!a!hash:~-1022!A!hash:~-1022!a!hash:~-1022!A" for better debug
set "job=0"
for /L %%. in (1,1,10) do (
    set /a job+=1
    set "line=!job!!long!"
    set "line=!line:~0,4092!" 4092=4*1023, this works but the next send will be blocked until at least the slave read from the pipe
   
    <nul set /p ".=!line!" Send the job
    echo MasterSend: !job! > CON
    <nul set /p ".=!hash!" Wait for the response
   (
      FOR /F "tokens=2* delims==" %%D in ('doskey /macros') do (
         echo Master Recv from Slave %%D
         echo(
      )
   ) > CON

)
set end=!time!
(
echo ---- Finish -----
echo !start!
echo !end!
) > con


slaveDoskey.bat

Code: Select all

@echo off
setlocal EnableDelayedExpansion
set count=0
ping -n 2 localhost > nul
for /l %%. in () do (
    set "partial="
    rem pause
    rem ping -n 2 localhost > nul
    set /p partial=
    if defined partial (
        rem echo !time! Recv: !partial!
        set "job=!job:A=!"
      set "job=!partial:#=!"
       
        if defined job (
            set /a count +=1
            echo SlaveRecv:..!job!
         doskey slave2master=Job!job! done
        )
    ) ELSE (
        echo SlaveCount:!count!
        exit
    )
)

einstein1969
Expert
Posts: 961
Joined: 15 Jun 2012 13:16
Location: Italy, Rome

Re: Access denied in concurrent access

#18 Post by einstein1969 » 07 Apr 2016 10:53

jeb wrote:I recommend the use of pipes to communicate between batch threads.

Finallly! I'm studing this piece of code...

It's a bit strange that the consumer reads up to 1023 bytes per set /p.

1) Is this link explain why?

But the pipe full size is at 4096 bytes (1024*4).

2) How did you find it?

3) Can you show me the code using the file solution instead of doskey macro. Just to make performance tests and see what suits.


einstein1969

Aacini
Expert
Posts: 1914
Joined: 06 Dec 2011 22:15
Location: México City, México
Contact:

Re: Access denied in concurrent access

#19 Post by Aacini » 07 Apr 2016 11:46

@einstein1969: All this stuff about pipes between threads, the 1023 bytes set /P point and semaphore files was fully discussed at Event-driven multi-thread scheme for Batch files topic, where you posted a couple replies some time ago...

Antonio

Post Reply