DosTips.com • Access denied in concurrent access

Page 1 of 2

Access denied in concurrent access

Posted: 01 Apr 2016 04:36

by einstein1969

Hi,

I have a problem with the dos batch raycast when executed on multicore. Expert needed!

I have semplified the code and found one possible cause.

EDIT:You have to run the test several times before catch errors and on a multicore machine is more difficult to catch them.

I have two file:

Slave.cmd

Code: Select all

@echo off
setlocal EnableDelayedExpansion

for /l %%. in () do if exist "job.bat" (
  echo Job.bat Exist
  echo Calling job.bat ...
  call "job.bat"
  echo Do job ... : "!job!"
  !job!
  echo deleting job.bat
  del job.bat
)

master.cmd:

Code: Select all

@echo off
setlocal EnableDelayedExpansion

rem start the slave process
start "Slave" "%comspec%" /c "slave.cmd"

Title Master

>nul ping -n 2 127.0.0.1

rem prepare job
>job.bat.tmp echo set "job=set /a 2*3"

ren job.bat.tmp *.

Echo Done.
pause

Run the master and the error is visible in the slave output
slave.out:

Code: Select all

Job.bat Exist
Calling job.bat ...
Impossibile accedere al file. Il file è utilizzato da un altro processo.
Do job ... : ""
deleting job.bat

master.out:

Code: Select all

Done.
Premere un tasto per continuare . . .

I thinks that the solution is LOCK the share resource (in this case job.bat) or catch the error with &&/|| or errorlevel

How can i do?

einstein1969

Re: Access denied in concurrent access

Posted: 01 Apr 2016 06:13

by einstein1969

On susseguent(not concurrent with other master/slave running) runs I achieve this other errors.

slave.out

Code: Select all

Job.bat Exist
Calling job.bat ...
Impossibile accedere al file. Il file è utilizzato da un altro processo.
Do job ... : ""
deleting job.bat
Job.bat Exist
Calling job.bat ...
Accesso negato.
Do job ... : ""
deleting job.bat
Impossibile trovare C:\Users\LORENZO\Desktop\job.bat

Seem that the DEL is executed deferred or not executed.

After the DEL the JOB.BAT exist! And the error is different. The next DEL not found the file job.bat!

How risolve this second problem?

Re: Access denied in concurrent access

Posted: 01 Apr 2016 06:32

by foxidrive

einstein1969 wrote:How risolve this second problem?

The master.cmd is launching the start command with slave.cmd, before the job.bat is created.

Is that the problem?

Re: Access denied in concurrent access

Posted: 01 Apr 2016 08:58

by einstein1969

foxidrive wrote:
einstein1969 wrote:How risolve this second problem?

The master.cmd is launching the start command with slave.cmd, before the job.bat is created.

Is that the problem?

No, the slave wait until the job.bat is created, than delete

Re: Access denied in concurrent access

Posted: 01 Apr 2016 20:44

by foxidrive

Ok einstein.

Here is the screen output.

Re: Access denied in concurrent access

Posted: 02 Apr 2016 02:24

by einstein1969

foxidrive wrote:Ok einstein.

Here is the screen output.

While it may seem that everything went ok in this case, actually you have met a third error. It was executed twice for the same code instead of once only. If you keep doing tests it can happen all three types of error.

I forgot to say that the first two errors are rare to come across. You have to run the test several times before meeting them and on a multicore machine is more difficult to catch them.

This is the case all ok:

Re: Access denied in concurrent access

Posted: 02 Apr 2016 03:46

by einstein1969

For increase the probability to catch errors you may execute this batch wich saturate the CPU near 100% (on windows 8.1 this not work well

)

saturate.cmd:

Code: Select all

@echo off
set /A proc=NUMBER_OF_PROCESSORS*2
for /L %%. in (1,1,%proc%) do (
  start "" /b "%comspec%" /q:on /c "for /L %%L in () do rem"
)
exit /b

Launch saturate.cmd than master.cmd to probe to catch erros.

Re: Access denied in concurrent access

Posted: 02 Apr 2016 05:26

by foxidrive

einstein1969 wrote:This is the case all ok:

Is the slave meant to hang there at that point? Is it waiting for the job.bat file again?

I've done a set of tests and so far it hasn't errored again yet.

Re: Access denied in concurrent access

Posted: 02 Apr 2016 06:16

by einstein1969

Yes , the slave wait for next job. But the master is finished.

It's possible to loop in the master for repeat automatically. Like this:

Master.cmd

Code: Select all

@echo off
setlocal EnableDelayedExpansion

rem start the slave process
start "Slave" "%comspec%" /c "slave.cmd"

Title Master

>nul ping -n 2 127.0.0.1

For /L %%. in (1,1,10000) do (

  rem prepare job
  >job.bat.tmp echo set "job=set /a 2*3"

  if not exist job.bat ren job.bat.tmp *.

  Echo Done.
)

pause

But there isn't a exit condition for the slave. You must close it manually.

Re: Access denied in concurrent access

Posted: 02 Apr 2016 09:02

by Aacini

I don't like to complete extensive tests to catch errors that just happen now and then. However, I see a strange point in your code:

Code: Select all

ren job.bat.tmp *.

I suppose that the way this command is executed is rename "job.bat.tmp" to "job.bat" first, without release the handle used for such wild-card rename operation, but until detected that there are no more files in the wild-card rename operation. If the other process try to access the just renamed "job.bat" file before such handle is released, this problem may arise (this is just an educated guess).

I would just change "ren job.bat.tmp *." by "ren job.bat.tmp job.bat" and test the code again...

Anyway, I don't see any reason to use "ren job.bat.tmp *." in the very first place (excepting to be cryptic and write code more difficult to understand and maintain, perhaps). If you write code with "strange things" and your code present a "strange behavior", the obvious thing to test first is to eliminate the strange things (just my opinion)...

Antonio

Re: Access denied in concurrent access

Posted: 03 Apr 2016 03:29

by einstein1969

Thanks Antonio for point this.

I have tested the new code. But the problem remains.

This code is for execute on multicore machine in parallel/concurrent thread. This is for dualcore or plus. But if run on monocore it should run too!

I have modified the master
Master.cmd

Code: Select all

@echo off
setlocal EnableDelayedExpansion

rem start the slave process
start "Slave" "%comspec%" /c "slave.cmd"

Title Master

>nul ping -n 2 127.0.0.1

set count=0

For /L %%. in (1,1,10000) do (

  if !count! lss 30 if not exist job.bat (
    rem prepare job
    >job.bat.tmp echo set "job=set /a 2*3"
    rem ren job.bat.tmp *.
    ren job.bat.tmp job.bat
    Echo Done.
    set /A count+=1
  ) 

)

pause

and
slave.cmd

Code: Select all

@echo off
setlocal EnableDelayedExpansion

for /l %%. in () do if exist "job.bat" (
 call "job.bat"
 echo Do job ... : "!job!"
 set job=
 del job.bat
)

slave.out:

Code: Select all

Impossibile accedere al file. Il file è utilizzato da un altro processo.
Do job ... : ""
Impossibile accedere al file. Il file è utilizzato da un altro processo.
Do job ... : ""
Accesso negato.
Do job ... : ""
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
Do job ... : ""
C:\Users\ACER\Desktop\job.bat
...

einstein1969

Re: Access denied in concurrent access

Posted: 03 Apr 2016 11:09

by dbenham

I cannot reproduce your results - when I run your code, everything seems to execute properly.

But I get similar illogical results as yours if I restructure the code and attempt to TYPE the job.bat instead of CALL it.

master.bat

Code: Select all

@echo off
setlocal EnableDelayedExpansion
title Master
start "Slave" "%comspec%" /c "slave.bat"
>nul ping -n 2 127.0.0.1
set count=0
for /l %%. in (1,1,10000) do (
  if !count! lss 30 if not exist job.bat (
    set /a count+=1
    >job.bat.tmp echo !count!
    ren job.bat.tmp job.bat
    set count
  )
)

slave.bat

Code: Select all

@echo off
for /l %%. in () do if exist "job.bat" (
  type "job.bat"
  del job.bat
)

--Master output--

Code: Select all

C:\test\>master
count=1
count=2
count=3
count=4
count=5
count=6
count=7
count=8
count=9
count=10
count=11
count=12
count=13
count=14
count=15
count=16
count=17
count=18
count=19
count=20
count=21
count=22
count=23
count=24
count=25
count=26
count=27
count=28
count=29
count=30

--Slave output--

Code: Select all

1
2
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
6
The process cannot access the file because it is being used by another process.
8
9
10
11
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
16
The process cannot access the file because it is being used by another process.
18
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
24
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
27
The process cannot access the file because it is being used by another process.
29
30

Just as in your more recent code, I must manually close the Slave window.

I don't understand why I get that particular error message - I cannot think of a logical reason for the job.bat file to be inaccessible to TYPE.

Here is another weird thing - if I change the temp file name from "job.bat" to "job.txt", then the failure rate is drastically reduced (not shown), but it still does fail a few times.

But anyway, after restoring the temp file name to job.bat, I then modify slave.bat to try again if the first TYPE fails. I also hide any error message from the first attempt:

slave.bat

Code: Select all

@echo off
for /l %%. in () do if exist "job.bat" (
  2>nul type "job.bat"||type "job.bat"
  del job.bat
)

-- Slave Output --

Code: Select all

Everything works now as expected :!:

:!:

Okay... I can't explain why, but there seems to be some weird timing issue that makes the file unavailable for a time. So I thought we just need an appropriate delay...

But at viewtopic.php?f=3&t=5824&p=46046#p46046 I tried incorporating a delay in the raycast demo before each compute engine attempts to execute the temporary job script, and it didn't solve the issue. :evil:

:evil:

Dave Benham

Re: Access denied in concurrent access

Posted: 04 Apr 2016 15:30

by einstein1969

I have probed the slave with the:

Code: Select all

@echo off
for /l %%. in () do if exist "job.bat" (
  2>nul type "job.bat"||type "job.bat"
  del job.bat
)

on monocore the result is different from your

slave output:

Code: Select all

Impossibile accedere al file. Il file è utilizzato da un altro processo.
Impossibile accedere al file. Il file è utilizzato da un altro processo.
Impossibile accedere al file. Il file è utilizzato da un altro processo.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
Accesso negato.
C:\Users\ACER\Desktop\job.bat
Accesso negato.
...

I will probe on multicore .

Re: Access denied in concurrent access

Posted: 05 Apr 2016 05:41

by dbenham

Well, all the evidence points to a timing issue that is highly variable, so I am not very surprised you get a different result.

Just for investigative purposes, try putting a PING delay before TYPE. I certainly hope that eliminates the errors, but obviously that is not a viable solution because it is too slow.

I imagine you could string enough 2>NUL (TYPE "JOB.BAT"||TYPE "JOB.BAT"||TYPE "JOB.BAT")||TYPE "JOB.BAT" etc. together that eventually it would work for you on your machine. The good thing is that it would execute as few TYPE as is necessary. The bad thing is it is not a general solution, there is no way to predict how many are needed.

I would really like to know why any delay is needed in the first place. I don't understand how any process is blocking the TYPE command (or the CALL command). If we understood the mechanism, maybe we could better arrive at a solution.

Dave Benham

Re: Access denied in concurrent access

Posted: 05 Apr 2016 06:15

by jeb

I suppose that the rename command is not atomic, it takes some time while the file is visible but not accessible.

A simple solution is to use two files, even the structure is more obvious then.

To simulate a one core system I used
start /affinity 1 master.bat

master.bat

Code: Select all

@echo off
setlocal EnableDelayedExpansion
title Master
start /affinity 1 "Slave" "%comspec%" /c "slave.bat"
>nul ping -n 2 127.0.0.1
set count=0
for /l %%. in (1,1,10000) do (
  if !count! lss 30 if not exist flag.tmp (
    set /a count+=1
    >job.bat echo !count!
    > flag.tmp break
    set count
  )
)

slave.bat

Code: Select all

@echo off
for /l %%. in () do if exist "flag.tmp" (
  type "job.bat"
  del flag.tmp
)