creating a true mutex for exclusive acces

[Ed Dyreen]

pseudocode this is not the actual code I use, just a simplification to focus on the problem:

Code: Select all

set "$fileMutex=(pause>&3)3>"%systemDrive%\ED\mutex\!m!.mutex"||!exception.get!"

Ah it works, now I need to be able to launch this code from the commandline..

Code: Select all

( %macroToCmdMacro_% $fileMutex, $fileMutex )

cmd /V:on /E:on /T:0B /Q /C "%%$fileMutex%% lockThisFileFromWriting"

That worked, but I need to launch it in a separate thread otherwise it will just sit the duck..

Code: Select all

start "" /LOW /B cmd /V:on /E:on /T:0B /Q /C "%%$fileMutex%% lockThisFileFromWriting"

It works but, how am I gonna close the mutex now ?

I can't communicate through a for loop as that would wait for the new thread to finish, that will never happen obviously.
Actually at this point I lost all means to communicate to my child process at all :/
The only secure solution I can come up with is to figure out my child process ID, so I can taskkill the PID, but the only way I know of to retrieve the child process is by using WMIC and that's so painfully slow

Hmmmmz, any thoughts ? thanks ..

[OperatorGK]

The only way I know is using window titles.
I think you should change your command to

Code: Select all

set P_NAME_ID=%random%_%random%_%random%_%random%
start "LOCK_%P_NAME_ID%" /LOW /B cmd /V:on /E:on /T:0B /Q /C "%%$fileMutex%% lockThisFileFromWriting"

And then close it like:

Code: Select all

taskkill /IM cmd.exe /F /FI "WINDOWTITLE eq LOCK_%P_NAME_ID%"

And can you please write code of your exception (I mean what is behind "!exception.get!") mechanism? This interests me .

[Ed Dyreen]

The only way I know is using window titles.
I think you should change your command to

Code: Select all

set P_NAME_ID=%random%_%random%_%random%_%random%
start "LOCK_%P_NAME_ID%" /LOW /B cmd /V:on /E:on /T:0B /Q /C "%%$fileMutex%% lockThisFileFromWriting"

You wrote start /B but start /B doesn't create a separate windowTitle, hence it won't be possible to differentiate the parent- from the child process. Leaving out /B fixes this but it's insecure and requires a separate window for every instance.

And can you please write code of your exception (I mean what is behind "!exception.get!") mechanism? This interests me .

it stores the last errorLevel, nothing special there.

exception.get.CMD

Code: Select all

:: (exception.get:disDelayed) '34' bytes on file, '34' bytes in memory.
:: (
%=   =%for %%# in ("") do set ^"exception.get=if errorLevel 1 set/A£e=errorLevel" !
:: )

[OperatorGK]

Hmm. If you're running Vista+, you may use waitfor + parallel piping:
I'll not use actual code just to simplify my solution.

Code: Select all

rem #Setting internal process id
set IPID=%random%_%random%
rem #Launching process
start "" /B cmd /C "@echo off & ( (waitfor STOP_%IPID% & exit) | rem Do the file lock )"

And stopping file lock:

Code: Select all

waitfor /SI STOP_%IPID%

Even if this code isn't correct, I think you'll understand the main idea.

EDIT: Remove "_" in waitfor calls and IPID or it won't work!

[Ed Dyreen]

Hmm. If you're running Vista+, you may use waitfor + parallel piping

I am developing for XP, also I don't understand how you can know the PID before it's even created, then I see %random% which further confuses me. Can u explain how it works and how it is secure ?

I've been refusing to use jeb's solution to have the mutex call back to the parent function until now, this only requires a single thread and a few extra call's. But jeb's way:

Code: Select all

9>"%systemDrive%\ED\mutex\%~1.mutex" (

       call :%~2

)

Can only be used at the very beginning of the script, otherwise I'll end up with a stack of functions I can never return from without releasing the mutex Imagine the following situation:

Code: Select all

main();
exit 0

:main() {
       if mutexCreate( "mutex" ) {
              ... do stuff ...
              mutexRelease( "mutex" );
              return 0;
       };
       return 1;
}
:mutexCreate( m=mutex ) {
       call mutex.CMD( m return );
       :return () {
              // How am I gonna return ? If I return, the mutex is released and the program is finished early.
              return 0;
       }
       return 1;
}

[OperatorGK]

So I just have uploaded waitfor.exe for you:
http://1drv.ms/1GPOQAa
Download waitfor.exe and put it next to your batch file.

[OperatorGK]

IPID is internal, that means IPID is only used in batch file, not in actual process. So I use %random% to prevent one-time terminating of all file locks.

But anyway solution with waitfor.exe don't work as it is cause we can't exit in pipe. So, you need to use either title way or WMIC way. It is a choose between speed and security.

EDIT: I have an idea how we can exit in pipe. Just wait a bit more, and I'll write working prototype.
EDIT 2: Programmatically pressing Ctrl-C isn't working. No more ideas left now.

[Ed Dyreen]

IPID is internal, that means IPID is only used in batch file, not in actual process. So I use %random% to prevent one-time terminating of all file locks.

But anyway solution with waitfor.exe don't work as it is cause we can't exit in pipe. So, you need to use either title way or WMIC way. It is a choose between speed and security.

EDIT: I have an idea how we can exit in pipe. Just wait a bit more, and I'll write working prototype.

Ok, I'm looking forward to it.

The reason I try to avoid WMIC on XP machines is because it has several issues which need to be addressed to make it work reliably on all versions of XP. The first issue is that WMIC will return 0 and the message "Initializing WMIC for first time use.." the first time it runs on XP. The second issue is that on a pre SP2 WMIC outputs in UNICODE where later versions output in ASCII. This causes an extra carriage return to be added which has to be removed later on.

For now it looks like I have no other choice, as jeb's fileMutex is not working for the scenario described earlier.

[dbenham]

It works but, how am I gonna close the mutex now ?

I can't communicate through a for loop as that would wait for the new thread to finish, that will never happen obviously.
Actually at this point I lost all means to communicate to my child process at all :/

Why do you say that

Your mutex lock process does not need to be static. It can be in a loop that checks for the existence of a signal file and either exit, or sleep before it checks again.

Here is a crude demonstration of asynchronous mutex locks:

testMutex.bat

Code: Select all

@echo off
setlocal

:: initialize
set "sleep=pathping /p 100 /q 1 localhost >nul 2>nul"

:: test
echo requesting lock for %~1
call :lockMutex %~1
echo mutex lock established for %~1
for /l %%N in (1 1 100) do %sleep%
call :releaseMutex %~1
echo mutex lock released for %~1
exit /b


:lockMutex
setlocal
:: request mutex lock
start /b cmd /c "2>nul >nul (for /l %%A in () do 9>mutex.lock ((call )>mutex%~1.active&for /l %%B in () do if exist mutex%~1.active (%sleep%) else exit)&%sleep%)"
:: wait for lock
cmd /c "for /l %%A in () do @if exist mutex%~1.active (exit) else %sleep%"
exit /b 0


:releaseMutex
del mutex%~1.active
exit /b

To run the test, open two cmd.exe console windows and run the script in each, passing 1 as an argument for the first, and 2 for the 2nd.

The PATHPING delay is used to prevent the mutex from sucking up CPU cycles, but it limits the responsiveness of the mutex. If you need "instantaneous" response, then of course you can eliminate the delay, but then you will be abusing your poor CPU.

Dave Benham

[dbenham]

Here is a more impressive, and easier to run demonstration. It creates two test processes within the same window that alternate control via the asynchronous mutex. I've eliminated the sleep within the mutex code, so it is a CPU hog. But it shows the nice, orderly synchronization between the two processes.

Each test process can have as many as two sub-processes active at any given time - one for the lock, and another to wait for the lock to be established.

Code: Select all

@echo off
if "%~1" neq "" goto :test
setlocal

:: initialize
set "sleep=pathping /p 500 /q 1 localhost >nul 2>nul"

start /b "" "%~f0" 1
cmd /c "%~f0" 2
exit /b

:test
for /l %%N in (1 1 10) do (
  call :lockMutex %~1
  echo mutex lock established for %~1
  %sleep%
  echo releasing mutex lock for %~1
  call :releaseMutex %~1
)
exit


:lockMutex
setlocal
:: request mutex lock
start /b cmd /c "2>nul >nul (for /l %%A in () do 9>mutex.lock ((call )>mutex%~1.active&for /l %%B in () do if not exist mutex%~1.active exit))"
:: wait for lock
cmd /c "for /l %%A in () do @if exist mutex%~1.active (exit)"
exit /b 0


:releaseMutex
del mutex%~1.active
exit /b

-- Sample Output--

Code: Select all

mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2
mutex lock established for 1
releasing mutex lock for 1
mutex lock established for 2
releasing mutex lock for 2

Dave Benham

[Ed Dyreen]

You are right, I am using a file to simulate a mutex, why not just use a file as a communication link, it's so simple.

Well by now I already figured out how to retrieve myPID and childPID in a secure way, but I don't like it, too slow and depends on an external application cmdow.EXE

Code: Select all

cmdow.EXE @        // returns my PID
WMIC process WHERE name='cmd.exe' GET ProcessId, ParentProcessId
... do stuff ...
taskkill childPID

Yes, I'll do it as you suggest, thanx ben


ps; Why do you prefer pathPing over ping ?

[dbenham]

PATHPING provides better precision for sub-second delays. PING rounds to the nearest 1/2 second, PATHPING does not (but it still is not particularly accurate)


Dave Benham

[Ed Dyreen]

ok, cool but now I have a different problem. I need to clean the DOS environment before I start a new instance of CMD. The batchfile I use to develop and test uses 250MB of memory when it reaches this point. Actual memory is only 5MB but in DOS, once memory is taken, it's never released even if the environment is cleared. It takes a long time to prepare the environment for a new instance:

Code: Select all

for /F "tokens=1 delims==" %%? in (

       'set'

) do   if defined %%? set "%%?="

There is no secret switch to make the new cmd instance NOT inherit it's parent environment ?

Maybe there is a better way ?

[dbenham]

Ugh - We might have to arrest you for child (process) abuse.

I've never seen a method that allows creation of a child CMD process that does not inherit the parent's environment.

Perhaps this is a dead end.


Dave Benham

[aGerman]

Well, at least no direct child processes.
To work around it you could use a scheduled task that you could call via SCHTASKS /RUN.
Also WMIC could help. E.g.

Code: Select all

@echo off &setlocal
if "%~1"=="test" (
  set xyz
  pause
  exit /b
)

set "xyz=Hello!"
set xyz
>nul wmic process call create "cmd /c %~fs0 test"
pause

Regards
aGerman