DosTips.com

A Forum all about DOS Batch
https://www.dostips.com/forum/

exit function and preserve variable over endlocal barrier

https://www.dostips.com/forum/viewtopic.php?f=3&t=6496

Page 1 of 2

exit function and preserve variable over endlocal barrier

Posted: 09 Jun 2015 11:36
by jeb
The topic Tricky way to detect calls from another batch script brings me to a new way of breaking the endlocal barrier.

Once, I proclaimed that you need a macro to breaking the endlocal barrier,
because caliing a function for this obviously :? can't work, as in a called function you can only ENDLOCAL the SETLOCAL of exactly this function.

But I'm wrong, it can be done with a simple GOTO, as shown in the other post.

The advantage is here that you can exit a function and bring a variable over the barrier without knowing the exact count of SETLOCALs.

Code: Select all

@echo off
setlocal
set "result="
setlocal EnableDelayedExpansion
call :func result
echo The result is !result!
exit /b

:func
setlocal DisableDelayedExpansion
setlocal
setlocal
setlocal
set "%1=caret ^ and bang !"
call :exitAndPreserve %1
exit /b Will never be reached

:exitAndPreserve
setlocal EnableDelayedExpansion
set "varName=%1"
set "varContent=!%1!"
echo !varName!=!varContent!
for /F "delims=" %%N in ("!varName!") DO (
    for /F "delims=" %%C in ("!varContent!") DO (
        (goto) 2>NUL
        (goto) 2>NUL
        set "%%N=%%C"
    )
)


As you see this is only a simple demonstration, as it currently lacks the handling of some special characters (caret, exclamation mark, linefeed and carriage return)

A fully realized implementation of RETURN.BAT is available at viewtopic.php?f=3&t=6496&p=41929#p41929

Re: exit function and preserve variable over endlocal barrie

Posted: 09 Jun 2015 12:50
by Aacini
I like this! :D A slightly modified version:

Code: Select all

:exitAndPreserve
setlocal EnableDelayedExpansion
set "varName=%1"
set "varContent=!%1!"
echo !varName!=!varContent!
for /F "tokens=1* delims=:" %%N in ("!varName!:!varContent!") DO (
   rem Cancel exitAndPreserve (this) function
   (goto) 2>NUL
   rem Cancel the last caller function, func in this case
   (goto) 2>NUL
   rem ... and define the value in the environment of the original caller
   set "%%N=%%O"
)
exit /b Will never be reached

Antonio

Re: exit function and preserve variable over endlocal barrie

Posted: 09 Jun 2015 13:29
by jeb
Aacini wrote:for /F "tokens=1* delims=:" %%N in ("!varName!:!varContent!") DO (
rem Cancel exitAndPreserve (this) function

Thank you, more easy and better comments (okay, I haven't comments at all).

But then it should be more stable without any delimiter to avoid problems with the beginning of the varContent.

Code: Select all

...
for /F "delims=" %%N in ("!varName!=!varContent!") DO (
   ...
   rem ... and define the value in the environment of the original caller
   set "%%N"
)

Re: exit function and preserve variable over endlocal barrie

Posted: 09 Jun 2015 15:14
by Aacini
jeb wrote:
Aacini wrote:for /F "tokens=1* delims=:" %%N in ("!varName!:!varContent!") DO (
rem Cancel exitAndPreserve (this) function

Thank you, more easy and better comments (okay, I haven't comments at all).

But then it should be more stable without any delimiter to avoid problems with the beginning of the varContent.

Code: Select all

...
for /F "delims=" %%N in ("!varName!=!varContent!") DO (
   ...
   rem ... and define the value in the environment of the original caller
   set "%%N"
)


Yes, you are right! :)

Here it is another application of this trick:
EDIT: I did a very small modification in original code.

Code: Select all

@echo off

rem whereIs.bat fileName [baseDir]

setlocal
set "baseDir=%~2"
if not defined baseDir set "baseDir=%cd%"
set level=0
set "result="
call :recursiveSearch %1 "%baseDir%"
if defined result (
   echo File "%~1" found at "%result%"
) else (
   echo File not found
)
goto :EOF

:recursiveSearch fileName dir
setlocal
set /A level+=1
cd %2
if exist %1 (
   for /L %%i in (1,1,%level%) do (goto) 2> NUL
   set "result=%cd%"
)

for /D %%d in (*) do if not defined result (
   call :recursiveSearch %1 "%%d"
)
cd ..
exit /b

Something strange happened with previous code. Accordingly with the expected behaviour, the %level% number of goto's should cancel the same number of pending recursive returns, but after that, the execution returned to the next iteration of the FOR command of the first level of the recursive subroutine. If I incremented the level by 1, the execution returned to the comand-line level, so that was wrong. The way to solve this problem was inserting the "if not defined result" part in this line:

Code: Select all

for /D %%d in (*) do if not defined result (

It seems that a (goto) command cancel the execution of a subroutine excepting the first one if the call is executed inside an iterative FOR command! More testings on this point are needed...

Antonio

Re: exit function and preserve variable over endlocal barrie

Posted: 09 Jun 2015 15:51
by jeb
Aacini wrote:It seems that a (goto) command cancel the execution of a subroutine excepting the first one if the call is executed inside an iterative FOR command! More testings on this point are needed...

Not quite correct, the (goto) pop the call stack, BUT it can't cancel any cached parenthesis blocks (that's also true for multiple commands in a line, seperated by ampersands) :!:

Code: Select all

@echo off
setlocal
set count=1
call :subFunc1
echo End of %0 count=%count%
exit /b

:subFunc1
setlocal
set /a count+=1
echo   Func START %0  count=%count%
(
  call :subFunc2 param_subFunc2
  echo FUNC %0 !count! This is visible!!
)
echo FUNC %0 This will never be executed
echo   Func END %0
exit /b

:subFunc2
set var=start.1
setlocal EnableDelayedExpansion
set var=start.2
set /a count+=1
echo   Func START %0  count=%count%

  call :subFunc3 param_subFunc3   &  echo FUNC %0 count=!count! This is visible!!

echo FUNC %0 This will never be executed
echo   Func END   %0
exit /b

:subFunc3
setlocal DisableDelayedExpansion
set /a count+=1
echo   Func START %0  count=%count%
call :cancel
echo FUNC %0 count=!count! This is visible!!
echo   Func END   %0
exit /b

:cancel
echo(
echo *************** CANCEL STARTS  %count%
set CANCEL=3
for /L %%n in (5 -1 0) DO (
    (goto) 2>NUL 
    call set "isCmdLineContext=%%"
    if defined isCmdLineContext (
        echo   #%%n * isCmdLineContext
    ) ELSE (
        call echo   #%%n - %%0 %%1 cnt=%%count%% %%=undefined%%
    )
    if %%n == 0 (
        echo *************** CANCEL ENDS
        echo(
    )
)
echo *************** NEVER COMES HERE CANCEL ENDS
echo(
exit /b NEVER REACHED THIS



output wrote: Func START :subFunc1 count=2
Func START :subFunc2 count=3
Func START :subFunc3 count=4

*************** CANCEL STARTS 4
#5 - :subFunc3 param_subFunc3 cnt=4
#4 - :subFunc2 param_subFunc2 cnt=3
#3 - :subFunc1 cnt=2
#2 - gotoTest1.bat cnt=1
#1 * isCmdLineContext
#0 * isCmdLineContext
*************** CANCEL ENDS

FUNC :subFunc2 count=!count! This is visible!!
FUNC :subFunc1 !count! This is visible!!


That's my main problem in solving the call stack dumper problem with the GetLineNumber technic.

How to cancel a parenthesis block or an multiple command block :?: :(
And the file position should be still directly behind the block, so the line number can be determined.

Re: exit function and preserve variable over endlocal barrie

Posted: 09 Jun 2015 17:14
by dbenham
Great technique. :D

It is also very nice that there is no need to include the exit function in your script. Instead it can be called as a stand-alone script that resides somewhere within your PATH.


Dave Benham

Re: exit function and preserve variable over endlocal barrie

Posted: 05 Jul 2015 07:56
by dbenham
Here is a fully developed RETURN function that can safely return any value across the ENDLOCAL barrier, regardless whether the parent context has delayed expansion enabled or disabled.

I want the function to be able to return to a command line context, so I cannot use IF "!!" EQU "" to test for delayed expansion. Instead I rely on the fact that COMSPEC should always be defined, so I can use IF "!COMSPEC!" EQU "%COMSPEC%".

EDIT - I've adopted jeb's brilliant idea to use IF "^!^" equ "^!" instead.

2017-04-29 EDIT - v2.1 Bug fix for when return value should be undefined. Thanks to random for reporting the bug

2017-08-21 EDIT - v3.0 Added support for return of multiple variables

The RETURN.BAT code is designed to be a stand-alone utility that can be placed in a folder referenced by PATH so it can easily be called by any script. But instructions are included within the code on how to embed the function directly within your own script.

RETURN.BAT Version 3.0

Code: Select all

::RETURN.BAT Version 3.0
@if "%~2" equ "" (goto :return.special) else goto :return
:::
:::call RETURN  ValueVar  ReturnVar  [ErrorCode]
:::  Used by batch functions to EXIT /B and safely return any value across the
:::  ENDLOCAL barrier.
:::    ValueVar  = The name of the local variable containing the return value.
:::    ReturnVar = The name of the variable to receive the return value.
:::    ErrorCode = The returned ERRORLEVEL, defaults to 0 if not specified.
:::
:::call RETURN "ValueVar1 ValueVar2 ..." "ReturnVar1 ReturnVar2 ..." [ErrorCode]
:::  Same as before, except the first and second arugments are quoted and
:::  space delimited lists of variable names.
:::
:::  Note that the total length of all assignments (variable names and values)
:::  must be less then 3.8k bytes. No checks are performed to verify that all
:::  assignments fit within the limit. Variable names must not contain space,
:::  tab, comma, semicolon, caret, asterisk, question mark, or exclamation point.
:::
:::call RETURN  init
:::  Defines return.LF and return.CR variables. Not required, but should be
:::  called once at the top of your script to improve performance of RETURN.
:::
:::return /?
:::  Displays this help
:::
:::return /V
:::  Displays the version of RETURN.BAT
:::
:::
:::RETURN.BAT was written by Dave Benham and DosTips user jeb, and was originally
:::posted within the folloing DosTips thread:
:::  http://www.dostips.com/forum/viewtopic.php?f=3&t=6496
:::
::==============================================================================
::  If the code below is copied within a script, then the :return.special code
::  can be removed, and your script can use the following calls:
::
::    call :return   ValueVar  ReturnVar  [ErrorCode]
::
::    call :return.init
::

:return  ValueVar  ReturnVar  [ErrorCode]
:: Safely returns any value(s) across the ENDLOCAL barrier. Default ErrorCode is 0
setlocal enableDelayedExpansion
if not defined return.LF call :return.init
if not defined return.CR call :return.init
set "return.normalCmd="
set "return.delayedCmd="
set "return.vars=%~2"
for %%a in (%~1) do for /f "tokens=1*" %%b in ("!return.vars!") do (
  set "return.normal=!%%a!"
  if defined return.normal (
    set "return.normal=!return.normal:%%=%%3!"
    set "return.normal=!return.normal:"=%%4!"
    for %%C in ("!return.LF!") do set "return.normal=!return.normal:%%~C=%%~1!"
    for %%C in ("!return.CR!") do set "return.normal=!return.normal:%%~C=%%2!"
    set "return.delayed=!return.normal:^=^^^^!"
  ) else set "return.delayed="
  if defined return.delayed call :return.setDelayed
  set "return.normalCmd=!return.normalCmd!&set "%%b=!return.normal!"^!"
  set "return.delayedCmd=!return.delayedCmd!&set "%%b=!return.delayed!"^!"
  set "return.vars=%%c"
)
set "err=%~3"
if not defined err set "err=0"
for %%1 in ("!return.LF!") do for /f "tokens=1-3" %%2 in (^"!return.CR! %% "") do (
  (goto) 2>nul
  (goto) 2>nul
  if "^!^" equ "^!" (%return.delayedCmd:~1%) else %return.normalCmd:~1%
  if %err% equ 0 (call ) else if %err% equ 1 (call) else cmd /c exit %err%
)

:return.setDelayed
set "return.delayed=%return.delayed:!=^^^!%" !
exit /b

:return.special
@if /i "%~1" equ "init" goto return.init
@if "%~1" equ "/?" (
  for /f "tokens=* delims=:" %%A in ('findstr "^:::" "%~f0"') do @echo(%%A
  exit /b 0
)
@if /i "%~1" equ "/V" (
  for /f "tokens=* delims=:" %%A in ('findstr /rc:"^::RETURN.BAT Version" "%~f0"') do @echo %%A
  exit /b 0
)
@>&2 echo ERROR: Invalid call to RETURN.BAT
@exit /b 1


:return.init  -  Initializes the return.LF and return.CR variables
set ^"return.LF=^

^" The empty line above is critical - DO NOT REMOVE
for /f %%C in ('copy /z "%~f0" nul') do set "return.CR=%%C"
exit /b 0


Test script:

Code: Select all

@echo off
setlocal disableDelayedExpansion
call return init
for /f %%C in ('copy /z "%~f0" nul') do set "\r=%%C"
set ^"\n=^

^"

set test1="This & that" ^& the other thing
set test2="This & that" ^& the other thing!
set test3="&<>|%%^" ^&^<^>^|%%^^
set test4="&<>|%%^!" ^&^<^>^|%%^^!
setlocal enableDelayedExpansion
set "test5=!\n!Line One!\n!hidden!\r!Line Two"
set "test6="

set "err1=0"
set "err2=1"
set "err3=2"
set "err4="
set "err5="
set "err6=1"

set "normal=preExistingValue"
set "delayed=preExistingValue"

setlocal disableDelayedExpansion
echo Delayed expansion is DISABLED
:loop
echo ------------------------------------------
for /l %%N in (1 1 6) do (
  set "result1="
  set "result2="
  call :test test%%N result err%%N
  call set "err=%%errorlevel%%"
  setlocal enableDelayedExpansion
  echo result=!result!
  echo errorlevel=!err!
  endlocal
)
for /l %%N in (1 1 6) do set "result%%N="
echo(
echo Multiple values test:
call :test2
set "err=%errorlevel%"
setlocal enableDelayedExpansion
for /l %%N in (1 1 6) do (
  echo   test%%N=!test%%N!
  echo result%%N=!result%%N!
)
echo errorlevel=%err%
endlocal
if "!!" equ "" exit /b
echo(
echo(
setlocal enableDelayedExpansion
echo Delayed expansion is ENABLED
goto :loop


:test
setlocal enableDelayedExpansion
set rtn=!%1!
echo(
echo  %1=!rtn!
call return "rtn" "%2" !%3!
exit /b

:test2
setlocal enableDelayedExpansion
for /l %%N in (1 1 6) do set "rtn%%N=!test%%N!"
call return "rtn1 rtn2 rtn3 rtn4 rtn5 rtn6" "result1 result2 result3 result4 result5 result6"  -99
exit /b
--OUTPUT--

Code: Select all

Delayed expansion is DISABLED
------------------------------------------

 test1="This & that" & the other thing
result="This & that" & the other thing
errorlevel=0

 test2="This & that" & the other thing!
result="This & that" & the other thing!
errorlevel=1

 test3="&<>|%^" &<>|%^
result="&<>|%^" &<>|%^
errorlevel=2

 test4="&<>|%^!" &<>|%^!
result="&<>|%^!" &<>|%^!
errorlevel=0

 test5=
Line One
Line Two
result=
Line One
Line Two
errorlevel=0

 test6=
result=
errorlevel=1

Multiple values test:
  test1="This & that" & the other thing
result1="This & that" & the other thing
  test2="This & that" & the other thing!
result2="This & that" & the other thing!
  test3="&<>|%^" &<>|%^
result3="&<>|%^" &<>|%^
  test4="&<>|%^!" &<>|%^!
result4="&<>|%^!" &<>|%^!
  test5=
Line One
Line Two
result5=
Line One
Line Two
  test6=
result6=
errorlevel=-99


Delayed expansion is ENABLED
------------------------------------------

 test1="This & that" & the other thing
result="This & that" & the other thing
errorlevel=0

 test2="This & that" & the other thing!
result="This & that" & the other thing!
errorlevel=1

 test3="&<>|%^" &<>|%^
result="&<>|%^" &<>|%^
errorlevel=2

 test4="&<>|%^!" &<>|%^!
result="&<>|%^!" &<>|%^!
errorlevel=0

 test5=
Line One
Line Two
result=
Line One
Line Two
errorlevel=0

 test6=
result=
errorlevel=1

Multiple values test:
  test1="This & that" & the other thing
result1="This & that" & the other thing
  test2="This & that" & the other thing!
result2="This & that" & the other thing!
  test3="&<>|%^" &<>|%^
result3="&<>|%^" &<>|%^
  test4="&<>|%^!" &<>|%^!
result4="&<>|%^!" &<>|%^!
  test5=
Line One
Line Two
result5=
Line One
Line Two
  test6=
result6=
errorlevel=-99


And here is a script to partially test returning to a command line context:

testReturn.bat

Code: Select all

@echo off
setlocal disableDelayedExpansion
set ^"original=^

Hello world!^

"&<>|%%^!" ^&^<^>^|%%^^!^"
set original
call return original %1 0
Test results:

Code: Select all

C:\test>testReturn disabledResult
original=
Hello world!
"&<>|%^!" &<>|%^!

C:\test>set disabledResult
disabledResult=
Hello world!
"&<>|%^!" &<>|%^!

C:\test>cmd /v:on
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\test>testReturn enabledResult
original=
Hello world!
"&<>|%^!" &<>|%^!

C:\test>set enabledResult
enabledResult=
Hello world!
"&<>|%^!" &<>|%^!


Dave Benham

Re: exit function and preserve variable over endlocal barrie

Posted: 05 Jul 2015 12:50
by Aacini
[OFF TOPIC]

Every time I read "the endlocal barrier" phrase, I think on images from the Great Ice Barrier in Antarctica... :mrgreen:

In structured programming languages there is a concept, called "scope", that refers to the place in a program where variables can be read or set. When I see this problem, I think on "return values to the environment of the parent program"; I don't see any "barrier" here... :roll:

Antonio

Re: exit function and preserve variable over endlocal barrie

Posted: 28 Oct 2015 09:57
by dbenham
@Aacini: I believe we are all aware of the concept of scope - and the environment variable scope is a barrier when it comes to batch programming.

Most languages have one or more formal mechanisms to transfer values across scope boundaries. But the only formal mechanism batch offers is ERRORLEVEL, and that is limited to 32 bit integers, and it may not be possible to differentiate errors from return values, depending on context.

The intrepid batch community has developed various strategies to pass values across the ENDLOCAL (scope) boundary, but all the simple strategies have limitations that make them unsuitable as a general purpose transfer mechanism. It is impossible to guarantee success unless you know that the set of all possible return values will never violate any of the limitations. So I would say that ENDLOCAL is a formidable barrier indeed.

The techniques in this thread are perhaps the most reliable and efficient way to robustly solve the problem in a relatively easy fashion that is easy to use.


Dave Benham

Re: exit function and preserve variable over endlocal barrie

Posted: 28 Oct 2015 22:54
by Aacini
@Dave: I agree in all your points, excepting in the term used: "barrier". Of course, we all -experienced programmers- know what this point is about, and we could call this mechanism in any other way ("limit", "boundary", "constrain", etc) and still we would know what the term is about. The important point is that the mechanisms developed here for Batch files works in the exact same way of the formal mechanisms that most languages have to solve the same problem!. The problems with this nomenclature appears when a beginner comes here for advice and realize that the experts call "endlocal barrier" a mechanism that blocks a value from pass through it...

IMHO it is very important that the beginner realize that is not the value what is "blocked" by the "barrier", but the variable itself. However, how the setlocal barrier can block a variable?. This point is very hard to explain using the "barrier" terminology! The mechanism can be explained in a simpler way if the "scope" concept is introduced from the very beginning. The endlocal is not a barrier that "blocks" a variable, but the variable terminate its existence at the endlocal command and there is no way to "break" this mechanism!. Hence, the goal is to pass a value (not a variable) from the scope before the endlocal to the scope after the endlocal. This is true no matter how you call this management, nor the lack of official commands or documentation about this management, nor the additional problems and special cases that the very nature of Batch files imposes in this type of management.

There is a question I frequently did on similar topics: If this same concept/mechanism exist in other programming languages and in all of them it is called with a given name, why we call it with a different name in Batch files?

If a beginner don't completely understand the explanation of the endlocal mechanism given here and look for additional information at other sites (Wikipedia for example), the OP will not find any information about "endlocal barrier". If the search term is changed to just "barrier" then several articles will appear, two of which are computer-related: a synchronization method in parallel computing, and an instruction for block memory operations.

On the other hand, if the OP look for "endlocal scope" instead, a link to cmd.exe will appear that indicate: "SetLocal and EndLocal commands limit the scope of changes to the environment". If the search term is changed to just "scope", then a link to "Scope (computer science)" will appear with the full information about this topic.

Why call "barrier" what is commonly known as "scope"?

Antonio

Re: exit function and preserve variable over endlocal barrier

Posted: 30 Nov 2015 09:30
by jfl
FYI I've posted in http://www.dostips.com/forum/viewtopic.php?f=3&t=6796&p=44199#p44199 a %RETURN% macro, partly inspired by your return.bat, that allows returning any number of variables back to the caller's variable scope.
The previous posts in that thread explain the associated %FUNCTION% and %UPVAR% macros; How they work; And the differences with your return method.

Jean-François

Re: exit function and preserve variable over endlocal barrier

Posted: 16 Aug 2016 03:32
by jeb
dbenham wrote:I want the function to be able to return to a command line context, so I cannot use IF "!!" EQU "" to test for delayed expansion. Instead I rely on the fact that COMSPEC should always be defined, so I can use IF "!COMSPEC!" EQU "%COMSPEC%".


I didn't like this part, as it's needs a more or less changeable variable.
I first thought of using

Code: Select all

IF "!=exitcode!" EQU "%=exitcode%"

But I found a better, total simple solution

Code: Select all

IF "^!" == "^!^"


It relies on the fact, that the last caret will be removed only in delayed mode.

Re: exit function and preserve variable over endlocal barrier

Posted: 16 Aug 2016 05:27
by dbenham
Brilliant :idea: I love it :!: :D

I've updated my code to version 2.0 with this improvement.


Dave Benham

Re: exit function and preserve variable over endlocal barrier

Posted: 22 Aug 2016 00:03
by thefeduke
I am risking some basic questions on scope and barriers. I have been wrestling with CHDIR and PUSHD.

Is there such a concept as "implied variables" that would apply to these commands? HELP is not much. I cannot find any help for one of the most basic of commands: the "<drive letter><colon>". If I enter "E:" from a C:\ prompt I appear to get the same result as if I had entered "CD /D E:"

I was attracted to some code I saw in "Post subject: Re: Better way to find a folder and list contents?". I modified it to do a PUSHD of the found folder, and naively used SETLOCAL so as to keep a few variables from cluttering the environment, only to find that my prompt did not change. Because I could correct the obstacle with:

Code: Select all

EndLOCAL & PushD "%Folder%"
I concluded that %CD% had been localized.

My next step would have been to add one of the subdirectories to the PATH, but now I expect to be respecting the scope of %PATH%.

So, what really happens when you enter "E:"?

John A.

Re: exit function and preserve variable over endlocal barrier

Posted: 22 Aug 2016 02:18
by foxidrive
thefeduke wrote:So, what really happens when you enter "E:"?

John A.


It changes the working drive.
You will be plonked into the last current directory on that drive.

Code: Select all

x:\>cd d:\backup

x:\>d:

d:\Backup>


The first command changed the working directory on drive d: but didn't change drive.
The second one changed the working drive.

Each drive has it's own working directory that is remembered by MSDOS/Windows when it is changed.
All times are UTC-06:00
Page 1 of 2
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/