A batch file debugging library
Posted: 16 Nov 2015 17:53
Hello,
I've been silently following this mailing list for quite some time, and I'd like to thank all participants for the great things I learned here.
Now it's time to contribute something back: First and foremost, I'd like to contribute my batch debugging library.
This library has allowed me to successfully debug and maintain very large system management scripts.
And the techniques published in this forum have allowed me to improve it a lot!
There's still a lot of room for further improvements though, and some of the new discoveries published here this year have the potential to lift many of its limitations and improve it much further.
Hopefully this can inspire some of you!
This library has a long history, and it actually has sister libraries for other scripting languages that I'm not going to detail here.
Over the years they have evolved around the following principles:
Features:
The library is downloadable at:
http://jf.larvoire.free.fr/progs/Library.bat
This file contains the debugging library, many useful routines (many of which were borrowed here!), and debug code for testing the debugger.
Library.bat structure:
I'm not going to describe everything in this post, but just one basic feature:
How to define a traceable function, and running it in normal and debug mode.
As an illustration, I'll use the factorial function defined in Library.bat, which calls itself recursively:
In normal mode, %FUNCTION% evaluates to rem, and %RETURN% evaluates to exit /b. The function runs at full speed, and returns the factorial.
Ex, using Library.bat self-testing capabilities:
In debug mode, %FUNCTION% evaluates to a "call :Debug.Entry", and %RETURN% evaluates to "call :Debug.Return & exit /b". These two calls manage the function tracing. Ex, again using Library.bat self-testing capabilities:
Recently I've added a more sophisticated %RETURN#% macro, that can output any comment in debug mode, not just the RETVAL value like call :Debug.Return does.
Eventually I'd like to define a %RETURN_VARS% macro, that can pass multiple variables back to the parent scope, and display them all in debug mode.
To create new scripts with built-in debugging, the simplest is to copy the whole library.bat into a new script file, then...
To add the debugging library to old scripts
Here the need arises when an old script with no debugging options mysteriously fails. How to debug it, specially when you don't remember how it works, or worse still if it was written by somebody else, unreachable now.
The method is actually very similar to the case for new scripts:
Enjoy!
Jean-François
I've been silently following this mailing list for quite some time, and I'd like to thank all participants for the great things I learned here.
Now it's time to contribute something back: First and foremost, I'd like to contribute my batch debugging library.
This library has allowed me to successfully debug and maintain very large system management scripts.
And the techniques published in this forum have allowed me to improve it a lot!
There's still a lot of room for further improvements though, and some of the new discoveries published here this year have the potential to lift many of its limitations and improve it much further.
Hopefully this can inspire some of you!
This library has a long history, and it actually has sister libraries for other scripting languages that I'm not going to detail here.
Over the years they have evolved around the following principles:
- - Scripts should have built-in debugging capabilities.
- It should be easy to retrofit these debugging capabilities into existing scripts.
- Scripts should work "normally" by default, and have standard options for enabling special debugging modes.
- There should be no, or minimal performance impact when _not_ debugging a script.
- The debug output should be valid batch code (Or their respective scripting languages for its sister libs).
This is very imporatant, as this allows copying and pasting that code into another cmd window to reproduce problems.
Features:
- - Enforce structured programming techniques.
- Defines macros %KEYWORDS% with names and functions that closely match the equivalent batch keyword. Ex: %ECHO% extends what echo does.
- A reusable command-line parsing loop.
- A verbose mode, enabled by the -v option, displaying more details than normal. This is to help users understand what is being done.
- A debug mode, enabled by the -d option, displaying intermediate variables values, tracing code execution, etc. This is to help the script author fix bugs.
- A no-execute mode, aka. WhatIf mode in PowerShell, enabled by the -X option, displaying all "dangerous" commands that should run, without actually running them. This is to help cautious people (both the author and the users) forecast catastrophes that may or may not happen if you run the script.
- In debug mode, trace the execution of function calls and returned values; dump variables; etc.
- Trace and conditionally execute external programs. (If not using the -X option for the NOEXEC mode)
- Debug output indented by function call depth. (Very legible)
The library is downloadable at:
http://jf.larvoire.free.fr/progs/Library.bat
This file contains the debugging library, many useful routines (many of which were borrowed here!), and debug code for testing the debugger.
Library.bat structure:
- - A header comment block, with the file name, description, general notes, MIT license, and global change log.
- A short library initialization: Defines a few important variables, and calls initialization routines for the core modules.
- A macro definition module. (A new addition, now used my the optional %RETURN#% macro.)
- The debug library core module. Begins by a header block describing it in details, with a change log. This module is required in all cases.
- The execution tracing library module. Begins by a header describing it in details, with a change log. This module is optional.
- A banner showing the end of the core debug library modules.
- An eclectic list of useful routines collected over time. (Many of which coming from this forum.)
- Routines for debugging the debugging library itself. (Yes, debuggers may have bugs too :-)
- The header comment block for the main routine
- The Help routine
- The main routine, beginning by command-line arguments parsing.
I'm not going to describe everything in this post, but just one basic feature:
How to define a traceable function, and running it in normal and debug mode.
As an illustration, I'll use the factorial function defined in Library.bat, which calls itself recursively:
Code: Select all
:Fact
%FUNCTION% Fact %*
setlocal enableextensions enabledelayedexpansion
set N=%1
if .%1.==.. set N=0
if .%N%.==.0. (
set RETVAL=1
) else (
set /A M=N-1
call :Fact !M!
set /A RETVAL=N*RETVAL
)
endlocal & set "RETVAL=%RETVAL%" & %RETURN%
- * The function label must be followed by a line with the macro %FUNCTION%, followed by the function name, and %*.
* The setlocal directive is optional, but recommended to allow defining local variables.
* The last line uses the now standard (in this forum) technique for returning strings to the parent scope.
* All exits from a %FUNCTION% MUST be done via a %RETURN%.
* In debug mode, the value of the RETVAL variable will be displayed.
In normal mode, %FUNCTION% evaluates to rem, and %RETURN% evaluates to exit /b. The function runs at full speed, and returns the factorial.
Ex, using Library.bat self-testing capabilities:
Code: Select all
C:\JFL\SRC\Batch>Library.bat -c "call :Fact 5" "call :echovars RETVAL"
set "RETVAL=120"
In debug mode, %FUNCTION% evaluates to a "call :Debug.Entry", and %RETURN% evaluates to "call :Debug.Return & exit /b". These two calls manage the function tracing. Ex, again using Library.bat self-testing capabilities:
Code: Select all
C:\JFL\SRC\Batch>Library.bat -d -c "call :Fact 5" "call :echovars RETVAL"
call :Fact 5
call :Fact 4
call :Fact 3
call :Fact 2
call :Fact 1
call :Fact 0
return 1
return 1
return 2
return 6
return 24
return 120
set "RETVAL=120"
Recently I've added a more sophisticated %RETURN#% macro, that can output any comment in debug mode, not just the RETVAL value like call :Debug.Return does.
Eventually I'd like to define a %RETURN_VARS% macro, that can pass multiple variables back to the parent scope, and display them all in debug mode.
To create new scripts with built-in debugging, the simplest is to copy the whole library.bat into a new script file, then...
- Update the name, description, notes, and change log in the file header.
Remove the unneeded sections. Typically everything between the end banner of the debugging library, and the main routine header in the end.
Remove the unnecessary options in the command-line processing loop in the main routine. (Those used for testing the library itself)
Update the help screen.
Replace the main routine content with your new code.
Prefix all "dangerous" commands with the %EXEC% macro. This automates the no-exec operation.
Add "%ECHOVARS.D% variable ..." lines for all tricky intermediate variables that change value.
Use %FUNCTION% and %RETURN% or %RETURN#% directives for all your subroutines that need to be traced.
All this is actually easier to do than it sounds. Writing the main routine has to be done anyway. Writing a good header and help screen is good practice. And the time lost in deleting the unneeded sections is negligible, compared to the time saved writing a command-line options processing loop.
To add the debugging library to old scripts
Here the need arises when an old script with no debugging options mysteriously fails. How to debug it, specially when you don't remember how it works, or worse still if it was written by somebody else, unreachable now.
The method is actually very similar to the case for new scripts:
- Insert a copy of the begugging library core sections immediately behind your script header comment.
Add to your command-line processing loop the management for options -d, and possibly -v and -X from the library.
Prefix all "dangerous" commands in the script with the %EXEC% macro. (Commands deleting files, writing data, etc.)
Insert "%ECHOVARS.D% variable ..." lines for all tricky intermediate variables that change value.
Add %FUNCTION% and %RETURN% or %RETURN#% directives for all your subroutines that need to be traced.
Finally run the modified script with the -d option, to see what happens internally before the problem occurs.
Enjoy!
Jean-François
ebug.Entry", and %RETURN% evaluates to "call 
)
endlocal bypass techniques!