:Concatenate function, advanced version, custom separator, multiple inputs, byval, byref and byref array ! (but oneprob)

[shodan]

Hi,

I needed to take all the elements of an array and make one string with them.

So I wrote this :Concatenate function which is very versatile and works in most cases

See below for a demonstration

Code: Select all

::Usage Call :Concatenate optional (SEPARATOR "X") InputArray1 InputArray2 InputArrayN OutputValue
:Concatenate
if "[%~1]" EQU "[SEPARATOR]" ( set "_Concatenate_separator=%~2" & shift & shift & GoTo :Concatenate )
set "_Concatenate_prefix=_CA"
if defined %1.ubound ( set "_Concatenate_input=%~1" ) else ( set "_Concatenate_input=Concatenate_placeholder" )
if defined %1.lbound call set /a _Concatenate_lbound=%%%~1.lbound%%
if defined %1.ubound call set /a _Concatenate_ubound=%%%~1.ubound%%
if not defined %1.lbound set /a "_Concatenate_lbound=0"
if not defined %1.ubound set /a "_Concatenate_ubound=0"
set /a "_Concatenate_index=%_Concatenate_lbound%"
setlocal enabledelayedexpansion
if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] if defined %1 set %_Concatenate_input%[%_Concatenate_index%]=!%1!
if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] set "%_Concatenate_input%[%_Concatenate_index%]=%~1"
if defined !_Concatenate_separator! ( set _Concatenate_separator=!%_Concatenate_separator%! )
if not defined _Concatenate_separator set "_Concatenate_separator= "
:Concatenate-loop
set _Concatenate_buffer=!_Concatenate_buffer!!%_Concatenate_input%[%_Concatenate_index%]!!_Concatenate_separator!
set /a "_Concatenate_index+=1"
if %_Concatenate_index% LEQ %_Concatenate_ubound% GoTo :Concatenate-loop
for /f "tokens=* delims=" %%a in ('echo.!_Concatenate_buffer!') do (
																endlocal
																set _Concatenate_buffer=%%a
																)
if "[%~3]" NEQ "[]" ( shift & GoTo :Concatenate ) 
setlocal enabledelayedexpansion
for /f "tokens=* delims==" %%a in ('echo.!_Concatenate_buffer!') do (
																endlocal
																set %~2=%%a
																)
Call :ClearVariablesByPrefix %_Concatenate_prefix% _Concatenate
GoTo :EOF

First, you can give it arguments by value

Code: Select all

Call :Concatenate "abc" "def" "ghi" OutputVar

result:

Code: Select all

abc def ghi

Arguments by reference

Code: Select all

set myvar=jkl
set myothervar=mno
set myotherothervar=pqr

Call :Concatenate myvar myothervar myotherothervar OutputVar

result :

Code: Select all

jkl mno pqr

Arguments by reference array

Code: Select all

set myarray[0]=stu
set myarray[1]=vwx
set myarray[2]=yzA
set /a "myarray.ubound=2"

Call :Concatenate myarray OutputVar

result:

Code: Select all

stu vwx yzA

You can specify a separator, here all the previous methods are used together

Code: Select all

Call :Concatenate SEPARATOR 1 "abc" "def" "ghi" myvar myothervar myotherothervar myarray OutputVar

result:

Code: Select all

abc1def1ghi1jkl1mno1pqr1stu1vwx1yzA1

And, you can change the moderator between each arguments

Code: Select all

Call :Concatenate "abc" SEPARATOR 1 "def" "ghi" myvar SEPARATOR 2 myothervar myotherothervar SEPARATOR " " myarray OutputVar

result:

Code: Select all

abc def1ghi1jkl1mno2pqr2stu vwx yzA

Here is the full DEMO file
Includes a debug version to explain "the problem"

Concatenate-DEMO.zip

(2.21 KiB) Downloaded 519 times

Now here is "the problem"


I would like to use a newline for the separator and without using enabledelayedexpansion too !
It almost works, the function works, but getting it past endlocal is proving to be a challenge.

Here is the code for that version

Code: Select all

set myarray[0]=Mary had a little lamb,
set myarray[1]=Its fleece was white as snow, yeah.
set myarray[2]=Everywhere the child went,
set myarray[3]=The little lamb was sure to go, yeah.
set /a "myarray.lbound=0"
set /a "myarray.ubound=3"

set "myarray2[-1]= "
set myarray2[0]=He followed her to school one day,
set myarray2[1]=And broke the teacher's rule.
set myarray2[2]=What a time did they have,
set myarray2[3]=That day at school.
set /a "myarray2.lbound=-1"
set /a "myarray2.ubound=3"

set "myarray3[-1]= "
set myarray3[0]=Tisket, tasket,
set myarray3[1]=A green and yellow basket.
set myarray3[2]=Sent a letter to my baby,
set myarray3[3]=On my way I passed it.
set /a "myarray3.lbound=-1"
set /a "myarray3.ubound=3"

echo -------------------------------
Call :Concatenate myarray myarray2 myarray3 _myConcatenatedString1

echo.&echo.&echo result for Call :Concatenate myarray myarray2 myarray3 _myConcatenatedString1
echo Demonstrate concatenate of all elements within 3 array, default separator of " " will be used
echo.&echo %_myConcatenatedString1%

The output of that is, as expected

Code: Select all

Mary had a little lamb, Its fleece was white as snow, yeah. Everywhere the child went, The little lamb was sure to go, yeah.   He followed her to school one day, And broke the teacher's rule. What a time did they have, That day at school.   Tisket, tasket, A green and yellow basket. Sent a letter to my baby, On my way I passed it.

Here is the version with newline attempt

It does not work, no output is set

Code: Select all

BTW you can specify SEPARATOR both byval and byref

rem Two empty lines above are essential
set newline=^


rem Two empty lines above are essential

set debug=true
echo.&echo.&echo.

Call :Concatenate SEPARATOR newline myarray myarray2 myarray3 TheOutput

echo.&echo.&echo result for Call :Concatenate SEPARATOR newline myarray myarray2 myarray3 TheOutput
echo.&echo %TheOutput%

The problem is, the for loop that do the endlocal, break. So it stays in setlocal until GoTo :EOF at which point the data is lost.

I have created a debug version, it doesn't do much different, but it shows that it largely works except you can't get the data out !

Only solution I see is, a temp file, but I rather not !

If I am not mistaken this is a very high difficulty question this time !

Good luck

Here is the output of the debug version

Code: Select all

so close to working

Mary had a little lamb,
 Its fleece was white as snow, yeah.
 Everywhere the child went,
 The little lamb was sure to go, yeah.

 He followed her to school one day,
 And broke the teacher's rule.
 What a time did they have,
 That day at school.

 Tisket, tasket,
 A green and yellow basket.
 Sent a letter to my baby,
 On my way I passed it.


but I can't get the text out of setlocal :(
set TheOutput=_Concatenate_buffer


result for Call :Concatenate SEPARATOR newline myarray myarray2 myarray3 TheOutput

ECHO is off.
Environment variable TheOutput not defined

Here is the debug code, it is not much help unfortunately

Code: Select all

::Usage Call :Concatenate optional (SEPARATOR "X") InputArray1 InputArray2 InputArrayN OutputValue
:Concatenate-debug
if "[%~1]" EQU "[SEPARATOR]" ( set "_Concatenate_separator=%~2" & shift & shift & GoTo :Concatenate-debug )
set "_Concatenate_prefix=_CA"
if defined %1.ubound ( set "_Concatenate_input=%~1" ) else ( set "_Concatenate_input=Concatenate_placeholder" )
if defined %1.lbound call set /a _Concatenate_lbound=%%%~1.lbound%%
if defined %1.ubound call set /a _Concatenate_ubound=%%%~1.ubound%%
if not defined %1.lbound set /a "_Concatenate_lbound=0"
if not defined %1.ubound set /a "_Concatenate_ubound=0"
set /a "_Concatenate_index=%_Concatenate_lbound%"
setlocal enabledelayedexpansion
if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] if defined %1 set %_Concatenate_input%[%_Concatenate_index%]=!%1!
if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] set "%_Concatenate_input%[%_Concatenate_index%]=%~1"
if defined !_Concatenate_separator! ( set _Concatenate_separator=!%_Concatenate_separator%! )
if not defined _Concatenate_separator set "_Concatenate_separator= "
:Concatenate-debug-loop
set _Concatenate_buffer=!_Concatenate_buffer!!%_Concatenate_input%[%_Concatenate_index%]!!_Concatenate_separator!
set /a "_Concatenate_index+=1"
if %_Concatenate_index% LEQ %_Concatenate_ubound% GoTo :Concatenate-debug-loop
REM ----------DEBUG----------
if "[%debug%]" NEQ "[true]" for /f "tokens=* delims=" %%a in ('echo.!_Concatenate_buffer!') do (
																endlocal
																REM ----------DEBUG----------
																if "[%debug%]" EQU "[true]" echo This never runs 1 a%%aa
																REM ----------DEBUG----------
																set _Concatenate_buffer=%%a
																)
REM ----------DEBUG----------
if "[%~3]" NEQ "[]" ( shift & GoTo :Concatenate-debug ) 

REM ----------DEBUG----------
if "[%debug%]" EQU "[true]" ( echo so close to working&echo. )
if "[%debug%]" EQU "[true]" echo !_Concatenate_buffer!
if "[%debug%]" EQU "[true]" ( echo.&echo but I can't get the text out of setlocal :^( )
REM ----------DEBUG----------

setlocal enabledelayedexpansion
REM ----------DEBUG----------
if "[%debug%]" NEQ "[true]" for /f "tokens=* delims==" %%a in ('echo.!_Concatenate_buffer!') do (
																endlocal
																REM ----------DEBUG----------
																if "[%debug%]" EQU "[true]"  echo This never runs 2 a%%aa
																REM ----------DEBUG----------
																set %~2=%%a
																)
REM ----------DEBUG----------
if "[%debug%]" EQU "[true]" echo set %~2=_Concatenate_buffer
REM if "[%debug%]" EQU "[true]" set %~2=!_Concatenate_buffer!
set %~2=!_Concatenate_buffer!
REM ----------DEBUG----------
if "[%debug%]" NEQ "[true]" Call :ClearVariablesByPrefix %_Concatenate_prefix% _Concatenate
REM ----------DEBUG----------
GoTo :EOF

[shodan]

I have asked chatgpt to document how this function works

https://chat.openai.com/share/6209be7d- ... e1fbc1352c


Please explain the theory of operation of this batch subfunction
(script is pasted here)

ChatGPT

This script defines a :Concatenate subfunction that concatenates the values of one or more input arrays with a separator. Here's the breakdown of its theory of operation:

Usage:
The function can be called like this:
Call :Concatenate optional (SEPARATOR "X") InputArray1 InputArray2 ... InputArrayN OutputValue
If SEPARATOR "X" is provided, then "X" will be used as the separator. Otherwise, a space will be used.

Checking for Separator:
If the first argument is SEPARATOR, the second argument is taken as the separator, then the arguments are shifted twice and execution jumps back to :Concatenate.

Initial Setup:
Some initialization is done, such as setting a prefix for variable names and determining if the current argument is an array.

Determining Boundaries:
If the current array has defined lower and upper boundaries (lbound and ubound respectively), they are fetched.
If no bounds are defined, they default to 0.

Filling Placeholder Array (For non-array arguments):
If the current argument is not an array, it gets stored in the placeholder array at the current index.

Setting Up Separator:
If the separator hasn't been explicitly set earlier, it defaults to a space.

Concatenation Loop:
It loops through the values of the current input array, appending each value to a buffer string separated by the specified separator.
This is done until all elements of the current array have been processed.

Processing Multiple Arrays:
If there are more arguments (arrays) to process, the function recursively processes the next array by shifting the arguments and jumping back to the beginning of :Concatenate.

Setting Output Value:
After all arrays have been processed, the concatenated string is set to the OutputValue variable.

Cleanup:
The function then calls another subroutine :ClearVariablesByPrefix to clean up any variables it used during its execution.

End of Function:
The function then terminates with a GoTo :EOF.

Note:

The script uses the "delayed variable expansion" feature (!variable! syntax) to evaluate variables within loops and conditions. This allows for variables to be dynamically updated and read within a loop.
The use of %~1 or %~2 etc. is a way of accessing the value of arguments passed to the subfunction or script. The ~ modifier is used to remove any enclosing quotes from the argument.
The function is designed to be flexible and can handle both arrays and individual values. If it detects an array, it will process its elements; if it detects a single value, it will just concatenate that value.
The script also makes use of the for /f loop to handle and process strings, which is a common method in batch scripting to manage strings with spaces or special characters.

Please explain in more detail what arguments can be used with this function

Also, why does it use setlocal and endlocal twice ?

What is clearvariablesbyprefix

How does the user of this function use the separator ?

Could this function use a newline character as the separator ?

ChatGPT

Sure, let's dive deeper into the :Concatenate function.
Arguments:

SEPARATOR (optional):
The first argument can optionally be the string SEPARATOR.
If this argument is provided, the next argument (%~2) is taken as the separator value. The script then shifts twice to remove these two arguments from the list, and continues processing.

InputArray1, InputArray2, ... , InputArrayN:
These are the arrays or single values to concatenate.
If you provide an array name, the function expects that array to have associated .lbound and .ubound (lower and upper bounds) properties. If these aren't defined, it defaults to 0.
If you provide a single value instead of an array, the function stores that value in a temporary placeholder array.

OutputValue:
This is the variable where the concatenated result will be stored. It's provided by the caller of the function.

Use of setlocal and endlocal:

The use of setlocal and endlocal is a common technique in batch scripting:

setlocal creates a localized environment where variable changes don't affect the global environment. This is useful to prevent unwanted side effects on other parts of the script.

endlocal ends that localized environment and returns to the previous environment. Any variables defined or changed within the local environment are discarded, except the ones that are explicitly passed out.

The function uses this pair twice:

The first setlocal enabledelayedexpansion enables delayed variable expansion, which allows for dynamic evaluation of variables inside loops. The corresponding endlocal is used after the :Concatenate-loop to pass the concatenated buffer (_Concatenate_buffer) to the global environment.

The second setlocal enabledelayedexpansion is used just before setting the OutputValue. Its corresponding endlocal ensures that only the desired OutputValue is returned to the caller, without any of the temporary variables.

ClearVariablesByPrefix:

This function is presumably another subroutine in the script which isn't shown in the code you provided. Its likely purpose is to clear any variables that have a specific prefix. This is probably for cleaning up any temporary variables created by the :Concatenate function, ensuring they don't pollute the global namespace.
Using the Separator:

The user can provide a separator in two ways:

If the SEPARATOR argument is used, followed by the desired separator string. For example: SEPARATOR "-" would use a hyphen as the separator.
If no separator is specified, the function defaults to using a space.

Newline as Separator:

To use a newline character as a separator in a batch script is tricky because command-line arguments generally can't contain newline characters. However, a workaround is possible:

One could modify the script to recognize a placeholder (e.g., NEWLINE) as the separator and then replace it with an actual newline character inside the function.

For instance:

batch

if "!_Concatenate_separator!"=="NEWLINE" set "_Concatenate_separator=^

"

Note the intentional line break between ^ and the second ". This creates a newline.

Then, when calling the function, you could use SEPARATOR NEWLINE to indicate a newline separator.

With these adjustments, the :Concatenate function should be able to use a newline character as a separator.

[shodan]

I have created a new version of the :Concatenate function.

I have moved the outputvariable to be the first parameter instead of the last, I find that generally easier to handle for "infinite" argument functions even though that is less intuitive than having the inputs first and the output second. I believe it makes the function better able to accept poisonous characters on the input.

I have not tested this function for poisonous characters or what the maximum input / output lenghts are.

I have removed the going in and out of setlocal enabledelayedexpansion on every per-argument looping.
Now it enters delayedexpansion and stays in there until the end.

So this is more compact and a little faster.

It all works EXCEPT for using newline character as a concatenation separator.
It also is not possible to not use any separator, if I touch this function again, I would like to add this possibility.
It will probably through using a keyword like SEPARATOR EMPTY.

Here is the function

Code: Select all

::Usage Call :Concatenate OutputValue optional (SEPARATOR "X") InputArray1 InputArray2 InputArrayN 
:Concatenate
if not defined _Concatenate_localscope ( setlocal enabledelayedexpansion & set "_Concatenate_localscope=true" )
if not defined _Concatenate_output ( set "_Concatenate_output=%~1" & shift & GoTo :Concatenate )
if "[%~1]" EQU "[SEPARATOR]" ( set "_Concatenate_separator=%~2" & shift & shift & GoTo :Concatenate )
set "_Concatenate_prefix=_CA"
if defined %1.ubound ( set "_Concatenate_input=%~1" ) else ( set "_Concatenate_input=_Concatenate_placeholder" & set "_Concatenate_placeholder[0]=" )
if defined %1.lbound call set /a _Concatenate_lbound=%%%~1.lbound%%
if defined %1.ubound call set /a _Concatenate_ubound=%%%~1.ubound%%
if not defined %1.lbound set /a "_Concatenate_lbound=0"
if not defined %1.ubound set /a "_Concatenate_ubound=0"
set /a "_Concatenate_index=%_Concatenate_lbound%"
if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] if defined %1 set %_Concatenate_input%[%_Concatenate_index%]=!%1!
if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] set "%_Concatenate_input%[%_Concatenate_index%]=%~1"
if defined !_Concatenate_separator! ( set _Concatenate_separator=!%_Concatenate_separator%! )
if not defined _Concatenate_separator set "_Concatenate_separator= "
:Concatenate-loop
set _Concatenate_buffer=!_Concatenate_buffer!!%_Concatenate_input%[%_Concatenate_index%]!!_Concatenate_separator!
set /a "_Concatenate_index+=1"
if %_Concatenate_index% LEQ %_Concatenate_ubound% GoTo :Concatenate-loop
if "[%~2]" NEQ "[]" ( shift & GoTo :Concatenate ) 
for /f "tokens=* delims==" %%a in ('echo.!_Concatenate_buffer!') do (
																endlocal
																set %_Concatenate_output%=%%a
																)
Call :ClearVariablesByPrefix %_Concatenate_prefix% _Concatenate
GoTo :EOF

Here is the mini-demo function

Code: Select all

:Concatenate-mini-DEMO

echo -------------------------------
Call :Concatenate _myConcatenated-Mini-1 "abc" "def" "ghi"

echo.&echo.&echo result for Call :Concatenate _myConcatenated-Mini-1 "abc" "def" "ghi"
echo Demonstrate concatenate of 3 strings by value, default separator of " " is used
echo.&echo %_myConcatenated-Mini-1%

echo -------------------------------

set myvar=jkl
set myothervar=mno
set myotherothervar=pqr

Call :Concatenate _myConcatenated-Mini-2 myvar myothervar myotherothervar 

echo.&echo.&echo result for Call :Concatenate _myConcatenated-Mini-2 myvar myothervar myotherothervar 
echo Demonstrate concatenate of 3 strings by reference, default separator of " " is used
echo.&echo %_myConcatenated-Mini-2%

echo -------------------------------

set myarray[0]=stu
set myarray[1]=vwx
set myarray[2]=yzA
set /a "myarray.ubound=2"

Call :Concatenate _myConcatenated-Mini-3 myarray

echo.&echo.&echo result for Call :Concatenate _myConcatenated-Mini-3 myarray 
echo Demonstrate concatenate of 3 strings in an array
echo.&echo %_myConcatenated-Mini-3%

echo -------------------------------

Call :Concatenate  _myConcatenated-Mini-4 "abc" "def" "ghi" myvar myothervar myotherothervar myarray

echo.&echo.&echo result for Call :Concatenate _myConcatenated-Mini-4 "abc" "def" "ghi" myvar myothervar myotherothervar myarray
echo Demonstrate concatenate with all 3 previous methods combined
echo.&echo %_myConcatenated-Mini-4%

echo -------------------------------

Call :Concatenate _myConcatenated-Mini-5 SEPARATOR 1 "abc" "def" "ghi" myvar myothervar myotherothervar myarray 

echo.&echo.&echo result for Call :Concatenate _myConcatenated-Mini-5 SEPARATOR 1 "abc" "def" "ghi" myvar myothervar myotherothervar myarray 
echo Demonstrate same as previous, but a separator is specified to "1"
echo.&echo %_myConcatenated-Mini-5%

echo -------------------------------
Call :Concatenate _myConcatenated-Mini-6 "abc" "def" "ghi" myvar SEPARATOR 1 myothervar myotherothervar myarray

echo.&echo.&echo result for Call :Concatenate _myConcatenated-Mini-6 "abc" "def" "ghi" myvar SEPARATOR 1 myothervar myotherothervar myarray 
echo Demonstrate same as previous, but a separator is specified to "1" starting from myothervar/vwx only
echo.&echo %_myConcatenated-Mini-6%

echo -------------------------------
Call :Concatenate _myConcatenated-Mini-7 "abc" SEPARATOR 1 "def" "ghi" myvar SEPARATOR 2 myothervar myotherothervar SEPARATOR " " myarray 

echo.&echo.&echo result for Call :Concatenate _myConcatenated-Mini-7 "abc" SEPARATOR 1 "def" "ghi" myvar SEPARATOR 2 myothervar myotherothervar SEPARATOR " " myarray 
echo Demonstrate same as previous, but multiple separators are used
echo.&echo %_myConcatenated-Mini-7%

echo -------------------------------


GoTo :EOF

Here is the output of the mini demo function

Code: Select all

Concatenate-DEMO.bat
-------------------------------


result for Call :Concatenate _myConcatenated-Mini-1 "abc" "def" "ghi"
Demonstrate concatenate of 3 strings by value, default separator of " " is used

abc def ghi
-------------------------------


result for Call :Concatenate _myConcatenated-Mini-2 myvar myothervar myotherothervar
Demonstrate concatenate of 3 strings by reference, default separator of " " is used

jkl mno pqr
-------------------------------


result for Call :Concatenate _myConcatenated-Mini-3 myarray
Demonstrate concatenate of 3 strings in an array

stu vwx yzA
-------------------------------


result for Call :Concatenate _myConcatenated-Mini-4 "abc" "def" "ghi" myvar myothervar myotherothervar myarray
Demonstrate concatenate with all 3 previous methods combined

abc def ghi jkl mno pqr stu vwx yzA
-------------------------------


result for Call :Concatenate _myConcatenated-Mini-5 SEPARATOR 1 "abc" "def" "ghi" myvar myothervar myotherothervar myarray
Demonstrate same as previous, but a separator is specified to "1"

abc1def1ghi1jkl1mno1pqr1stu1vwx1yzA1
-------------------------------


result for Call :Concatenate _myConcatenated-Mini-6 "abc" "def" "ghi" myvar SEPARATOR 1 myothervar myotherothervar myarray
Demonstrate same as previous, but a separator is specified to "1" starting from myothervar/vwx only

abc def ghi jkl mno1pqr1stu1vwx1yzA1
-------------------------------


result for Call :Concatenate _myConcatenated-Mini-7 "abc" SEPARATOR 1 "def" "ghi" myvar SEPARATOR 2 myothervar myotherothervar SEPARATOR " " myarray
Demonstrate same as previous, but multiple separators are used

abc def1ghi1jkl1mno2pqr2stu vwx yzA
-------------------------------

Here is the complete DEMO file

Concatenate-DEMO.zip

(2.36 KiB) Downloaded 490 times

[shodan]

Here is more explanation of how the function works by chatgpt

https://chat.openai.com/share/6209be7d- ... e1fbc1352c

Code: Select all

Certainly! Here's a breakdown of the syntax for the :Concatenate function, followed by examples:

Syntax:

Call :Concatenate OutputVariable [optional: SEPARATOR SeparatorValue] Input1 [Input2] [Input3] ... [InputN]

    OutputVariable: The variable that will hold the concatenated result.

    SEPARATOR SeparatorValue: This is optional. If specified, the SeparatorValue will be used as the separator between subsequent inputs. If not specified, the function will use a space (" ") as the default separator. You can also specify different separators for different inputs.

    Input1, Input2, ... InputN: The inputs can be arrays (with or without defined .lbound and .ubound properties) or individual string values.

Examples:

    Simple Concatenation with Default Separator (space):


Call :Concatenate ResultVar Value1 Value2 Value3

Output in ResultVar: Value1 Value2 Value3

Concatenation with a Custom Separator:


Call :Concatenate ResultVar SEPARATOR - Value1 Value2 Value3

Output in ResultVar: Value1-Value2-Value3

Concatenating Arrays with Default Separator:
Assuming Array1 contains elements "A", "B", and "C":

Call :Concatenate ResultVar Array1

Output in ResultVar: A B C

Concatenating Arrays with Custom Separators:
Assuming Array1 contains elements "A", "B", and "C", and Array2 contains "1", "2", and "3":


Call :Concatenate ResultVar SEPARATOR - Array1 SEPARATOR . Array2

Output in ResultVar: A-B-C.1.2.3

Mix of Strings and Arrays:
Assuming Array1 contains elements "A", "B", and "C":


    Call :Concatenate ResultVar Value1 SEPARATOR - Array1 Value2

    Output in ResultVar: Value1-A-B-C-Value2

This function is robust, allowing for a mix of individual values and arrays with variable or custom separators, offering a high degree of flexibility in constructing complex strings.

Here is a line-by-line explanation of how the function works by chatgpt

Alright, let's break down the :Concatenate function line by line:

Code: Select all

    ::Usage Call :Concatenate OutputValue optional (SEPARATOR "X") InputArray1 InputArray2 InputArrayN
        This is a comment that provides the usage instructions for the function. It's not executed; it's there for documentation purposes.

    :Concatenate
        This labels the start of the Concatenate function.

    if not defined _Concatenate_localscope ( setlocal enabledelayedexpansion & set "_Concatenate_localscope=true" )
        Checks if the variable _Concatenate_localscope is defined. If it's not defined, this means it's the function's first run, so it sets the local scope (localizes variable changes to the function) and enables delayed expansion (allows the use of !variable! for real-time value evaluation).

    if not defined _Concatenate_output ( set "_Concatenate_output=%~1" & shift & GoTo :Concatenate )
        Checks if _Concatenate_output is defined. If not, it sets _Concatenate_output to the first argument passed to the function, shifts the arguments (so %2 becomes %1, %3 becomes %2, etc.), and then jumps back to the start of :Concatenate.

    if "[%~1]" EQU "[SEPARATOR]" ( set "_Concatenate_separator=%~2" & shift & shift & GoTo :Concatenate )
        Checks if the first argument is "SEPARATOR". If true, sets _Concatenate_separator to the value of the second argument, shifts arguments twice, and jumps back to the start.

    set "_Concatenate_prefix=_CA"
        Sets a prefix for variables; this helps later in cleanup.

    if defined %1.ubound ( set "_Concatenate_input=%~1" ) else ( set "_Concatenate_input=_Concatenate_placeholder" & set "_Concatenate_placeholder[0]=" )
        Checks if the upper bound of the current input array is defined. If true, the current input is treated as an array; otherwise, as a plain string.

    if defined %1.lbound call set /a _Concatenate_lbound=%%%~1.lbound%%
        If the lower bound of the array is defined, sets _Concatenate_lbound to that value.

    if defined %1.ubound call set /a _Concatenate_ubound=%%%~1.ubound%%
        Similarly, if the upper bound of the array is defined, sets _Concatenate_ubound to that value.

    if not defined %1.lbound set /a "_Concatenate_lbound=0"
        If the lower bound is not defined, it defaults to 0.

    if not defined %1.ubound set /a "_Concatenate_ubound=0"
        If the upper bound isn't defined, it defaults to 0.

    set /a "_Concatenate_index=%_Concatenate_lbound%"
        Initializes the _Concatenate_index variable with the lower bound.

    if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] if defined %1 set %_Concatenate_input%[%_Concatenate_index%]=!%1!
        If no bounds are defined and the current index of the input is not set, but an argument is provided, then set the current index of the input to that argument.

    if not defined %1.lbound if not defined %1.ubound if not defined %_Concatenate_input%[%_Concatenate_index%] set "%_Concatenate_input%[%_Concatenate_index%]=%~1"
        Similar to the previous line, but treats the input as a plain string if the bounds aren't defined.

    if defined !_Concatenate_separator! ( set _Concatenate_separator=!%_Concatenate_separator%! )
        If a separator is defined and it is a reference to another variable, sets _Concatenate_separator to the value of that referenced variable.

    if not defined _Concatenate_separator set "_Concatenate_separator= "
        If no separator has been specified, defaults to a space.

    :Concatenate-loop
        Marks the beginning of the loop that will iterate through the input elements.

    set _Concatenate_buffer=!_Concatenate_buffer!!%_Concatenate_input%[%_Concatenate_index%]!!_Concatenate_separator!
        Concatenates the current element of the input with the previous values in the buffer, using the specified separator.

    set /a "_Concatenate_index+=1"
        Increments the index for the next iteration.

    if %_Concatenate_index% LEQ %_Concatenate_ubound% GoTo :Concatenate-loop
        If the current index is less than or equal to the upper bound, the loop continues.

    if "[%~2]" NEQ "[]" ( shift & GoTo :Concatenate )
        If there's another argument after the current input, the arguments are shifted and the function goes back to the start to process the next input.

    for /f "tokens=* delims==" %%a in ('echo.!_Concatenate_buffer!') do (
        Iterates over the concatenated buffer.

    endlocal
        Ends the local environment, restoring original state.

    set %_Concatenate_output%=%%a
        Sets the output variable to the concatenated buffer.

    )
        Ends the for loop.

    Call :ClearVariablesByPrefix %_Concatenate_prefix% _Concatenate
        Calls another function to clear temporary variables. (The actual function isn't provided, but it presumably cleans up variables by their prefixes to prevent pollution.)

    GoTo :EOF
        Exits the function.

In summary, this function takes in various arrays or plain strings, concatenates them with an optional separator, and provides the concatenated result in the specified output variable. It can handle multiple arrays or strings in a single call and uses local environment and delayed expansion to manage its operation.

[OJBakker]

I have attached 3 adapted versions of Concatenate-DEMO-For batch file.

Your version uses a for /f loop to save over endlocal.

The -For version is the shortest. It uses a for loop and for-param to save over endlocal.

The -NoFor version uses no for but escapes the linefeeds before saving the param over endlocal.
This is based on code from aschipfl on stackoverflow:
link https://stackoverflow.com/questions/619 ... -variables

The -Safe version is the most complex but seems to be the safest way to save a param over endlocal.
This version uses a for loop and a for/f loop and 4 for parameters to get the parameter safe over endlocal.
This is based on code from Jeb on dostips:
link: viewtopic.php?p=6930#p6930

The 3 versions only differ in the concatenate part.

I have made a few changes to ClearVariablesByPrefix.
I have added more info after the last demo, the demo with the line feed.
I have also added modifying the output for echo multi-linefeed variable without delayed expansion.

The 3 files in concat.zip:
Concatenate-DEMO-For.cmd
Concatenate-DEMO-NoFor.cmd
Concatenate-DEMO-Safe.cmd

concat.zip

(6.97 KiB) Downloaded 501 times

[shodan]

Thank you @OJBakker
for giving this a shot !


-------------------------

This formulation of the escaping of variable through endlocal is more elegant than what I had been using

Code: Select all

for %%A in ("!var!") do (
   endlocal
   set "%_Concatenate_output%=%%~A"
)

-------------------------

I had never seen this other form
I suppose it is similar to endlocal & command1 & command2, but they happen in sequence, so that command1's changes are present for command2, but then how are the variables of setlocal still existing after endlocal ?

Code: Select all

(
   endlocal
   set "_Concatenate_output=%_Concatenate_output%"
   set %_Concatenate_output%=%var%
)

-------------------------

I also really like this method to detect if delayedexpansion is enabled

Code: Select all

set "NotDelayedFlag=!"

-------------------------

Could you explain the meaning of the following, is it equivalent to empty spaces ?

Code: Select all

%==%

-------------------------

I like your changes to

Code: Select all

:: Usage Call :ClearVariablesByPrefix myPrefix
:ClearVariablesByPrefix
if "[%~1]" NEQ "[]" for /f "tokens=1 delims==" %%a in ('set "%~1" 2^>nul') do set "%%a="
if "[%~2]" NEQ "[]" shift & GoTo :ClearVariablesByPrefix
GoTo :EOF

The ,2 was superfluous and not this version can handle variables with spaces and a few more poison characters in them !

-------------------------

In the following passage

(
endlocal
set "_Concatenate_output=%_Concatenate_output%"
set %_Concatenate_output%=%var%
)

I don' t understand why _Concatenate_output needs to be set with its own value ?
If it has lost its own value, then how does %var% contain a value ?


-------------------------


Did you use the %var% variable only for conciseness,
Or is there a specific reason why the buffer variable and output variables cannot be used AS IS ?

Code: Select all

set var=!TheOutput!
set "var=!_Concatenate_buffer!"

-------------------------

I will need more time think about how this works

Here is the colored diffs of these 3 new versions compared with my original

Compared to Concatenate-DEMO-NoFor.cmd

2023-10-02 04_18_47-_new 1 - Notepad++.png (33.15 KiB) Viewed 17229 times

Compared to Concatenate-DEMO-Safe.cmd

2023-10-02 04_19_16-_new 2 - Notepad++.png (45.25 KiB) Viewed 17229 times

Compared to Concatenate-DEMO-For.cmd

2023-10-02 04_19_47-_new 3 - Notepad++.png (28.19 KiB) Viewed 17229 times

Thanks, I will think about it and try to understand what the differences mean for usability, side-effects and performance