DosTips.com

A Forum all about DOS Batch
https://www.dostips.com/forum/

Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

https://www.dostips.com/forum/viewtopic.php?f=3&t=11244

Page 1 of 1

Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 22 Apr 2024 03:54
by shodan
Hello

I made this function to copy arbitrary lines of text, from one file to another

Code: Select all

::Usage Call :AppendFileLineToFile inputfile outputfile 3 4 50-75 5 6 7 ... N
:AppendFileLineToFile
set "_AppendFileLineToFile_prefix=_AFLTF"
set "_AFLTF_InputFile=%~1"
set "_AFLTF_OutputFile=%~2"
:AppendFileLineToFile-arg
for /f "delims=- tokens=1,2" %%a in ("%~3") do ( set "_AFLTF_Start=%%a" & set "_AFLTF_Stop=%%b"  )
if not defined _AFLTF_Stop set /a _AFLTF_Stop=%_AFLTF_Start%
Setlocal enabledelayedexpansion
if %_AFLTF_Start% GTR 1 set /a "_AFLTF_skip=%_AFLTF_Start%-1"
if %_AFLTF_Start% GTR 1 ( set "_AFLTF_skip=skip^=%_AFLTF_skip%^" ) else ( set "_AFLTF_skip=" )
for /f %_AFLTF_skip% delims^=^ eol^= %%a in (' ^( type "%_AFLTF_InputFile%" ^| %SystemRoot%\System32\findstr /N /R /C:".*" ^) 2^>nul ') do ( 
	for /f "delims=:" %%f in ("%%a") do if %%f GTR %_AFLTF_Stop% GoTo :AppendFileLineToFile-end
	set _AFLTF_buffer=%%a
	if defined _AFLTF_buffer >>"%_AFLTF_OutputFile%" echo(!_AFLTF_buffer:*:=!
	) 
endlocal
:AppendFileLineToFile-end
if "[%~4]" NEQ "[]" ( shift & GoTo :AppendFileLineToFile-arg )
Call :ClearVariablesByPrefix %_AppendFileLineToFile_prefix% _AppendFileLineToFile_prefix & GoTo :EOF
The logic of copying the correct rows to the right place LARGELY works
Unfortunately, it does damage some lines of text and it completely breaks if it encounters ">>" possibly other characters


For example this line

Code: Select all

for /f delims^=:^ tokens^=1 %%a in ('%SystemRoot%\System32\findstr /I /N "^:%~2" "%~1" ^| findstr /I /V "::%~2[a-zA-Z0-9\-\/\?\!\@\%%\$\#\^\*\)\{\}\[\]\:\_]"') do ( if "[%~3]" NEQ "[]" set "%~3=%%a" & exit /b %%a )
becomes this line

Code: Select all

for /f delims=: tokens=1 %%a in ('%SystemRoot%\System32\findstr /I /N ":%~2" "%~1" | findstr /I /V "::%~2[a-zA-Z0-9\-\/\?\\_]"') do ( if "[%~3]" NEQ "[]" set "%~3=%%a" & exit /b %%a )
And this line

Code: Select all

	if defined _AFLTF_buffer >>"%_AFLTF_OutputFile%" echo(!_AFLTF_buffer:*:=!
breaks it in a very strange way that I do not understand.

Here is a screenshot
The original file is on the right, the newly created on the left
notepad++_dx5YxoUAgP.png
notepad++_dx5YxoUAgP.png (615.34 KiB) Viewed 13765 times

This is part of a larger project, which allows copying selected batch functions to another batch file.

I have created a special "function switching" batch file, when you call it, the first argument is the function that will be executed with the rest of your arguments.

Here I demonstrate the "listfunctions" function, which outputs a list of all function existing in a batchfile

Code: Select all

bfw listfunctions bfw.bat

ShiftedArgumentCaller AddFunctionToBatch GetLastToken IsLastToken GetLabelsOnly IsFile GetFunctionRows GetLabelRow GetFunctionExit GetFunctionPreambleRow GetFunctionPostscriptRow ClearVariablesByPrefix GetFunctionName GetBatchCore GetNextExitRow ListFunctions GetNextFunctionName GetNextFunctionRow GetPreviousExitRow GetEOFrow countLines IsFunctionLabelExcluded AppendFileLineToFile AppendFileLineToFile-arg GetPreviousEmptyRow GetNextEmptyRow BFWFunctionSwitcher-text EndOF_BFWFunctionSwitcher-text
And now I demontrate the "AddFunctionToBatch" function, which copies chosen functions from a batchfile to another batchfile. It ALMOST works

Code: Select all

bfw AddFunctionToBatch test\mybfw.bat CORE ShiftedArgumentCaller AddFunctionToBatch GetLastToken IsLastToken GetLabelsOnly IsFile GetFunctionRows GetLabelRow GetFunctionExit GetFunctionPreambleRow GetFunctionPostscriptRow ClearVariablesByPrefix GetFunctionName GetBatchCore GetNextExitRow ListFunctions GetNextFunctionName GetPreviousExitRow GetEOFrow countLines IsFunctionLabelExcluded AppendFileLineToFile AppendFileLineToFile-arg GetPreviousEmptyRow GetNextEmptyRow BFWFunctionSwitcher-text
This previous command, creates the test\mybfw.bat file, which contains (almost) everything it needs to perform the task of the original bfw.bat file.

Unfortunately the functions :GetLabelRow and :AppendFileLineToFile are mangled by this process

The original bfw.bat

https://pastebin.com/456kfhTi

The new test\mybfw.bat file

https://pastebin.com/vkkMGin9

You can also observe the bfw.bat file at

https://github.com/batchfileframework/B ... fw/bfw.bat

I also include it as a zip here
bfw.zip
(8.57 KiB) Downloaded 728 times

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 22 Apr 2024 04:54
by shodan
Looking in my older code, I found a function called "SimpleFileToArray" that takes lines of text from a file and loads them in an array of variable

I just tested it and I think it might work for this.

Here is the function itself.

Code: Select all

::Usage Call :SimpleFileToArray OutputArray Filename
:SimpleFileToArray
set /a "%~1.lbound=%%f"
for /f delims^=^ eol^= %%a in ('%SystemRoot%\System32\findstr.exe /N "^" "%~2"') do ( 
	for /f "tokens=1,2* delims=:" %%f in ("%%a") do set /a "%~1.ubound=%%f" & set %~1[%%f]=%%a
	)
set /a "_SFTA_index=1"
call set /a "_SFTA_ubound=%%%~1.ubound%%"
:SimpleFileToArray-loop
setlocal enabledelayedexpansion
set _SFTA_localscope=true
set %~1[%_SFTA_index%]=!%~1[%_SFTA_index%]:*:=!
for /f "delims=" %%a in ('set %~1[%_SFTA_index%] 2^>nul') do (
		endlocal
		set %%a
	)
if defined _SFTA_localscope endlocal & set %~1[%_SFTA_index%]=
set /a "_SFTA_index+=1"
if %_SFTA_index% LEQ %_SFTA_ubound% GoTo :SimpleFileToArray-loop
GoTo :EOF
And here is what happens if I read the previous file

The problematic line 150 of :GetLabelRow, was as follows

Code: Select all

for /f delims^=:^ tokens^=1 %%a in ('%SystemRoot%\System32\findstr /I /N "^:%~2" "%~1" ^| findstr /I /V "::%~2[a-zA-Z0-9\-\/\?\!\@\%%\$\#\^\*\)\{\}\[\]\:\_]"') do ( if "[%~3]" NEQ "[]" set "%~3=%%a" & exit /b %%a )
It became

Code: Select all

bfw[150]=for /f delims^=:^ tokens^=1 %%a in ('%SystemRoot%\System32\findstr /I /N "^:%~2" "%~1" ^| findstr /I /V "::%~2[a-zA-Z0-9\-\/\?\!\@\%%\$\#\^\*\)\{\}\[\]\:\_]"') do ( if "[%~3]" NEQ "[]" set "%~3=%%a" & exit /b %%a )
Great, it's a match !


And looking at the larger function


It was :AppendFileLineToFile

Code: Select all

::Usage Call :AppendFileLineToFile inputfile outputfile 3 4 50-75 5 6 7 ... N
:AppendFileLineToFile
set "_AppendFileLineToFile_prefix=_AFLTF"
set "_AFLTF_InputFile=%~1"
set "_AFLTF_OutputFile=%~2"
:AppendFileLineToFile-arg
for /f "delims=- tokens=1,2" %%a in ("%~3") do ( set "_AFLTF_Start=%%a" & set "_AFLTF_Stop=%%b"  )
if not defined _AFLTF_Stop set /a _AFLTF_Stop=%_AFLTF_Start%
Setlocal enabledelayedexpansion
if %_AFLTF_Start% GTR 1 set /a "_AFLTF_skip=%_AFLTF_Start%-1"
if %_AFLTF_Start% GTR 1 ( set "_AFLTF_skip=skip^=%_AFLTF_skip%^" ) else ( set "_AFLTF_skip=" )
for /f %_AFLTF_skip% delims^=^ eol^= %%a in (' ^( type "%_AFLTF_InputFile%" ^| %SystemRoot%\System32\findstr /N /R /C:".*" ^) 2^>nul ') do ( 
	for /f "delims=:" %%f in ("%%a") do if %%f GTR %_AFLTF_Stop% GoTo :AppendFileLineToFile-end
	set _AFLTF_buffer=%%a
	if defined _AFLTF_buffer >>"%_AFLTF_OutputFile%" echo(!_AFLTF_buffer:*:=!
	) 
endlocal
:AppendFileLineToFile-end
if "[%~4]" NEQ "[]" ( shift & GoTo :AppendFileLineToFile-arg )
Call :ClearVariablesByPrefix %_AppendFileLineToFile_prefix% _AppendFileLineToFile_prefix & GoTo :EOF
And in the array it is

Code: Select all

bfw[283]=::Usage Call :AppendFileLineToFile inputfile outputfile 3 4 50-75 5 6 7 ... N
bfw[284]=:AppendFileLineToFile
bfw[285]=set "_AppendFileLineToFile_prefix=_AFLTF"
bfw[286]=set "_AFLTF_InputFile=%~1"
bfw[287]=set "_AFLTF_OutputFile=%~2"
bfw[288]=:AppendFileLineToFile-arg
bfw[289]=for /f "delims=- tokens=1,2" %%a in ("%~3") do ( set "_AFLTF_Start=%%a" & set "_AFLTF_Stop=%%b"  )
bfw[290]=if not defined _AFLTF_Stop set /a _AFLTF_Stop=%_AFLTF_Start%
bfw[291]=Setlocal enabledelayedexpansion
bfw[292]=if %_AFLTF_Start% GTR 1 set /a "_AFLTF_skip=%_AFLTF_Start%-1"
bfw[293]=if %_AFLTF_Start% GTR 1 ( set "_AFLTF_skip=skip^=%_AFLTF_skip%^" ) else ( set "_AFLTF_skip=" )
bfw[294]=for /f %_AFLTF_skip% delims^=^ eol^= %%a in (' ^( type "%_AFLTF_InputFile%" ^| %SystemRoot%\System32\findstr /N /R /C:".*" ^) 2^>nul ') do (
bfw[295]=       for /f "delims=:" %%f in ("%%a") do if %%f GTR %_AFLTF_Stop% GoTo :AppendFileLineToFile-end
bfw[296]=       set _AFLTF_buffer=%%a
bfw[297]=       if defined _AFLTF_buffer >>"%_AFLTF_OutputFile%" echo(!_AFLTF_buffer:*:=!
bfw[298]=       )
bfw[299]=endlocal
bfw[300]=:AppendFileLineToFile-end
bfw[301]=if "[%~4]" NEQ "[]" ( shift & GoTo :AppendFileLineToFile-arg )
bfw[302]=Call :ClearVariablesByPrefix %_AppendFileLineToFile_prefix% _AppendFileLineToFile_prefix & GoTo :EOF
Well, that also looks the same !



I guess now what I need is "ArrayToFile"


I did find something I wrote earlier, but it looks .. too simple

Code: Select all

:ArrayToFile byref InputArray OutputFile
Call :GetArrayParameters %~1 _ArrayToFile_input "" 0
set _ArrayToFile_input 
:ArrayToFile-loop
call echo %%%_ArrayToFile_input%[%_ArrayToFile_input.index%]%%>>%~2
set /a _ArrayToFile_input.index+=1
if %_ArrayToFile_input.index% leq %_ArrayToFile_input.ubound% GoTo :ArrayToFile-loop
GoTo :EOF

::Usage Call :GetArrayParameters InputArray ParameterVariable optional Initialize optional StartIndex=0
:GetArrayParameters
call set _GetArrayParameters.lbound=%%%~1.lbound%%
call set _GetArrayParameters.ubound=%%%~1.ubound%%
call set _GetArrayParameters.count=%%%~1.count%%
call set _GetArrayParameters.index=%%%~1.index%%
if "[%~3]"=="[Initialize]" (
	if "[%_GetArrayParameters.lbound%]"=="[]" set /a _GetArrayParameters.lbound=0
	if "[%_GetArrayParameters.ubound%]"=="[]" set /a _GetArrayParameters.ubound=-1
	if "[%_GetArrayParameters.count%]"=="[]" set /a _GetArrayParameters.count=%_GetArrayParameters.ubound%-%_GetArrayParameters.lbound%+1  2>nul
	if "[%_GetArrayParameters.index%]"=="[]" if "[%~4]"=="[]" set /a "_GetArrayParameters.index=0"
) else (
	if "[%_GetArrayParameters.lbound%]"=="[]" set "_GetArrayParameters_bounds_incomplete=true" & set "_GetArrayParameters_lbound_empty=true"
	if "[%_GetArrayParameters.ubound%]"=="[]" set "_GetArrayParameters_bounds_incomplete=true" & set "_GetArrayParameters_ubound_empty=true"
	if "[%_GetArrayParameters_bounds_incomplete%]"=="[true]" Call :FindArrayBounds %%%~1%% _GetArrayParameters_bounds_incomplete
	if "[%_GetArrayParameters_lbound_empty%]"=="[true]" set "_GetArrayParameters.lbound=%_GetArrayParameters_bounds_incomplete.lbound%"
	if "[%_GetArrayParameters_ubound_empty%]"=="[true]" set "_GetArrayParameters.ubound=%_GetArrayParameters_bounds_incomplete.ubound%"
	if "[%_GetArrayParameters.count%]"=="[]" set /a _GetArrayParameters.count=%_GetArrayParameters.ubound%-%_GetArrayParameters.lbound%+1  2>nul
)
if "[%~4]" NEQ "[]" ( set /a "_GetArrayParameters.index=%~4" 2>nul )
set _GetArrayParameters.name=%~1
set "%~2.lbound=%_GetArrayParameters.lbound%"
set "%~2.ubound=%_GetArrayParameters.ubound%"
set "%~2.count=%_GetArrayParameters.count%"
set "%~2.index=%_GetArrayParameters.index%"
if "[%_GetArrayParameters.name%]" NEQ "" set "%~2=%_GetArrayParameters.name%"
Call :ClearVariablesByPrefix _GetArrayParameters & if "[%_GetArrayParameters.ubound%]"=="[]" ( exit /b 1 ) else ( exit /b 0 )
GoTo :EOF
::GetArrayParameters-END

I gave it a shot anyway
cmd_gBmUBTRATs.png
cmd_gBmUBTRATs.png (20.65 KiB) Viewed 13752 times
Unfortunately, :GetLabelRow didn't make it

Code: Select all

::Usage Call :GetLabelRow BatchFile FunctionName optional OutputRow
:GetLabelRow
exit /b 0

Looking at :AppendFileLineToFile, it is also mangled, but differently

Code: Select all

ECHO is off.
::Usage Call :AppendFileLineToFile inputfile outputfile 3 4 50-75 5 6 7 ... N
:AppendFileLineToFile
set "_AppendFileLineToFile_prefix=_AFLTF"
set "_AFLTF_InputFile=%~1"
set "_AFLTF_OutputFile=%~2"
:AppendFileLineToFile-arg
if not defined _AFLTF_Stop set /a _AFLTF_Stop=%_AFLTF_Start%
Setlocal enabledelayedexpansion
if %_AFLTF_Start% GTR 1 set /a "_AFLTF_skip=%_AFLTF_Start%-1"
if %_AFLTF_Start% GTR 1 ( set "_AFLTF_skip=skip^=%_AFLTF_skip%^" ) else ( set "_AFLTF_skip=" )
for /f %_AFLTF_skip% delims= eol= %%a in (' ( type "%_AFLTF_InputFile%" | %SystemRoot%\System32\findstr /N /R /C:".*" ) 2>nul ') do ( 
	for /f "delims=:" %%f in ("%%a") do if %%f GTR %_AFLTF_Stop% GoTo :AppendFileLineToFile-end
	set _AFLTF_buffer=%%a
	if defined _AFLTF_buffer  echo(!_AFLTF_buffer:*:=!
	) 
endlocal
:AppendFileLineToFile-end
ECHO is off.
Everywhere there was an empty line, now says ECHO is off.

Also the 343 line original, is now 286 lines :\

Here is the whole thing

https://pastebin.com/JRia6CKe

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 22 Apr 2024 13:44
by Sponge Belly
Hi Shodan,

I’m not sure what you’re trying to accomplish, but I suggest you read Calling Functions from a Library, especially Post #4 by Penpen.

HTH! :)

- SB

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 22 Apr 2024 15:06
by shodan
Thank you that is some top tier wizardry

In my current design, I planned to mostly rely on %~0 for function switching.

I would achieve that with shortcuts / softlinks and/or hardlink

I am creating a function, that lists all functions in a batch file, then creates/updates/deletes shortcuts/softlink/hardlink to that batch file but each is the name of an internal function.

And I put in as a backup, if the filename is "bfw" to use the first argument as a function call, then shift the arguments. Which shifts the arguments once then GOTO instead of call the destination function.

So far it seems this approach only breaks %* and only when using the ShiftedArgumentCaller function

I will have to think how I can use this new technique to improve on my design !

However for today the question remains
How to expand a variable with poison characters and print it or write it to a file.
I think I used to have this solution, but I cannot find it anywhere ...

thanks !

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 23 Apr 2024 15:03
by shodan
I have cracked it

Posting before the forum times out again !

This will write the line number and line ranges from inputfile to output file

Code: Select all

::Usage Call :AppendFileLineToFile inputfile outputfile 3 4 50-75 5 6 7 ... N
:AppendFileLineToFile
set "_AppendFileLineToFile_prefix=_AFLTF"
set "_AFLTF_InputFile=%~1"
set "_AFLTF_OutputFile=%~2"
:AppendFileLineToFile-args
for /f "delims=- tokens=1,2" %%a in ("%~3") do ( set "_AFLTF_Start=%%a" & set "_AFLTF_Stop=%%b"  )
if not defined _AFLTF_Stop set /a _AFLTF_Stop=%_AFLTF_Start%
if %_AFLTF_Start% GTR 1 set /a "_AFLTF_skip=%_AFLTF_Start%-1"
if %_AFLTF_Start% GTR 1 ( set "_AFLTF_skip=skip^=%_AFLTF_skip%^" ) else ( set "_AFLTF_skip=" )
for /f %_AFLTF_skip% delims^=^ eol^= %%a in ('%SystemRoot%\System32\findstr /N "^" "%_AFLTF_InputFile%"') do ( 
	for /f "delims=:" %%f in ("%%a") do if %%f GTR %_AFLTF_Stop% GoTo :AppendFileLineToFile-end
	set _AFLTF_buffer=%%a
	Setlocal enabledelayedexpansion
	set _AFLTF_buffer=!_AFLTF_buffer:*:=!
	>> "%_AFLTF_OutputFile%" echo(!_AFLTF_buffer!
	endlocal
	) 
:AppendFileLineToFile-end
if "[%~4]" NEQ "[]" ( shift & GoTo :AppendFileLineToFile-arg )
Call :ClearVariablesByPrefix %_AppendFileLineToFile_prefix% _AppendFileLineToFile_prefix & GoTo :EOF

:: Usage Call :ClearVariablesByPrefix myPrefix
:ClearVariablesByPrefix
if "[%~1]" NEQ "[]" for /f "tokens=1 delims==" %%a in ('set "%~1" 2^>nul') do set %%a=
if "[%~2]" NEQ "[]" shift & GoTo :ClearVariablesByPrefix
GoTo :EOF

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 25 Apr 2024 03:08
by miskox
Very good. Tried and works. But there is a typo:

Code: Select all

:AppendFileLineToFile-args
Missing 'S' at the end:

Code: Select all

if "[%~4]" NEQ "[]" ( shift & GoTo :AppendFileLineToFile-arg )
Saso

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 29 Apr 2024 14:01
by Sponge Belly
Hi Shodan,

I refer you to Jeb’s answer to SO: How to Read a File.

HTH! :)

- SB

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 29 Apr 2024 17:26
by shodan
Thanks,

My current solution is certainly derived from this comment.
I found it in an old version of my functions but the use of echo( and the very similar structure, I'm sure that's where I got it from !


Looking at the differences

He explicitely uses SETLOCAL DisableDelayedExpansion
Which I don't because it's assume that it's not enabled prior.

But adding it would probably make my version more robust.
However, he uses setlocal twice but endlocal only one.
I imagine there needs to be another endlocal after the for loop to restore cmd status to original.

It will be hard to make a FileLinesToArray function with crossing two endlocals !


I'm not sure if I should pipe the output of TYPE into findstr or leave it as is (for some unicode compatibility ?)

I'm not sure what is going on with the usebackq and my version with the escaped for /f parameters.
I've got that from ... somewhere, but I don't know why I'm doing it like that !

Code: Select all

for /f %_AFLTF_skip% delims^=^ eol^= %%a in ('%SystemRoot%\System32\findstr /N "^" "%_AFLTF_InputFile%"')
FOR /F "usebackq delims=" %%a in (`"findstr /n ^^ t.txt"`) do (
Last thing

He uses doublequotes to set the buffer variable

Code: Select all

set "var=%%a"
I worried about double quotes in the variable messing that up ? Seem to work both ways however.

Code: Select all

set _AFLTF_buffer=%%a

Re: Need help copying a line of text from a file to another (without mangling it) :AppendFileLineToFile

Posted: 01 May 2024 05:03
by Sponge Belly
Hi Shodan,

There should indeed be a terminating endLocal somewhere after the end of the for /f loop.

Delayed expansion is explicitly disabled to prevent any ^ and ! characters in the line of input from being corrupted. But once the line has been read into the variable, delayed expansion can be safely turned on to allow the line number appended by findStr to be removed, and the line to be echoed on screen verbatim (even if it contains poison characters).

It’s common practice in Batch programming to add double quotes around the definition of a variable (eg, set "var=%%a"). This is to ensure poison characters in the variable’s value don’t cause errors. And it also prevents the accidental inclusion of unwanted spaces. The double quotes will not be included in the variable’s name or its value.

Lastly, useBackq tells the for /f loop to watch out for spaces in filenames. Without it, for /f will treat a filename containing spaces as a string.

HTH! :)

- SB
All times are UTC-06:00
Page 1 of 1
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/