Page 1 of 1
Performance issues variable VS file
Posted: 20 Feb 2012 12:02
by Ed Dyreen
'
Trying to figure out the best way to throw help on the console
Using a variable is easy, but it wastes memory
Code: Select all
set ^"$Usage.%$defines%=^
use: ( %%%$defines%%% #vr, #fullPathFile, #section, #key, #sErr,0 )%$n1c%
con: contents of $key%$n1c%
err: Unaffected, panic otherwise"
So I try some clever file enum trick
Code: Select all
:: ------------------------------------------------------------------------------------------
set "$defines=File.readINI_" &set "$details=ini file handler"
:: ------------------------------------------------------------------------------------------
:: last updated : 18/02/2012
:: support : naDelayed, delayedChars, related chaining
::
:H use: ( %$defines% #vr, #fullPathFile, #section, #key, #sErr,0 )
:H con: contents of $key
:H err: Unaffected, panic otherwise
::
Code: Select all
set "$Debug.Use=File.readINI_"
for /f "delims=[]" %%? in (
'type "!$DOS-KIT.fullPathFile!" ^|find /v /n "type" ^|find "$defines=%$Debug.Use%"'
) do set "$start=%%?"
for /f %%? in (
'type "!$DOS-KIT.fullPathFile!" ^|find /c /v ""'
) do set "$end=%%?"
set "$start"
set "$end"
< "!$DOS-KIT.fullPathFile!" ( set "$at=0"
for /l %%ç in (
1, 1, !$end!
) do set "$=" &set /p "$=" &if %%ç geq !$start! if defined $at if "!$:~0,3!" == ":H " (
set "$at=1" &for %%? in ( "!$:~3!" ) do call set "$=%%~?" &echo( !$!
) else if !$at! neq 0 set "$at="
)
But now it wastes 60secs to process a file of +-18.000 lines
Code: Select all
$start=17428
$end=18430
use: ( File.readINI_ #vr, #fullPathFile, #section, #key, #sErr,0 )
con: contents of $key
err: Unaffected, panic otherwise
endoftest Druk op een toets om door te gaan. . .
I want to reduce memory load
and have it fast
I was also thinking about a solution where I initially load the help variables and when the loading is done, write them out to a help file that can be smaller and searched more easily.
Re: Performance issues variable VS file
Posted: 21 Feb 2012 10:31
by Ed Dyreen
'
Great, I save 12kB of memory and have a 12kB usage file that can be browsed easily by both user and program.
Gen file from mem
Code: Select all
(
for /f "tokens=1 delims==" %%? in ( 'set $Usage.' ) do set "?=%%?" &if "!?:~0,7!" == "$Usage." (
echo.
echo.[!?:~7!]
echo.!%%?!
)
)> "!$DOS-KIT.fullPath!\DOS-KIT.TMP"
more < "!$DOS-KIT.fullPath!\DOS-KIT.TMP" > "!$DOS-KIT.fullPath!\DOS-KIT.NFO" &del /f /q "!$DOS-KIT.fullPath!\DOS-KIT.TMP"
enum and extract (any)
Code: Select all
set "$Debug.Use=User.Logon_"
for /f "delims=[]" %%? in (
'type "!$DOS-KIT.fullPath!\DOS-KIT.NFO" ^|find /n "[%$Debug.Use%]"'
) do set "$start=%%?"
for /f %%? in (
'type "!$DOS-KIT.fullPath!\DOS-KIT.NFO" ^|find /c /v ""'
) do set "$end=%%?"
< "!$DOS-KIT.fullPath!\DOS-KIT.NFO" ( set "$at=0"
for /l %%! in (
1, 1, !$end!
) do set "$=" &set /p "$=" &if %%! geq !$start! if defined $at if "!$!" == "[%$Debug.Use%]" (
set "$at=1"
) else if !$at! neq 0 if defined $ (echo.!$!) else set "$at="
)
release mem
Code: Select all
for /f "tokens=1 delims==" %%? in ('set $Usage.') do if defined %%? (
set "%%?="
)
Re: Performance issues variable VS file
Posted: 21 Feb 2012 14:54
by Ed Dyreen
'
I can decide where to start file enum using
I can decide where to end file enum using
Code: Select all
<file (for /l %%! in (1,1,!end!) do set $=&set/p$=)
But I have no clue how to use em both together
Code: Select all
<file (for /l %%! in (!start!,1,!end!) do set $=&set/p$=)
I investigate the
more command, it say
"P n shows next n lines". Can't seem to get it working
Is it possible
Re: Performance issues variable VS file
Posted: 21 Feb 2012 15:29
by Squashman
P n is an extended feature and as far as I know it is an interactive command. Not sure if you can use it in a script.
I actually found out today that more +n will stop at 65534 lines and will then output -- more -- to your output file. But I would assume you don't deal with large files like I do.
Re: Performance issues variable VS file
Posted: 21 Feb 2012 23:06
by dbenham
Squashman wrote:I actually found out today that more +n will stop at 65534 lines and will then output -- more -- to your output file. But I would assume you don't deal with large files like I do.
What version of Windows
I wrote a batch implementation of tail using FIND /C to count the lines and MORE to skip all but the last n lines. I tested on Vista and successfully printed the last 10 lines of a file with 131,000+ lines.
Dave Benham
Re: Performance issues variable VS file
Posted: 22 Feb 2012 02:15
by foxidrive
Squashman wrote:found out today that more +n will stop at 65534 lines and will then output -- more -- to your output file.
Confirmed on XP Pro SP3
Re: Performance issues variable VS file
Posted: 22 Feb 2012 02:16
by foxidrive
dbenham wrote:I wrote a batch implementation of tail using FIND /C to count the lines and MORE to skip all but the last n lines. I tested on Vista and successfully printed the last 10 lines of a file with 131,000+ lines.
try skipping the first 10 lines...
I confirmed it on Vista and Windows 7 too:
for /L %a in (1,1,70000) do @echo a>>abc.txt
and wait...
more +10 abc.txt >abc2.txt
wait and press enter a few times then control C
find /i "more"<abc2.txt
Re: Performance issues variable VS file
Posted: 22 Feb 2012 06:21
by Squashman
foxidrive wrote:Squashman wrote:found out today that more +n will stop at 65534 lines and will then output -- more -- to your output file.
Confirmed on XP Pro SP3
Bingo.
I was helping another user on another forum and someone suggested that they use:
to skip the first 5 lines of a file and output to a new file.
That does work for most files but when I tried it on on my client files which had over 100,000 records I noticed the batch file was just hanging. I then looked in Explorer to see if the file was getting any bigger. It wasn't so I killed the batch file and then looked at the bottom of the file and saw this.
Ed, my apologies if we are getting off topic again.
Re: Performance issues variable VS file
Posted: 22 Feb 2012 06:59
by dbenham
Thanks Squashman and foxidrive
I misinterpreted the bad news. I thought Squashman was implying MORE can't skip more than 65534 lines. Now I realize that redirected or piped MORE cannot print more than 65534 lines without pausing and asking for a key press.
The behavior has nothing to do with the +n option. It pauses even if the +n option is absent and you start printing from the first line.
This is a major bummer
Dave Benham
Re: Performance issues variable VS file
Posted: 22 Feb 2012 10:26
by Squashman
dbenham wrote:This is a major bummer
I agree. I was hoping for a better outcome as well.