Speed Writing a File

[shadeclan]

Is there a faster (batch) way to write to a file besides the echo command? I could create an object in VBScript that accesses the scripting.FileSystemObject but I was wondering if there was a pure DOS method? The object of this question is to rewrite a larger file while leaving some lines out and echo seems to take a long time doing it.

[jeb]

Hi shadeclan,

the speed depends extremly by the way you choose.

Slow variant:

Code: Select all

>myfile.txt echo Line1
>>myfile.txt echo Line2
>>myfile.txt echo.
>>myfile.txt echo Line4

Faster is

Code: Select all

(
  echo Line1
  echo Line2
  echo(
  echo Line4
) > myFile.txt

It's faster to combine the echo's into one block and redirect only once.

Using of echo. or somthing else to create an empty line is slow as it accesses the disk, better use echo(

jeb

[!k]

Code: Select all

@echo off
>myFile.txt more +10 %0
for /f %%z in ("myFile.txt") do if not "%%~zz"=="23" (echo:Wrong myFile.txt!&pause&exit)

:: Your code is here
:: Your code is here

goto :eof

:: 10th line of batch
Line1
Line2

Line4

[shadeclan]

... It's faster to combine the echo's into one block and redirect only once ...

This hadn't occurred to me - I will try it and see if it speeds things up.

[shadeclan]

Code: Select all

@echo off
>myFile.txt more +10 %0
for /f %%z in ("myFile.txt") do if not "%%~zz"=="23" (echo:Wrong myFile.txt!&pause&exit)

:: Your code is here
:: Your code is here

goto :eof

:: 10th line of batch
Line1
Line2

Line4

!k - sorry, but I don't understand this, nor do I see how this is going to help me rewrite a file faster.

[!k]

read more/?

[shadeclan]

read more/?

I did. It displays one screen of output at a time. I'm not sure how that helps me write a file faster. Sorry - I'm stupid, OK? I don't get it.

[!k]

+n : Start displaying the first file at line n

more +10 will start display that batch at line №10

[shadeclan]

+n : Start displaying the first file at line n

more +10 will start display that batch at line №10

Wow - that was like, amazing! You're explanation was a little verbose but a little playing around with it resolved the problem completely! Thank you so very much!

[shadeclan]

Here is my final solution, heavily commented for us know-nothings:

Code: Select all

:: Generate name of temporary data file.
   set vDatFile="%vAppDir%DTF_FTP_%random%.dat"
:: Generate name of temporary working file.
   set vTmpFile="%vAppDir%DTF_FTP_%random%.tmp"
:: Generate name / location of log file.
   set vLogFile="%vAppDir%DTF_FTP.log"

   set /a "vCntr=0"

:: Delete logfile entries older than 52 runs ago, which should be about a year.
:: First, find the starting positions of the individual log entries in the log file
:: and put all the header lines into a data file along with the line positions where
:: they can be found.  Skip 2 lines for the header generated by FIND.
   find /n "*************** DTF FTP Start" %vLogFile% | more /s +2 > %vDatFile% rem (Yay MORE!!!  Thanks to !K!)
:: (First you find out how to use the pipe, then it becomes indispensable!)

:: Count the number of log headers deposited in the vDatFile file and put 
:: it into the vCntr variable.
:: Set the vCntr variable = the difference between the current number of log 
:: entries and the number of log entries expected over a year.  
:: Many thanks to Brett Batie 
:: (http://brett.batie.com/scripting/count-number-of-lines-in-a-file-using-dos/)
:: for the counting algorithym. 
   findstr /l /n "[" %vDatFile% | find /c ":"> %vTmpFile%
   set /p vCntr=<%vTmpFile% &del /q %vTmpFile%
   set /a vCntr-=52

:: Check to see if vCntr is > 0.  If not, skip log modification.
   if %vCntr% LEQ 0 (goto SkipLog)

:: If there are over a year's worth of logs in the logfile, delete earlier
:: log entries.
:: Set up the vCntr variable as a SKIP parameter for a FOR loop using the
:: calculated extra number of log entries as the SKIP parameter value.  
:: Thanks to dbenham (http://www.dostips.com/forum/viewtopic.php?f=3&t=2490)
:: for the explanation on how to do this.
   set vCntr=skip=%vCntr%
:: Access the vDatFile file and set the vCntr variable = the line number
:: of the beginning log entry which will not be deleted.  Subtract 1 from
:: that number to preserve the log header.
   for /f "usebackq %vCntr% delims=[] tokens=1,2,*" %%a in (%vDatFile%) do (
      set "vCntr=%%a"
      set /a vCntr-=1
      goto Break2
   )
   :Break2

:: Transfer the logfile, minus the earlier log entries, to the vDatFile file.
:: Tremendous amount of thanks to !K for this solution!
   more +%vCntr% %vLogFile% > %vDatFile%
:: Finally, copy over the old logfile with the revised log.
   copy %vDatFile% %vLogFile%

   :SkipLog

... and that's it! The last, vital part of a self-managing, self-generating FTP script!

Thanks again to everyone who helped!

[orange_batch]

Good. Commenting is good practice too. However I personally would use that as source, and have a working copy that's stripped and minified. No need on such a short script though.

[shadeclan]

Good. Commenting is good practice too. However I personally would use that as source, and have a working copy that's stripped and minified. No need on such a short script though.

Well, it may not be needful for you but I have a mind like a steel sieve and considering that I am programming in DOS Batch, JCL, M204 code, Postscript, HTML, CSS, Javascript and PL/SQL (Oracle) with occasional forays into VB6 / VBScript and VB.net - well, let's just say that by the time I get around the circle, I almost have to relearn the language again. Heavy comments in the code are one way I save myself when something breaks and I haven't programmed in that language for a while.

If you find the code useful, by all means remove the comments and use it all or in part - I don't believe in intellectual property rights.

[Ed Dyreen]

'

I don't believe in intellectual property rights.

What u mean, you don't have any

The Forum:
By submitting information to the forum you transfer the rights of its content to the domain owner. The domain owner reserve the right to add, modify, or remove information to/from the forum as the domain owner feels is appropriate.
The domain owner further reserves the right to use and distribute the information submitted in the forum in a different form, reformat it rearrange it and/or sell it.

See ya ( virtually ),

[shadeclan]

'

I don't believe in intellectual property rights.

What do u mean, you don't have any

The Forum:
By submitting information to the forum you transfer the rights of its content to the domain owner. The domain owner reserve the right to add, modify, or remove information to/from the forum as the domain owner feels is appropriate.
The domain owner further reserves the right to use and distribute the information submitted in the forum in a different form, reformat it rearrange it and/or sell it.

I don't believe in his intellectual property rights, either.

Did you see? Somebody actually liked something I pieced together!

[orange_batch]

Well, it may not be needful for you but I have a mind like a steel sieve

Heavy comments in the code are one way I save myself when something breaks and I haven't programmed in that language for a while.

If you find the code useful, by all means remove the comments and use it all or in part - I don't believe in intellectual property rights.

Haha hold on, you misread what I said, I comment my code too. But for long scripts I keep that as source, then strip and minify a copy for using.