Formatting a File

[shadeclan]

Suppose you have a rather large text file and you need to format each line for consumption by another application. You could do a FOR loop through the file, arrange the data and ECHO it out to a text file simply enough ...

Code: Select all

for /f "tokens=1-14,* delims=:. " %%a in (%cDataFileTmp%) do (
      @echo.%%d%%e%%f%%g%%h,%%i %%k:%%l%%m%%n,%%o>>%cDataFile%
      @echo.%%d %%e %%f %%g %%h,%%i>>%cM204File%
)

... but that takes about 14 minutes. Is there a faster way to do this?

3rd party applications are OK if they are free, portable and reliable, so REGEXP apps like GREP are out. I'd prefer a purely batch solution if there is one.

[Squashman]

3rd party applications are OK if they are free, portable and reliable, so REGEXP apps like GREP are out.

That confuses me as grep is free portable and reliable.

If you have a very large text file it is going to take a while because a for loop reads in the entire file into memory before it starts processing the file.

[Endoro]

look at awk to extract columns.

[Ed Dyreen]

Batch is interpreted line by line by cmd.EXE, batch is slow. Using external executables is always faster, they run from memory and don't need to be translated to machinecode. You could opt for another language like vbScript, that should be about 200 times faster here. Wonder if this is a little faster;

Code: Select all

@echo off &prompt $G

set cDataFileTmp=0.TXT
set    cDataFile=1.TXT
set    cM204File=2.TXT

:: > "%cDataFile%" type nul
:: > "%cM204File%" type nul

3>> "%cDataFile%" (

     4>> "%cM204File%" (

          for /f "useback tokens=1-14* delims=:. " %%a in (

               "%cDataFileTmp%"

          ) do (

               >&3 (echo(%%d%%e%%f%%g%%h,%%i %%k:%%l%%m%%n,%%o)
               >&4 (echo(%%d %%e %%f %%g %%h,%%i)
          )
     )
)

pause

See ya

[shadeclan]

3rd party applications are OK if they are free, portable and reliable, so REGEXP apps like GREP are out.

That confuses me as grep is free portable and reliable.

I didn't want to add the grep dependencies to our server. I have a hard enough time convincing my boss to let me use stuff I find on the internet without having to do a major install.

If you have a very large text file it is going to take a while because a for loop reads in the entire file into memory before it starts processing the file.

I know. That's my problem. I was hoping to find a solution close to the speed of the FindStr function. Even if I could cut the time down to 5 minutes, that would be helpful.

[shadeclan]

Batch is interpreted line by line by cmd.EXE, batch is slow. Using external executables is always faster, they run from memory and don't need to be translated to machinecode. You could opt for another language like vbScript, that should be about 200 times faster here. Wonder if this is a little faster;

Code: Select all

@echo off &prompt $G

set cDataFileTmp=0.TXT
set    cDataFile=1.TXT
set    cM204File=2.TXT

:: > "%cDataFile%" type nul
:: > "%cM204File%" type nul

3>> "%cDataFile%" (

     4>> "%cM204File%" (

          for /f "useback tokens=1-14* delims=:. " %%a in (

               "%cDataFileTmp%"

          ) do (

               >&3 (echo(%%d%%e%%f%%g%%h,%%i %%k:%%l%%m%%n,%%o)
               >&4 (echo(%%d %%e %%f %%g %%h,%%i)
          )
     )
)

pause

See ya

I've seen this sort of notation before. The double colons don't appear to be comments in this context and the redirection symbols don't make sense to me - they seem to be pointing in the wrong direction. Still, I'll give it a try. Thanks, Ed.

I know that VBScript would be faster. I just wanted to get this done as quickly as possible without having to figure out how to do it in another language.