IF Statement Syntax

[Acy Forsythe]

Yeah but what I am looking at for speed here is flow. I have some large files I'm looping through, and while CSVFix is fast, it doesn't do everything I need it to yet. Still working on that. So I end up looping through a few files several times to accomplish the end result and since it's the loop that is slow, cutting a few of them out of the script and making a single loop longer would definitely win some speed back I think...

[Ed Dyreen]

'
Post a small sample of your "slow" code, so we can wrap our minds around it...

still looking for the good old DOS solution are we, that's ok.

[Acy Forsythe]

Well, it's not ALL native DOS. I am using CSVFix for most of the file manipulation, comparisons, etc... I'm using DOS for things that CSVFix won't do yet. Like filling subsequent rows with data from previous rows, but only on certain conditions etc...

I could do the whole thing in DOS, but that would probably be a few thousand lines of script there and take for ever. CSVFix can parse through a 10k line file and spit out the results in a minute or two. It takes a few extra minutes to spit the results out to CON, but redirecting to a file is FAST.

Anyway, I was asked to do this and without any tools I am used to using for this kind of thing, so luckily I found this place and I found CSVFix, taught myself VS2010 ported the code and learned what I learned here which has been a ton.

So what I now have is a newly filled tool-box I can use for similar tasks. I'd rather avoid writing an executable for every 5 minute task that comes my way and used this 5-week task as a testing grounds for stuff. As an example, someone needed some data parsed out of a CSV file and I wrote the batch script to do it (no CSVFix necessary that time) in about 10 minutes based on what I know now. That would have taken a bit longer in C and then there's the executable and installing it etc...

[Acy Forsythe]

Ed, here are two sections I'm fairly certain I can get compressed into one, but after looking at the map-value (I've been there!) I'm not sure putting 10 thousand calls is going to resolve the problem...

First this section takes comma delimited output, %CMD% can be typed output from a CSV file or in my case, I'm using CSVFix. It pads column C with extra zeros to length of 4 digits.

Code: Select all

OR /F "tokens=1-26 delims=," %%A IN ('%CMD%') DO (

    SET Result=
    SET "Text=%%~C"
    %macro_Call% ("Text Result") %macro.StrLen%

    IF "!Result!" LSS "2" (
        SET "Text=000!Text!"
        ) ELSE (
        IF "!Result!" LSS "3" (
            SET "Text=00!Text!"
            ) ELSE (
            IF "!Result!" LSS "4" (
                SET "Text=0!Text!")
            )
        )
ECHO %%~A,%%~B,!Text!,%%~D,%%~E,%%~F,%%~G,%%~H,%%~I,%%~J             
) >> %WORKDIR%\temp.csv

I can shave off one loop by doing the above padding AFTER I've added a prefix to that column. So right now, it's taking a numeric value and padding leading zeros to get to a length of 4.

To do it in the next loop, I would need to add a 2 or 3 character prefix and THEN pad with zeros like this:

4 --> AB4 --> AB0004
5 --> ABC5 --> ABC0005
47 --> 2147 --> 210047

The prefixes are not always alpha, but they are 2 or 3 characters long and the un-prefixed value should always be a 4 digit number with leading zeros when necessary. I can handle the 3 character prefixes and 2 character prefixes seperately.

[Ed Dyreen]

'
Hmm, I don't see how I can boost performance, but it can look nicer

Code: Select all

>> "%WORKDIR%\temp.csv" (
   ::
   FOR /F "tokens=1-26 delims=," %%A IN (

      '%CMD%'

   ) DO (
      ::
      SET Result=
      SET "Text=%%~C"
      %macro_Call% ("Text Result") %macro.StrLen%
      IF "!Result!" LSS "2" (
         ::
         SET "Text=000!Text!"

      ) ELSE IF "!Result!" LSS "3" (
         ::
         SET "Text=00!Text!"

      ) ELSE IF "!Result!" LSS "4" (
         ::
         SET "Text=0!Text!"
      )
      ECHO %%~A,%%~B,!Text!,%%~D,%%~E,%%~F,%%~G,%%~H,%%~I,%%~J
   )         
)

But then again looks are subjective...

However, I do see a pattern

Code: Select all

set "$Fetch=00000"
::
set "$Fetch=!$Fetch:~%Result%!"
::
set "Text=!$Fetch!!Text!"

Remember that set is way faster than if so if you can avoid it...

You could also cheat using a RAMdrive

@johnxysilva, User is a bot, links to commercial site, verified in google's cache.

[johnxysilva]

explanation:-
the execution of if-else block starts from if part,first of all the condition is checked and if true then true part is executed else the false block is executed.
hcg activator

[dbenham]

Code: Select all

FOR /F "tokens=1-26 delims=," %%A IN ('%CMD%') DO (

    SET Result=
    SET "Text=%%~C"
    %macro_Call% ("Text Result") %macro.StrLen%

    IF "!Result!" LSS "2" (
        SET "Text=000!Text!"
        ) ELSE (
        IF "!Result!" LSS "3" (
            SET "Text=00!Text!"
            ) ELSE (
            IF "!Result!" LSS "4" (
                SET "Text=0!Text!")
            )
        )
ECHO %%~A,%%~B,!Text!,%%~D,%%~E,%%~F,%%~G,%%~H,%%~I,%%~J             
) >> %WORKDIR%\temp.csv

Code: Select all

set "$Fetch=00000"
::
set "$Fetch=!$Fetch:~%Result%!"
::
set "Text=!$Fetch!!Text!"

Remember that set is way faster than if so if you can avoid it...

I don't follow Ed's logic, but I think he might have been flirting with a good solution

There is no need for StrLen to left pad a number - Align Right

Code: Select all

FOR /F "tokens=1-26 delims=," %%A IN ('%CMD%') DO (
  SET "Text=000%%~C"
  ECHO %%~A,%%~B,!Text:~-4!,%%~D,%%~E,%%~F,%%~G,%%~H,%%~I,%%~J
) >> %WORKDIR%\temp.csv

If %%C has the potential of exceeding 4 digits, then:

Code: Select all

FOR /F "tokens=1-26 delims=," %%A IN ('%CMD%') DO (
  SET "Text=%%~C"
  IF "!Text:~3!" == "" (
    SET "Text=000!Text!"
    SET "Text=!Text:~-4!"
  )
  ECHO %%~A,%%~B,!Text!,%%~D,%%~E,%%~F,%%~G,%%~H,%%~I,%%~J
) >> %WORKDIR%\temp.csv

Obviously you can incorporate a %Prefix% into either solution above.

WARNING - FOR /F is very convenient for working with CSV as long as there are no empty fields. But be careful - Empty fields will shift subsequent columns to the left:

Code: Select all

for /f "tokens=1-26 delims=," %a in ("A,B,,D,E") do echo %a,%b,%c,%d,%e,

produces:

Code: Select all

A,B,D,E,

Can CSVFix not left pad a number
I've never used it, but I see that it has a Pad function.

I understand not wanting to break out the C compiler. But having to process a file multiple times can definitely slow the process down.

Years ago I did some serious CSV manipulation using Perl on Windows. It is the only time I used that free language, but it was successful and efficient.

Another free language that people seem to like is Ruby - I've not tried it.


Acy - I'm glad you've gotten some useful tools from this site. Until I discovered this site 6 months ago I was clueless to the level of sophistication that is possible with musty old Batch programming. Since then, I've had a blast exploring what is possible. Mostly it has been "mental masturbation' during my free time. But like you I have a set of batch tools that are real time savers in real world work situations.

Dave Benham

[jeb]

WARNING - FOR /F is very convenient for working with CSV as long as there are no empty fields. But be careful - Empty fields will shift subsequent columns to the left:
Code:
for /f "tokens=1-26 delims=," %a in ("A,B,,D,E") do echo %a,%b,%c,%d,%e,

This can be avoided by a simple replace of the comma with ",", so you get never empty fields (there are always two quotes).
And then remove the quotes with %%~a and so on

Code: Select all

setlocal EnableDelayedExpansion
set "line=A,B,,D,E"
set "line="!line:,=","!""
echo Helper result = !line!
for /f "tokens=1-26 delims=," %%a in ("!line!") do echo %%~a,%%~b,%%~c,%%~d,%%~e,

Helper result = "A","B","","D","E"
A,B,,D,E,

jeb

[dbenham]

Perfect simple solution jeb. I like it

Dave Benham

[Acy Forsythe]

@Dave,

Removing the IF block - The length of the number in %%C is never over 4, so your solution worked for shaving off some time, cutting that loop from 50-62 seconds down to 21-30 seconds.

CSVFix does have a pad command, but it's used to pad the number of fields and not the values in them, so if you needed every row to have the same number of fields and the original data did not, you could use pad to add up to X number of fields per row. I have not had much chance to go through the edit command functionality, but there really are no good examples that I can find, and there is a LOT of code to go through. My priorities were to get it to compile in VS2010 and then add any functionality I needed, but I'm fast approaching my deadline for this project so I want a working solution however slow it might be. I can revisit CSVFix and it's code later.

I just realized that I never posted the "Second section" in my post above.

Code: Select all

SET "CMD=%CMD% ^| CSVfix map -f 3 -fv "1 5^,4 58^,10 3F^,73 99^,79 9E^,70 30" -tv "22^,22^,22^,RI2^,38^,AB""
SET "CMD=%CMD% ^| CSVfix map -f 3 -fv "32 18^,64 56^,88 FD^,91 AF^,CityID CSID" -tv "BH^,TD^,RB^,RC^,PRFX""

I was padding the 0's before this point to avoid padding 0's into the middle of a string. This section takes the value of column 3 and maps a prefix to the padded number. If I can pad in the middle, or do the above command in DOS fast enough, I remove that loop, and go through 10k fewer CSV lines in the script.

@JEB,

I already ran into that before I got CSVfix, the output in CSVFix is "","","" etc... so it doesn't run into the double-delimiter problem.