FindRepl.bat:New regex utility to search and replace strings

[Dani]

I'm using /A to make multiple replaces at once. When using regular strings it works fine, however when using a regex with /A it replaces the string with a blank.

Example:

Content of the file (Regex.txt):
Dani.vbs
XXX
Y*&&^
test.txt
Dani.vbs
10.2.3.4
Y*&&^
test.txt

Command:
< ".\Regex.txt" FindRepl "test.txt|XXX" /A "SANITIZED_IPADDRESS|YYY"

Output:
Dani.vbs <--- First row in the file.
YYY <---- Replaced properly
Y*&&^
SANITIZED_IPADDRESS <---- Replaced properly
Dani.vbs
10.2.3.4
Y*&&^ <--- Last row in the file

When using a regex as part of the list:

D:\C#\SanitizeLogs>< ".\Regex.txt" FindRepl "^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$|XXX" /A "SANITIZED_IPADDRESS|YYY"

It replaces all with blanks:
Dani.vbs
<--- Blank
Y*&&^
test.txt
Dani.vbs
<--- Blank
Y*&&^
test.txt

I did another test, with a single regex, which works unless you add the /A switch

Command:

< ".\Regex.txt" FindRepl "^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$" SANITIZED_IPADDRESS

Output:
Dani.vbs
XXX
Y*&&^
test.txt
Dani.vbs
SANITIZED_IPADDRESS <---- Replaced properly
Y*&&^
test.txt

However, when using the /A switch, the output is:

< ".\Regex.txt" FindRepl "^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$" /A SANITIZED_IPADDRESS

Dani.vbs
XXX
Y*&&^
test.txt
Dani.vbs
<--- Blank
Y*&&^
test.txt

Is this a bug or am I doing something wrong?

Thanks in advance,
Dani

[foxidrive]

Feature request:

Antonio, would you think it's a good idea to add a feature to change a specified line number only, and using regular expressions as usual in the rSearch and sReplace ?

So findrepl would pass all other lines through unchanged, and only change the desired line number.

Command something like this:

Code: Select all

type file.txt | findrepl /LINE 3 "VALUETOCHANGE" "Given Value"

file.txt:

aaa VALUETOCHANGE bbb
aaa VALUETOCHANGE bbb
aaa VALUETOCHANGE bbb
aaa VALUETOCHANGE bbb
aaa VALUETOCHANGE bbb

changed text:

aaa VALUETOCHANGE bbb
aaa VALUETOCHANGE bbb
aaa Given Value bbb
aaa VALUETOCHANGE bbb
aaa VALUETOCHANGE bbb


or can this be done some way with findrepl itself?

[Aacini]

I'm using /A to make multiple replaces at once. When using regular strings it works fine, however when using a regex with /A it replaces the string with a blank.


Is this a bug or am I doing something wrong?

Thanks in advance,
Dani

Ok. This point is briefly described in the following phrase in the documentation (perhaps not clearly enough):

it is up to you that each alternative in the rSearch alternation does not match more than one result.

The way that this feature is implemented in FindRepl.bat program is defining in advance an array that directly maps each alternative in the source alternation with the corresponding one in the target alternation. This method is very efficient and it does not depends on the number of options in the alternations, but has the disadvantage that is not possible to use a regex in the rSearch alternation, just literal strings. Sorry...

However, the cure for this problem is simple: run FindRepl.bat one time for each replacement of regex -> constant, and one more time for the rest of "source1|source2|sourceEtc" -> "target1|target2|targetEtc" replacements.

--------------------------------------

@foxidrive: That sounds interesting; I will think about it.

Antonio

[Aacini]

Dave, I was wondering if you would consider a transliteration switch like GnuSed has in the Y switch.

y/abc/xyz replaces - a with x - and b with y - and x with z.

Do you think that is a worthwhile function to include?

@foxidrive:

I would like to note that my FindRepl.bat program have had such feature (called "alternation" in regexp terminology) since its first release:

Code: Select all

FindRepl "a|b|c" /A "x|y|z"

Antonio

[foxidrive]

Dave, I was wondering if you would consider a transliteration switch like GnuSed has in the Y switch.
y/abc/xyz replaces - a with x - and b with y - and c with z.
Do you think that is a worthwhile function to include?

@foxidrive:

I would like to note that my FindRepl.bat program have had such feature (called "alternation" in regexp terminology) since its first release:

Code: Select all

FindRepl "a|b|c" /A "x|y|z"

Antonio

That's something I wasn't aware of Antonio, and it works well, thanks.

I'd like to briefly explain that I had a lot of difficulty coming to grips with how FindRepl works.
Findrepl is a complex tool, and without being critical, I had a lot of trouble with the built-in help
for Findrepl which wasn't intuitive in a way that I could easily follow.

At one time I was going to include your sample usages from the forum into Findrepl.bat
but I had a lot of difficulty formatting the samples in an easy way and gave up.
EDIT: What I mean here is that I wanted to see the examples by giving a switch to findrepl.bat so
I could figure out how to use a feature again without going to the forum and finding the thread.


People like me need lots of examples to learn how to use complex tools effectively, so maybe Findrepl
could have a switch to open your web browser to the help pages, and also to a particular thread on dostips forum
which could have examples but without any discussion in the thread so browsing it would be easier.

What do you think?

[dbenham]

Dave, I was wondering if you would consider a transliteration switch like GnuSed has in the Y switch.

y/abc/xyz replaces - a with x - and b with y - and c with z.

Do you think that is a worthwhile function to include?

Interesting idea. However, I want to support delimited lists and regular expressions in the search terms. I've got some ideas, but it may take a while.

I would like to note that my FindRepl.bat program have had such feature (called "alternation" in regexp terminology) since its first release:

I too was not aware of this feature. Nice.
However, I think calling it "alternation" is misleading. It may use the same syntax as regex alternation, and the search may actually use alternation, but the replacement part is an entirely different concept in my mind. I've seen the words "translate", "transliterate", and "transform" to describe this feature in other utilities, but never "alternation".

I'd like to briefly explain that I had a lot of difficulty coming to grips with how FindRepl works.
Findrepl is a complex tool, and without being critical, I had a lot of trouble with the built-in help
for Findrepl which wasn't intuitive in a way that I could easily follow.

At one time I was going to include your sample usages from the forum into Findrepl.bat
but I had a lot of difficulty formatting the samples in an easy way and gave up.
EDIT: What I mean here is that I wanted to see the examples by giving a switch to findrepl.bat so
I could figure out how to use a feature again without going to the forum and finding the thread.


People like me need lots of examples to learn how to use complex tools effectively, so maybe Findrepl
could have a switch to open your web browser to the help pages, and also to a particular thread on dostips forum
which could have examples but without any discussion in the thread so browsing it would be easier.

What do you think?

I totally understand where foxidrive is coming from. I also was impressed with the power demonstrated by some of the posted examples, but I do not find the syntax intuitive, and I have difficulty understanding the built in help.


Dave Benham

[Aacini]

I completed several important modifications to FindRepl.bat, so I released a new version of this program called FindRepl.bat version 2 and additional updates in versions 2.1 and 2.2. The new code is in the first post of this thread above, as usual. The documentation placed after that code is the base that describe all features of the original version, so you should read it before try the features of the new version. In this post just the new features of FindRepl.bat versions 2+2.1+2.2 are described. In order to have an easier access to the documentation, it has been subdivided in numbered sections with a header before each one, so you may use the "Find in this page" feature of the explorer looking for the "#. " (number-dot-space) of the desired section in order to quickly locate it. This is the index:

1. Introduction to version 2
2. Execute the replacement string as a JScript expression (/J switch)
3. User defined variables (/VAR switch) and predefined functions
   
   1. Range functions
   2. Date functions
   3. File, Folder and Drive functions
   4. Data generating functions (/S switch with /J)
   5. Other functions
4. Creating additional data at end of file (/L switch)
5. Including expressions in alternations (|| separator)
6. Loading alternations from a file (/G switch)
7. A final word

1. Introduction to version 2

The core concept in the design of FindRepl.bat program is the regular expression (regexp). You must learn how to use a regexp in order to appropriately use this program (or any other regexp-based program, for that matter) and there is a large amount of documentation about this subject in the web. The last MS JScript documentation is suggested (version 5.8, before MS nomenclature changed it to IE's "JavaScript"). In FindRepl.bat program the search text is always a regexp and the replace text may only include the standard escaped characters of a JScript string, as described in .replace method at previous documentation.

On the other hand, FindRepl.bat allows to easily use the base JScript regexp features in very advanced ways with minimum effort. Since the original version, FindRepl has been capable of process a wide range of data in several related ways, so the user may use the same program to solve a wider range of different problems. The new FindRepl.bat version 2 increases this flexibility so it is now capable of solve a new range of problems, or solve the former problems in an simpler or faster way. In despite of these improvements, all new features are entirely compatible with previous version, so users of new FindRepl.bat version 2 program may use it in the same way than before.

This is a summary of the new features that appear in FindRepl.bat version 2 built-in help screen; these features are described in the rest of this post.

Code: Select all

         [/J[:n] [/L:jLast]] [/G:file] [/VAR:name=value,...]

  /J[:n]     Specifies that sReplace/sSource texts have JScript expressions.
  /L:jLast   Execute jLast as a JScript expression after the last line of file.
  /G:file    Specifies the file to get rSearch and sReplace texts from.
  /VAR:n=v,..Specifies a series of JScript "name=value" var's to initialize.

2. Execute the replacement string as a JScript expression (/J switch)

If the new /J switch is included, the replacement text is not used as a literal string, but executed as a JScript expression, so the value returned by this expression will be used as the replacement text. This feature is extremely powerful, because it allows to use the full capabilities of JScript language to process the data matched by the search regexp. Of course, in order to appropriately use this feature you must know how to write a valid JScript expression, but in its basic form this point is relatively simple and there are tons of examples and related documentation. For example:

Code: Select all

echo Convert to upper case | FindRepl "\w*" "$0.toUpperCase()" /J
COVERT TO UPPER CASE

echo Capitalize words | FindRepl "\w*" "$0.substr(0,1).toUpperCase()+$0.substr(1).toLowerCase()" /J
Capitalize Words

You may use the JScript Conditional Operator to assemble an If?Then:Else expression:

Code: Select all

(echo Create camel case names & echo One in each line) | FindRepl "([ ]+(\w*))|(\w*)" "$1?$2.substr(0,1).toUpperCase()+$2.substr(1).toLowerCase():$3.toLowerCase()" /J
createCamelCaseNames
oneInEachLine

Of course, any JScript operator/function/method may be used in the replacement expression, even those that does not manage strings; if the result is not a string, it is automatically converted before used in the replacement. You may browse this site: http://msdn.microsoft.com/en-us/library/htbw4ywd(v=vs.84).aspx and select a data type in order to review the methods that exist to manipulate different types of data. Hint: functions are written with parameters enclosed in parentheses: "function2(function1(param1),param2)", but methods are applied to the previous object separated by a dot: "object1.method1().method2(param2)".

You may also use auxiliary variables to store partial results in order to assemble a more complex result comprised of several intermediate steps, until a large procedure is completed. In this case, you should use variable names that start in a capital letter or underscore, to avoid the undesirable modification of a FindRepl.bat variable by chance. For example:

Code: Select all

set "ROT13=C=$0.charCodeAt(0),C+=C>109?-13:C>96?13:C>77?-13:13,String.fromCharCode(C)"
echo Perform ROT13 encryption | FindRepl /I "[A-Z]" "%ROT13%" /J > ROT13.txt
type ROT13.txt

Cresbez EBG13 rapelcgvba

< ROT13.txt FindRepl /I "[A-Z]" "%ROT13%" /J

Perform ROT13 encryption 

If the JScript expression uses a subexpression that was not defined in the search regexp, a run-time error may be issued. Normally you may use up to 10 saved submatched subexpressions in the JScript expression, from $1 to $10; if you want to use more, include the desired number as parameter of the /J switch. For example:

Code: Select all

FindRepl "^(\w+) (\w+) (\w+) (\w+) (\w+) (\w+) (\w+) (\w+) (\w+) (\w+) (\w+) (\w+)" "'The 12th word: '+$12" /J:12

The /J switch also modifies the /S:sSource parameter in the same way as described before, that is, when /J switch is included the sSource text is evaluated as a JScript expression that provide the actual text to process.


3. User defined variables (/VAR switch) and predefined functions

The /VAR:name1=value1,name2=value2... switch allows to initialize JScript variables with the names and values given, so they may be used in JScript expressions (in sReplace and /S:sSource texts). For example:

Code: Select all

set "Find=abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
set "Repl=nopqrstuvwxyzabcdefghijklmNOPQRSTUVWXYZABCDEFGHIJKLM"
echo Perform ROT13 encryption | FindRepl /VAR:Find='%Find%',Repl='%Repl%' /I "[a-z]" "Repl.substr(Find.indexOf($0),1)" /J

If the value of a variable may contain spaces or special Batch characters, enclose the whole switch in quotes; for example: FindRepl "/VAR:Find='%Find% _',Repl='%Repl%_ '" /I ... To insert a quote in the value of a variable, use \x22. You should use names that start in a capital letter or underscore, to avoid the undesirable modification of any FindRepl.bat variable by chance.

You may also use /VAR switch to define equivalent values for frequently used long terms, like: "/VAR:stdin=WScript.StdIn", or to define shorthand functions of long-named methods, like: "/VAR:lc=function(str){return(str.toLowerCase())}". Of course, you may define any function you wish in /VAR switch, so its code appear before the sSource and sReplace JScript expressions that uses it instead of being defined inside the hidden code of FindRepl.bat program.

There are a large number of JScript functions and methods that allows to obtain any possible result in both the sSource and sReplace JScript expressions. However, there are certain frequently used calculations that requires a somewhat large expression. In order to facilitate these operations, FindRepl.bat program provides several predefined functions and variables that may be used in a very simple way. Note that the names of all these values are case sensitive, so you must write them exactly as shown below.

3.1. Range functions

These functions works over a range of "fields" (saved submatched substrings) in each "line" (matched string) and get a result for each line. Also, "global results" (that encompass all processed lines) are obtained when the file ends, so they may be used in the /L last line expression switch. Note that the word "arguments" that appear in first parameter of some functions must be written exactly as shown, that is, "sum" function must be used this way: "sum(arguments,3,8)"; these functions can not be used on the "/Alternations" regular expressions described in section 4 below.

Code: Select all

RANGE FUNCTIONS

choose(arguments,i)             - Returns the value of $i.
hlookup(arguments,a,i,j)        - Returns the index of the largest value in $i..$j that is less or equal than a.
sum(arguments,i,j)              - Returns the sum of $i..$j values; set global SUM[#] and N[#] variables, and line "n" one.
prod(arguments,i,j)             - Returns the product of $i..$j values, set global PROD[#] variables and line "n" one.
max(arguments,i,j)              - Returns the maximum value in $i..$j values; set global MAX[#] variables.
min(arguments,i,j)              - Returns the minimum value in $i..$j values; set global MIN[#] variables.

GLOBAL RESULTS

SUM[1]=SUM[2]=...=0             - Global summations of terms, updated by sum(i,j) function.
N[1]=N[2]=...=0                 - Global numbers of terms, updated by sum(i,j) function.
PROD[1]=PROD[2]=...=1           - Global products of terms, updated by prod(i,j) function.
MAX[1]=MAX[2]=...=-Infinite     - Global maximum of terms, updated by max(i,j) function.
MIN[1]=MIN[2]=...=Infinite      - Global minimum of terms, updated by min(i,j) function.

3.2. Date functions

JScript language manages a Date type value that allows to perform several date and time related operations in an easier way. For example, you may convert a string to a Date value via "new Date(string)" expression that allows a wide range of different data formats in the string, so JScript Date methods may be used on this value. The following Date type predefined functions provides other frequently used Date operations.

Code: Select all

DATE FUNCTIONS

toDate(string)                  - Converts a string in locale date format ("M/D/Y" or "D/M/Y") to a Date value.
showDate(date,format)           - Converts the Date value to a string with the given format.
dateDiff(date1,date2)           - Returns the number of days from Date2 to Date1 (Date1-Date2).
days(date)                      - Returns the number of days from today's date to the given Date value.
dateAdd(date,days)              - Returns the given Date value plus/minus number of days.

You should note that dateDiff, days and dateAdd functions works over values of Date type, not strings, so a Date value must be created via a Date type function, like toDate. However, these functions may also work over "an integer value representing the number of elapsed milliseconds since midnight, January 1, 1970 GMT", as defined in the JScript documentation. The toDate function may also convert this integer value into a Date type one. When values of Date type are displayed, a standard format is used, but you may show dates with a certain format via the second parameter of showDate predefined function. The first example below show all currently supported formats; note that formats 5 and 6 are designed to display elapsed times (number of milliseconds) less than 24 hours.

Code: Select all

C:\> for /L %i in (0,1,12) do @echo/| FindRepl "" "'%i -\t'+showDate(new Date(),%i)" /J
0 -     Dec/15/2014
1 -     Mon Dec 15 2014
2 -     Mon Dec 15 16:18:01 CST 2014
3 -     16:18:01
4 -     04:18:01 p. m.
5 -     22:18:01
6 -     22:18:01.882
7 -     15-12-2014
8 -     2014-12-15
9 -     2014-12-15@16.18.02
10 -    20141215161802
11 -    lunes, 15 de diciembre de 2014
12 -    lunes, 15 de diciembre de 2014 04:18:02 p. m.

C:\> echo %date%
28/11/2014

C:\> echo %date% num | FindRepl "([^ ]*) ([^ ]*) " "'['+$1+'] - ['+$2+']'" /J
[28/11/2014] - [num]

C:\> echo %date% num | FindRepl "([^ ]*) ([^ ]*) " "'['+toDate($1)+'] - ['+$2+']'" /J
[Fri Nov 28 00:00:00 CST 2014] - [num]

C:\> echo %date% -1 | FindRepl "([^ ]*) ([^ ]*) " "'['+dateAdd(toDate($1),$2)+'] - ['+$2+']'" /J
[Thu Nov 27 00:00:00 CST 2014] - [-1]

C:\> echo %date% +7 | FindRepl "([^ ]*) ([^ ]*) " "'['+dateAdd(toDate($1),$2).toLocaleDateString()+'] - ['+$2+']'" /J
[viernes, 5 de diciembre de 2014] - [+7]

REM The following command-line list the names of the files that were modified more than 30 days ago:
(for %a in (*.*) do @echo %~Ta=%a) | FindRepl "([\d\/]+) .*=([^\r\n]*)\r\n" "days(toDate($1))>30?$2+'\r\n':''" /J

Please, note that a Date value is also a number of milliseconds. If you subtract two Date values you get an elapsed time in milliseconds that may be displayed via showDate function with formats 5 or 6. There is a predefined variable called startTime that get the Date (time) when FindRepl.bat program start run, so it allows to easily show elapsed times this way:

Code: Select all

echo/| FindRepl "/L:'Elapsed time: '+showDate(new Date()-startTime,6)" /J

3.3. File, Folder and Drive functions

Code: Select all

FILE AND FOLDER FUNCTIONS

fileExist(file)                 - Returns True if the specified file exists; False if it does not.
folderExist(file)               - Returns True if the specified folder exists; False if it does not.

fileRename(file,newname)        - Renames the name of the specified file.
folderRename(folder,newname)    - Renames the name of the specified folder.

Copy(name,destination)          - Copies the given file/folder to the specified destination (not wild-cards).
Move(name,destination)          - Moves the given file/folder to the specified destination (not wild-cards).
Delete(name)                    - Deletes the given file/folder even if folder is not empty (not wild-cards).

fileCopy(source,destination)    - Copies one or more files from one location to another (wild-cards allowed).
fileMove(source,destination)    - Moves one or more files from one location to another (wild-cards allowed).
fileDelete(filespec)            - Deletes files (wild-cards allowed).

folderCopy(source,destination)  - Recursively copies a folder from one location to another (wild-cards allowed).
folderMove(source,detination)   - Moves one or more folders from one location to another (wild-cards allowed).
folderDelete(filespec)          - Deletes folders and its contents (wild-cards allowed).

fileProperty(file,property)     - Returns the specified property of the given file.
folderProperty(folder,property) - Returns the specified property of the given folder.

FILE AND FOLDER PROPERTIES

Alias   Property name           Description

  A     "Attributes"            A number, the sum of: ReadOnly=1, Hidden=2, System=4, Volume=8,
                                Directory=16, Archive=32, Alias=1024 (shortcut), Compressed=2048.
  TC    "DateCreated"           These properties "returns an integer value representing the number
  TA    "DateLastAccessed"      of elapsed milliseconds since midnight, January 1, 1970 GMT"; these
  TW    "DateLastModified"      numbers may be used as Date type values in the predefined functions.
  D     "Drive"
  NX    "Name"                  Name and extension of File/Folder.
  N     "NameOnly"              This is NOT a File/Folder property, but a result assembled in ...Property function.
  P     "ParentFolder"
  F     "Path"
  SN    "ShortName"
  SF    "ShortPath"
  Z     "Size"                  For folders, returns the size, in bytes, of all files and subfolders contained in the folder.
  X     "Extension"             This is NOT a File/Folder property, but the result of GetExtensionName() method.
  T     "Type"                  For example, for files ending in .TXT, "Text Document" is returned.

EXTRA FOLDER PROPERTIES

        "Files"                 A Files collection consisting of all files contained in the specified folder.
        "IsRootFolder"          True if the specified folder is the root folder; False if it is not.
        "SubFolders"            A Folders collection consisting of all folders contained in the specified folder.

DRIVE FUNCTION AND PROPERTIES

driveProperty(drive,property)   - Returns the specified property of the given drive.

        "AvailableSpace"        Amount of space available to a user on the specified drive or network share.
        "DriveLetter"           Drive letter of a physical local drive or a network share.
        "DriveType"             Value indicating the type of a specified drive:
                                0="Unknown", 1="Removable", 2="Fixed", 3="Network", 4="CD-ROM", 5="RAM Disk"
        "FileSystem"            Type of file system in use for the specified drive: FAT, NTFS, and CDFS.
        "FreeSpace"             Amount of free space available to a user on the specified drive or network share.
        "IsReady"               True if the specified drive is ready; False if it is not.
        "Path"                  Path for a specified file, folder, or drive.
        "RootFolder"            Root folder of a specified drive.
        "SerialNumber"          Decimal serial number used to uniquely identify a disk volume.
        "ShareName"             Network share name for a specified drive.
        "TotalSize"             Total space, in bytes, of a drive or network share.
        "VolumeName"            Volume name of the specified drive.

These functions allows to get information of files, folders and drives, and perform some basic processing on them. The usage of fileProperty function is very simple: just put a file name in the first parameter and a FileSystemObject File object's property name in the second one, both as strings. For example:

Code: Select all

fileProperty("theFile.txt","DateCreated")
fileProperty($1,"Path")

The File properties are fully described at http://msdn.microsoft.com/en-us/library/1ft05taf(v=vs.84).aspx In order to use this function in a simpler way, there are several predefined variables with shorter names that works like "alias" of the properties, with the same names of equivalent features of FOR or DIR commands. This function is particularly useful to perform command-line queries based on the output of DIR /B command.

You must note that in order to appropriately use any function that requires a file name, you must cut the <CR><LF> line ending characters from the subexpressions used to get the function parameters. An example is given below:

Code: Select all

rem This is test.bat
dir /B *.bat
dir /B *.bat | FindRepl "([^\r\n]*)\r\n" "'['+$1+']\r\n'" /J
dir /B *.bat | FindRepl "([^\r\n]*)\r\n" "fileProperty($1,TC)+'\t'+$1+'\r\n'" /J
echo Created         Accessed                Modified        Filename & echo/& dir /B *.bat | FindRepl "([^\r\n]*)\r\n" "showDate(fileProperty($1,TC))+'\t'+showDate(fileProperty($1,TA))+'\t'+showDate(fileProperty($1,TW))+' @ '+showDate(fileProperty($1,TW),3)+'\t'+$1+'\r\n'" /J

The screen output:

Code: Select all

C:\ rem This is test.bat

C:\ dir /B *.bat
FindRepl-v1.bat
FindRepl-v2.1.bat
FindRepl-v2.bat
FindRepl.bat
test.bat

C:\ dir /B *.bat   | FindRepl "([^\r\n]*)\r\n" "'['+$1+']\r\n'" /J
[FindRepl-v1.bat]
[FindRepl-v2.1.bat]
[FindRepl-v2.bat]
[FindRepl.bat]
[test.bat]

C:\ dir /B *.bat   | FindRepl "([^\r\n]*)\r\n" "fileProperty($1,TC)+'\t'+$1+'\r\
n'" /J
Thu Jul 11 12:55:54 CDT 2013    FindRepl-v1.bat
Thu Jul 11 12:55:54 CDT 2013    FindRepl-v2.1.bat
Fri Nov 21 16:15:47 CST 2014    FindRepl-v2.bat
Sun Nov 30 09:43:11 CST 2014    FindRepl.bat
Sun Nov 30 17:05:23 CST 2014    test.bat

C:\ echo Created         Accessed                Modified        Filename   & ec
ho/ & dir /B *.bat   | FindRepl "([^\r\n]*)\r\n" "showDate(fileProperty($1,TC))+
'\t'+showDate(fileProperty($1,TA))+'\t'+showDate(fileProperty($1,TW))+' @ '+show
Date(fileProperty($1,TW),3)+'\t'+$1+'\r\n'" /J
Created         Accessed                Modified        Filename

Jul/11/2013     Jul/11/2013     Nov/13/2014 @ 10:19:10  FindRepl-v1.bat
Jul/11/2013     Nov/15/2014     Nov/30/2014 @ 09:01:05  FindRepl-v2.1.bat
Nov/21/2014     Nov/21/2014     Nov/27/2014 @ 18:32:01  FindRepl-v2.bat
Nov/30/2014     Nov/30/2014     Nov/30/2014 @ 16:12:48  FindRepl.bat
Nov/30/2014     Nov/30/2014     Nov/30/2014 @ 17:11:20  test.bat

The other File functions allows to perform simple file management operations. Some examples.

- Read a text file with "oldname|newname" filename pairs per line and perform a bulk rename with them (http://stackoverflow.com/questions/27210024/batch-renaming-of-files-hidden-meta-fields-using-text-file):

Code: Select all

< RenamePairs.txt FindRepl "([^\|]*)\|([^\r\n]*)\r\n" "(fileExist($1)?fileRename($1,$2):'File not exists: '+$1)+'\r\n'" /J

- Some examples taken from (http://www.dostips.com/forum/viewtopic.php?f=3&t=6081):

1) Convert all .txt files to lower case in the current directory:

Code: Select all

dir /B | FindRepl "([^\r\n]*)\r\n" "fileRename($1,$1.toLowerCase())" /J

2) Rename all 76 ".jpg" files in current directory to an increasing padded number followed by a constant string. Resulting file names should look like "01_Christmas2014.jpg", "02_Christmas2014.jpg", etc.:

Code: Select all

dir /B *.jpg | FindRepl "([^\r\n]*)\r\n" "/VAR:Num=100" "fileRename($1,(++Num).toString().substr(1)+'_Christmas2014.jpg')+'\r\n'" /J

3.4. Data generating functions (Collections)

The functions in this section are designed to provide the actual source text to be processed when /S:sSource text is evaluated as a JScript expression (if /J switch is also given), although they may also be used in sReplace text. These functions works over collections of items contained in a group, like the system (that have a collection of Drives), a folder (that have two collections: Files and SubFolders) or a WMI class. Each function returns a series of properties of every item that exists in the collection enclosed in quotes and separated by commas, one item per line (CSV format).

Code: Select all

DATA GENERATING FUNCTIONS

drivesCollection(prop1,prop2,...)       - Returns the specified list of properties for all drives in the system.
                                          If no parameters are given, returns the Path property of all drives.

filesCollection(folder,prop1,prop2...)  - Returns the specified list of properties for all Files in the given folder.
                                          If no parameters are given, returns the Name property of files in current folder.

foldersCollection(folder,prop1,prop2..) - Returns the specified list of properties for all SubFolders in the given folder,
                                          Files and SubFolders properties can not be directly given (see the description below).
                                          If no parameters are given, returns the Name property of subfolders in current folder.

specialFolders(special,prop1,prop2...)  - Returns the list of properties for all Files or SubFolders in the special folder;
                                          the desired collection is selected by: 'special.Files' or 'special.SubFolders'.
                                          If no parameters are given, returns the Path property of all special folders.

wmiCollection(class,prop1,prop2...)     - Returns the specified list of properties for all elements in the given WMI class.
                                          If just the class is given, returns the names of all properties in the class.
                                          If no parameters are given, returns the names of all classes (in default namespace).
                                          See: http://msdn.microsoft.com/en-us/library/aa394554(v=vs.85).aspx

Remember that Drive, File and Folder properties are given above. Examples of these functions follows, starting with Drives collection:

Code: Select all

C:\ FindRepl "/S:drivesCollection('Path','FreeSpace','TotalSize')" /J
"C:","423201288192","474821423104"
"D:","2942341120","24459079680"
"E:","Not ready","Not ready"

C:\ set "source=drivesCollection('Path','FreeSpace','TotalSize')"
C:\ FindRepl /S:=source /Q:# "#([^#]*)#,#([^#]*)#,#([^#]*)#\r\n" "$1+' = '+$2+' = '+$3+'\r\n'" /J
C: = 423201288192 = 474821423104
D: = 2942341120 = 24459079680
E: = Not ready = Not ready

C:\ set "search=#([^#]*)#,#([^#]*)#,#([^#]*)#\r\n"
C:\ FindRepl /S:=source /Q:# =search "$1+'\t'+(S=Math.floor(100-$2*100/$3))+'% Occupied'+(S>85?'\tRequires cleanup!':'')+'\r\n'" /J
C:      10% Occupied
D:      87% Occupied    Requires cleanup!
E:      NaN% Occupied

The values returned by filesCollection and foldersCollection functions are similar to the ones produced by DIR command and/or the modifiers of FOR command, with some interesting differences. For example, a simple DIR /A-D /B equivalent:

Code: Select all

FindRepl "/S:filesCollection()" /J

... and a DIR /AD /B one:

Code: Select all

FindRepl "/S:foldersCollection()" /J

Of course, you may specify the folder that will be retrieved and the list of properties that will be listed for each item in such a folder. For example:

Code: Select all

FindRepl "/S:filesCollection('.',TC,TA,TW,NX)" /J

Please, note that N ("NameOnly") and X ("Extension") are NOT JScript's File/Folder properties, so they can not be used in the list given in filesCollection/foldersCollection functions. You must use NX ("Name") name-and-extension property besides the rest of properties listed above.

The Files and SubFolders properties of each folder (that is, folder.Files and folder.SubFolders) can not be directly used in foldersCollection function because they are collections themselves, so they needs to be converted to a series of files or folders via another function. In the case of Files property, you must write a call to filesCollection function as parameter of foldersCollection, that is:

Code: Select all

FindRepl "/S:foldersCollection('.','Path','filesCollection(folder.Files,\'Name\')')" /J

In previous example the whole parameter that includes the "folder.Files" string is evaluated as a JScript expression, that is, the filesCollection function.

In the case of SubFolders property, when you place it as parameter in foldersCollection function there is an implicit recursive call of the same foldersCollection function with the same parameters, but processing the nested folder.SubFolders collection instead. For example:

Code: Select all

FindRepl "/S:foldersCollection('.','Path','folder.SubFolders')" /J

In this case, the parameter that contain the "folder.SubFolders" string is evaluated as JScript expression and, if the result is not empty, the nested SubFolders collection is processed. However, you may use this parameter to exclude certain SubFolders from being processed even if they are not empty; to do that, just write an expression that returns a false value that includes the "folder.SubFolders" string. For example, to process the contents of all SubFolders in the current folder, excepting the one called "Backups":

Code: Select all

FindRepl "/S:foldersCollection('.','Path','folder.Name!=\'Backups\'?folder.SubFolders:false')" /J

Additionally, you may insert another parameter after the "folder.SubFolders" one that will be evaluated after it, so certain advanced manipulations are possible. The command below allows to set the number of levels of nested SubFolders that will be processed. For example, to only process two additional nested levels:

Code: Select all

FindRepl "/VAR:Levels=2" "/S:foldersCollection('.','Path','Levels?(--Levels,folder.SubFolders):false','++Levels')" /J

As said before, the parameter that contains the "folder.Files" string is evaluated as a JScript expression, so it may include a conditional operator that serve to exclude certain Files collections in a similar way. For example, the next command does not process the files included in "Backups" folder, but all existent subfolders are processed including "Backups" folder and all folders (and files) under it:

Code: Select all

FindRepl "/S:foldersCollection('.',F,'folder.Name!=\'Backups\'?filesCollection(folder.Files,NX):\'\'','folder.SubFolders')" /J

Of course, any possible combination of values may be used this way, so foldersCollection predefined function is capable of generate precisely the set of values you need in order to solve your particular problem.


The specialFolders is a collection that contains the entire set of Windows special folders, such as the Desktop folder, the Start Menu folder, and the Personal Documents folder. The special folder name is used to retrieve the special folder you want, and the .Files or .SubFolders selector property specifies the type of items to get from the special folder. The specialFolders function returns an empty string if the requested special folder is not available. For example, Windows 95 does not have an AllUsersDesktop special folder and returns an empty string if the first parameter is 'AllUsersDesktop.Files' or 'AllUsersDesktop.SubFolders'.

The following special folders are available:

Code: Select all

AllUsersDesktop
AllUsersStartMenu
AllUsersPrograms
AllUsersStartup
Desktop
Favorites
Fonts
MyDocuments
NetHood
PrintHood
Programs
Recent
SendTo
StartMenu
Startup
Templates

A few examples:

Code: Select all

FindRepl "/S:specialFolders()" /J
FindRepl "/S:specialFolders('MyDocuments.SubFolders','Name')" /J
FindRepl "/S:specialFolders('Fonts.Files','Name')" /J
FindRepl "/S:specialFolders('Fonts.Files','Name')" /J ".*\.FNT"

The wmiCollection predefined function is the key to have access to a huge amount of information related to the system. In order to not waste even more words in an over-described topic, you are encouraged to review the WMI documentation at the site linked in the FindRepl /help screen. Below there is a list of just a few useful classes and properties that appear in the first two divisions at http://msdn.microsoft.com/en-us/library/aa394388(v=vs.85).aspx site. The elements in the list are written as parameters of wmiCollection function (the class first, followed by its properties) so you may easily copy and paste they in the command line in order to execute FindRepl.bat program and show the result.

Code: Select all

1- Computer System Hardware Classes

'Win32_Keyboard','Description','DeviceID','Layout'
'Win32_DiskDrive','Description','MediaLoaded','SerialNumber','Status'
'Win32_PhysicalMemory','Capacity','Name','Speed'
'Win32_Processor','CurrentClockSpeed','DataWidth','Description','Manufacturer','Name','NumberOfCores'
'Win32_Battery','BatteryStatus','Description','DeviceID','EstimatedChargeRemaining','EstimatedRunTime','Status'

2- Operating System Classes

'Win32_Environment','Name','SystemVariable','UserName','VariableValue'
'Win32_SystemDriver','Name','ServiceType','Started','State'
'Win32_Directory','CreationDate','Extension','FileName','FileSize','FileType','LastAccessed','LastModified','Name','Status'
'Win32_DiskPartition','BootPartition','Caption','Size','Type'
'Win32_LogicalDisk','DeviceID','DriveType','FileSystem','FreeSpace','MediaType','Size','VolumeName'
'Win32_ComputerSystem','Description','Manufacturer','Model','NumberOfProcessors','SystemType','TotalPhysicalMemory','UserName'
'Win32_OperatingSystem','CountryCode','LocalDateTime','Name','RegisteredUser','SerialNumber','Version','WindowsDirectory'
'Win32_Process','Handle','Name','ParentProcessId'
'Win32_Service','Name','ServiceType','Started','State'
'Win32_LocalTime','Day','DayOfWeek','Hour','Minute','Month','Quarter','Second','WeekInMonth','Year'
'Win32_Account','Caption','Name','SID','Status'
'Win32_UserAccount','Caption','FullName','Name','SID','Status'

For example:

FindRepl "/S:wmiCollection('Win32_Service','Name','ServiceType','Started','State')" /J

You must be aware that certain WMI requests may take several seconds, like class 'Win32_Directory' that returns the data of all files and folders that exists in the disk, or executing wmiCollection() with no parameters in order to display the names of all existent classes in the default name space (root\cimv2). For example:

Code: Select all

FindRepl "/S:wmiCollection()" "/L:'Elapsed time: '+showDate(new Date()-startTime,6)" /J

3.5. Other functions

The functions in this section does not fits in anyone of previous sections.

Code: Select all

OTHER FUNCTIONS

env(varName)                      - Returns the value of the given environment variable.
prompt(message)                   - Prompts the user with the given message, returns the value entered at keyboard.
FOR(arguments,init,test,inc,body) - Emulates the operation of "for" JScript statement:
                                    the successive values of "body" are joined together and returned as result.

The prompt function allows to prompt the user for the value that will be used in each replacement, so it can provide the final solution when the matching of search and replacement values can not be expressed in terms of any rules, or when you need to input data. This function allows FindRepl.bat program to complete prefilled forms or to interactively select options. A couple examples:

Code: Select all

C:\> echo NAME's favorite color is COLOR and favorite pets are PETS. | FindRepl "NAME|COLOR|PETS" "prompt('Enter your '+$0+': ')" /J
Enter your NAME: Antonio
Enter your COLOR: green
Enter your PETS: dogs
Antonio's favorite color is green and favorite pets are dogs.

- List files that were modified more than 30 days ago and size larger than 1 MB, and allows the user to selectively delete them:

Code: Select all

dir /B | FindRepl "([^\r\n]*)\r\n" "(days(fileProperty($1,TW))>30)&&(fileProperty($1,Z)>1048576)?prompt('Delete '+$1+'? ').substr(0,1).toUpperCase()=='Y'?Delete($1)+'\r\n':'':''" /J

There are some examples on FOR predefined function later.


4. Creating additional data at end of file (/L switch)

If /J switch was given to evaluate JScript expressions, then an additional JScript expression may be evaluated after the last line of data file. This feature allows to generate final results based on previously processed data that must be generated when the input file was processed. An example of such processing is sum(i,j) FindRepl predefined function that accumulate the $1, $2, ... processed values into SUM[1], SUM[2], ... "global" predefined array elements (and the number of processed terms in N[1], N[2], ...).

For example, given this MagicSquare.txt file:

Code: Select all

37 78 29 70 21 62 13 54 5 
6 38 79 30 71 22 63 14 46 
47 7 39 80 31 72 23 55 15 
16 48 8 40 81 32 64 24 56 
57 17 49 9 41 73 33 65 25 
26 58 18 50 1 42 74 34 66 
67 27 59 10 51 2 43 75 35 
36 68 19 60 11 52 3 44 76 
77 28 69 20 61 12 53 4 45 

... and the following code:

Code: Select all

@echo off
set "search=^(\d+) (\d+) (\d+) (\d+) (\d+) (\d+) (\d+) (\d+) (\d+)"
set "replace='         '+$1+' + '+$2+' + '+$3+' + '+$4+' + '+$5+' + '+$6+' + '+$7+' + '+$8+' + '+$9+'  =  '+sum(arguments,1,9)"
set "lastLine='\r\nTotals: '+SUM[1]+'  '+SUM[2]+'  '+SUM[3]+'  '+SUM[4]+'  '+SUM[5]+'  '+SUM[6]+'  '+SUM[7]+'  '+SUM[8]+'  '+SUM[9]"
< MagicSquare.txt FindRepl =search =replace /J /L:=lastLine

... this output is obtained:

Code: Select all

         37 + 78 + 29 + 70 + 21 + 62 + 13 + 54 + 5  =  369
         6 + 38 + 79 + 30 + 71 + 22 + 63 + 14 + 46  =  369
         47 + 7 + 39 + 80 + 31 + 72 + 23 + 55 + 15  =  369
         16 + 48 + 8 + 40 + 81 + 32 + 64 + 24 + 56  =  369
         57 + 17 + 49 + 9 + 41 + 73 + 33 + 65 + 25  =  369
         26 + 58 + 18 + 50 + 1 + 42 + 74 + 34 + 66  =  369
         67 + 27 + 59 + 10 + 51 + 2 + 43 + 75 + 35  =  369
         36 + 68 + 19 + 60 + 11 + 52 + 3 + 44 + 76  =  369
         77 + 28 + 69 + 20 + 61 + 12 + 53 + 4 + 45  =  369

Totals: 369  369  369  369  369  369  369  369  369

You may also obtain the same result with this lastLine, that uses the FOR FindRepl predefined function:

Code: Select all

set "lastLine='\r\nTotals: '+FOR(arguments,'I=1','I<=9','I++','SUM[I]+@  @')"

... that uses at-signs instead of quotes, so /Q:@ switch needs to be included this way:

Code: Select all

< MagicSquare.txt FindRepl =search =replace /J /L:=lastLine /Q:@

5. Including expressions in alternations (|| separator)

In the original version of FindRepl, when /A switch is used to define several pairs of search-replacement strings (separating each value by | to form an "alternation") the alternatives in both parts can only be literal strings:

Code: Select all

FindRepl "Sun|Mon|Tue|Wed|Thu|Fri|Sat" /A "Dom|Lun|Mar|Mié|Jue|Vie|Sab"

In new version 2, when /J switch is given both parts are comprised of expressions, that is, the search part have regular expressions and the replacement part have JScript expressions:

Code: Select all

FindRepl "Sun|Mon|Tue|Wed|Thu|Fri|Sat" /A "'A day of week'" /J

In this example all input data that matches anyone of the values of the search regexp are grouped in (replaced by) the same string. Please, note the difference: in the original example there are seven literal strings that are replaced by seven literal strings; in the new /J example, there is one regular expression (an alternation) that is replaced by one JScript expression.

You may include several expressions (regexp or JScript) in each part; to do that, just separate each expression by two pipe characters || in order to form an "hyper-alternation". Below there is an example with three alternatives that are all alternations themselves:

Code: Select all

FindRepl "Red|Green|Blue||Peter|Paul|Mary||North|South|East|West" /A "'A color'||'A name'||'A cardinal point'" /J

rem For example:
set "search=Red|Green|Blue||Peter|Paul|Mary||North|South|East|West"
set "replace='A color'||'A name'||'A cardinal point'"
echo Peter bought a red house at North of city | FindRepl /I =search /A =replace /J

A name bought a A color house at A cardinal point of city

Each alternative in both the search and replacement parts are independant from each other. You may use $0 in each one of the replacement JScript expressions in order to retrieve the data matched by its corresponding regular expression, like in:

Code: Select all

set "replace='A color: '+$0||'A name: '+$0||'A cardinal point: '+$0"

You may subdivide parts of different strings, enclose they in parentheses in a nested alternation and combine it with common parts in order to reassemble larger strings. For example: "UpperCase|LowerCase||North|South|East|West" /A "'A letter size'||'A cardinal point'" /J may also be written this way: "(Upp|Low)erCase||(Nor|Sou)th|(Ea|We)st" /A "'A letter size'||'A cardinal point'" /J.

The alternatives are evaluated in left-to-right order and stops at the first one that matchs. This means that if narrow regexps are placed first and wider ones after, the last regexps will get just those data that was not "captured" by anyone of previous regexps. For example:

Code: Select all

FindRepl "\d||\w||." /A "'Digits'||'Just letters \(not digits\)'||'The rest of characters \(not digits nor letters\)'" /J

If the regexp alternatives have subexpressions enclosed in parentheses (as explained in the base documentation of original FindRepl version and in the previous alternation documentation), then each one of the replacement JScript expressions may use $1, $2, $3, etc. in order to retrieve the corresponding saved submatched subexpression. An example of this point follows.

Suppose that a data file contain several email addresses and URI locations that needs to be processed, like this one:

Code: Select all

Line with an email address: joedoe@unknown.org
Line with an URI location: http://msdn.microsoft.com:80/scripting/default.htm
Please send mail to george@contoso.com and someone@example.com. Thanks!

Previous data file may be processed with original FindRepl.bat via two executions, one for each different regexp:

Code: Select all

rem Process the email addresses of the data file:
< theFile.txt FindRepl.bat "(\w+)@(\w+)\.(\w+)" "Email: $1 ATSIGN $2 DOT $3" > newFile.txt
move /Y newFile.txt theFile.txt

rem Process the URI locations of the data file:
< theFile.txt FindRepl.bat "(\w+):\/\/([^/:]+)(:\d*)?([^# ]*)" "URI: $1 SITE $2 PORT $3 PATH $4" > newFile.txt

Previous process may be achieved in one step grouping the regular expressions and the replacement strings (changed into JScript expressions) in two alternations and use they in just one execution of FindRepl.bat version 2:

Code: Select all

< theFile.txt FindRepl.bat "(\w+)@(\w+)\.(\w+)||(\w+):\/\/([^/:]+)(:\d*)?([^# ]*)" /A
        "'Email: '+$1+' ATSIGN '+$2+' DOT '+$3||'URI: '+$1+' SITE '+$2+' PORT '+$3+' PATH '+$4" /J > newFile.txt

When alternations are used, it is not necessary to define the maximum number of subexpressions that will be used in the regexps. If a number is given in /J:n switch, it is ignored. However, JScript language limits the total number of subexpressions that may be used in all alternations to 99.


6. Loading alternations from a file (/G switch)

The new /G switch allows to place the search/replace alternations in a file. The use of this feature is simple because all previously described facilities works in the same way; the only difference is that the search/replacement alternations are read from a file instead of placed in parameters.

In the description of || feature at previous point, the /J switch allows to use regexps/JScript expressions instead of literal strings, and /A switch is used to set a find-replace operation instead of a find-only one. The same 4 combinations of /A and /J switches are used in the same way when /G switch is added, as shown below:

Code: Select all

Command:                                File lines have:       Operation performed:

FindRepl /G:filename.txt                literal                Find literals
FindRepl /G:filename.txt  /A            literalS|literalR      Find literalS, replace by literalR
FindRepl /G:filename.txt      /J        regexp                 Find regexps
FindRepl /G:filename.txt  /A  /J        regexp||JScript        Find regexp, replace by JScript expression

This simple example of search alternation:

Code: Select all

FindRepl "one|two|three"

... is equivalent to have this Search.txt text file:

Code: Select all

one
two
three

... and execute FindRepl this way:

Code: Select all

FindRepl /G:Search.txt

To define search and replace alternations, include the /A switch and separate each search-replace literals pair by a single pipe | character. For example, this search-replace alternation:

Code: Select all

FindRepl "Sun|Mon|Tue|Wed|Thu|Fri|Sat" /A "Dom|Lun|Mar|Mié|Jue|Vie|Sab"

... may be placed in SearchRepl.txt file this way:

Code: Select all

Sun|Dom
Mon|Lun
Tue|Mar
Wed|Mié
Thu|Jue
Fri|Vie
Sat|Sab

... and execute FindRepl this way:

Code: Select all

FindRepl /G:SearchRepl.txt /A

In order to define regexp-JScript pairs, add the /J switch and separate each search-replace pair by two pipe || characters. This previous example:

Code: Select all

FindRepl "Red|Green|Blue||Peter|Paul|Mary||North|South|East|West" /A "'A color: '+$0||'A name: '+$0||'A cardinal point: '+$0" /J

... is equivalent to have this SearchReplaceJ.txt file:

Code: Select all

Red|Green|Blue||'A color: '+$0
Peter|Paul|Mary||'A name: '+$0
North|South|East|West||'A cardinal point: '+$0

... and execute FindRepl this way:

Code: Select all

FindRepl /G:SearchReplaceJ.txt /A /J

The same file used for a find-replace operation may also be used for a find-only operation; if /A switch is not given, just the search alternatives are loaded. This previous example:

Code: Select all

< theFile.txt FindRepl.bat "(\w+)@(\w+)\.(\w+)||(\w+):\/\/([^/:]+)(:\d*)?([^# ]*)" /A
        "'Email: '+$1+' ATSIGN '+$2+' DOT '+$3||'URI: '+$1+' SITE '+$2+' PORT '+$3+' PATH '+$4" /J > newFile.txt

... is equivalent to have this BothUsesJ.txt file:

Code: Select all

(\w+)@(\w+)\.(\w+)||'Email: '+$1+' ATSIGN '+$2+' DOT '+$3
(\w+):\/\/([^/:]+)(:\d*)?([^# ]*)||'URI: '+$1+' SITE '+$2+' PORT '+$3+' PATH '+$4

... and execute FindRepl this way:

Code: Select all

< theFile.txt FindRepl /G:BothUsesJ.txt /A /J > newFile.txt

... but the same file may be used in a find-only operation this way:

Code: Select all

< theFile.txt FindRepl /G:BothUsesJ.txt /J

The JScript regular expressions documentation does not specify any limit in the number of regexps that may be defined this way.

Remember that you can not include quotes in any line of the file, so you may use /Q switch in order to use a replacement character instead of quotes. However, you may insert quotes at beginning and end of the lines to clearly delimit the replacement part; this may be useful if the replace literal string is not given, that is commonly used to eliminate the search part. For example, the following find-replace file allows to convert a .m3u playlist into a different format(http://stackoverflow.com/questions/27280981/converting-a-m3u-playlist-with-sed):

Code: Select all

"\.\.\\\.\.\\|C:\\Users\\Me\\"
"\.\.\\|C:\\Users\\Me\\Music\\"
"#EXTM3U|"
":0,|:-1,"
"\\|\/"
"C:\/|file:\/\/\/C:\/"
"\.mp3\r\nfile|\r\nfile"
"^\r\n|"

In order to write clearer find-replace files you may include empty lines and comment lines that starts in a double-backslash, and also var lines with the same format of JScript's ones that allows to include the variables initialization in the file itself. For example, a find-replace file in charge of remove unwanted characters from an OCR scanned file may be large and complex (http://www.dostips.com/forum/viewtopic.php?f=3&t=6044&p=38111#p38111), but this complexity becomes more readable when an explanation is added to the file in the form of comments:

Code: Select all

// OCRfix.txt: Fix the data captured via OCR scan, used with FindRepl.bat program

// These characters must be escaped with backslash: \*+?^$.[]{}()|
// * match "zero or more", + match "one or more", ? match "zero or one"

// a) for the symbols ,?!. is found, they should follow a word(no space).
//    A space should be there after ,?!. before the next word
// -> Replace "zero or more spaces" ",|?|!|." "zero or more spaces" by
//    "no spaces" ",|?|!|." "one space"
// c) Capitalization of first character after ?!.
// -> Convert next character, after NOT ",", to uppercase
 *(,|\?|!|\.) *(.)||$1+' '+($1!=',')?$2.toUpperCase():$2

// b) "(open quote) should have a space before and "(close quote) a space after
// -> Initialize "inQuote" variable with false
var inQuote=false;
// -> Replace "zero or more spaces" "quote" IF NOT inQuote by "space" "quote" AND set inQuote to true
 *\x22||inQuote?$0:inQuote=true,' \x22'
// -> Replace "quote" "zero or more spaces" IF inQuote by "quote" "space" AND set inQuote to false
\x22 *||inQuote?inQuote=false,'\x22 \':$0

// Previous lines use \x22 for "quote", because a literal quote can NOT be included;
// this way, this file does NOT depend on using the /Q switch in FindRepl execution

// d) paragraph begins is determined by 2 times ENTER and end after with 2 times enter as well.
// -> paragraph ends at "\r\n\r\n"    (Or should it be "\r\r"?)

// e) A tab is needed at the beginning of a paragraph with first letter capitalize.
// -> Replace "\r\n\r\n" "any char" by "\r\n\r\n" "tab" "any char capitalized"
\r\n\r\n(.)||'\r\n\r\n\t'+$1.toUpperCase()

// f) If there is ENTER in between this paragraph, strip the enter
// -> Remove any ENTER that passed previous rules
\r\n||''

// g) Any other tab or extra spaces should be strip.
//    Only a single space before and after a word.
// -> Replace multiple tabs/spaces by one space
(\t| )+||' '

7. A final word

FindRepl.bat version 2 is a large and complex program. There are many possible situations in which this program may fail. By its nature, any JScript program may present run-time errors and this possibility augments by the fact that FindRepl allows to insert executable code provided by the user. It is simply impossible that FindRepl may detect all possible problematic situations and run error-free all the time.

FindRepl.bat program grow from a series of features that were added after the other, so it is not as efficient as could be. It requires a deep redesign and a large rewrite, but I have not enough time to do that right now. Unfortunately, the new version 2 features were added in the same unplanned way, but they works pretty well in the cases I tested. However, they may present unexpected behaviour when old and new features are combined in unusal ways. I had to write and post this program this way because I was in urgent need of publish my new ideas as soon as possible!

You should not forget that the goal is to obtain useful results from FindRepl.bat program, not to test in which situations it may fail. If you provide the right parameters accordingly to the documentation, you surely will obtain useful results. If you are sure that your parameters are right and you get a wrong result, please report this situation with complete details so I am able to fix it. If you help me to debug this program, all of us will get a better utility in the benefit of the community...

Antonio

[Aacini]

I'd like to briefly explain that I had a lot of difficulty coming to grips with how FindRepl works.
Findrepl is a complex tool, and without being critical, I had a lot of trouble with the built-in help
for Findrepl which wasn't intuitive in a way that I could easily follow.

At one time I was going to include your sample usages from the forum into Findrepl.bat
but I had a lot of difficulty formatting the samples in an easy way and gave up.
EDIT: What I mean here is that I wanted to see the examples by giving a switch to findrepl.bat so
I could figure out how to use a feature again without going to the forum and finding the thread.


People like me need lots of examples to learn how to use complex tools effectively, so maybe Findrepl
could have a switch to open your web browser to the help pages, and also to a particular thread on dostips forum
which could have examples but without any discussion in the thread so browsing it would be easier.

What do you think?

I totally understand where foxidrive is coming from. I also was impressed with the power demonstrated by some of the posted examples, but I do not find the syntax intuitive, and I have difficulty understanding the built in help.


Dave Benham

The built-in help screen is just a quick reference guide; FindRepl.bat is complex enough so it can not be adequately described in just text screens. However, the documentation in this site is very good: it explains FindRepl features with detail, starting from the basic ones and advancing to the more complex with numerous examples. Most of the examples had been taken from real problems posted elsewhere and the link to each one of them is included, so the interested user may review the original problem description and solution, and compare such solution with the way to achieve it with FindRepl...

Perhaps the problem is the same than before: this site is not a good place to post documentation. I subdivided the documentation in sections and included an index at beginning so the user may know in advance its contents and look for specific parts; I also included a way to easily open the different sites referred in the documentation via the FindRepl help screen. I hope these changes may encourage a more easier and complete review of the documentation. What do you think?

However, I think calling it "alternation" is misleading. It may use the same syntax as regex alternation, and the search may actually use alternation, but the replacement part is an entirely different concept in my mind. I've seen the words "translate", "transliterate", and "transform" to describe this feature in other utilities, but never "alternation".

I always use the same nomenclature of original JScript/regexp documentation when I use the same features, so any user that wants to review them in the original source can found the same terms; otherwise, the documentation may look confusing. For example: "This feature is called transliteration, but if you want to review its description in the JScript/regexp documentation, look for alternation instead" (?). Besides, I think that the regexp description of this feature is clear enough: 'Alternation in a regular expression enables you to group choices between two or more alternatives. You can essentially specify "this OR that" in a pattern'. I just have extended this concept a little in order to apply it in the same way to the replacement part...

Antonio

[foxidrive]

Thank you for all your work Antonio, the extra help options are very useful.

Some suggestions and comments follow, after an initial look:

Help Option 3 and 4 seem to display the same page in my browser.

Would you consider changing some help text that may make it clearer for your readers?

Code: Select all

  /B:rBlock  Extra text to be searched for in the blocks of lines.
  /R         Prints only lines that are changed.
  /A         "a|b" /A "j|k" will replace a with j and b with k :extra terms can be added

On an initial reading I am not sure what these refer to Antonio.

Code: Select all

  /$:1:2...  Specifies to print saved submatched substrings instead of lines.
  /ARG1:...  Specifies texts to initialize JScript variables with same names.

Is this what the second one means?

Code: Select all

 /ARG:... Specifies and initializes JScript variable names.

Some other readers might like to comment too - I find the following format to be easier to read, if it outlines what you mean.

Code: Select all

The top three methods define Blocks of lines when using the rSearch option:
                                                 
/O:s:e                 Add S and E (Start/End with optional signs) to matching lines.
/E:rEndBlk             From rSearch line  to rEndBlk line.
/E:rEndBlk /O:s:e      Add S to rSearch line and E to rEndBlk line.
/O:s:e                 When rSearch is not given: From line S to line E.

If S or E is negative: It specifies counting lines from the end of file.
If E is not given:     It defaults to the last line of the file (same as -1).

The following is difficult for me to understand properly: it was a big issue with me and findrepl V1 help.
I am not sure how to reformat it to be clearer.

Code: Select all

The output varies with the parameters and switches used, as follows:

rSearch        /V                 Block            /B:rBlock        /$:1:2...
-------------------------------------------------------------------------------
Matched        Non-matched        Blocks of        Search /B:       Saved
lines.         lines.             lines.           in blocks        submatches.

sReplace       /R                 Block            /B:rBlock
-------------------------------------------------------------------------------
All file       Only replaced      Search /B:rBlock in blocks
lines.         file lines.        and replaces matched lines.

I hope you find this constructive.

[foxidrive]

Antonio, a small bug here - I had a use for findrepl today and V2.0 doesn't allow this syntax:

Code: Select all

findrepl /o:1:1 <file.txt

FindRepl: No search part given

[Aacini]

Antonio, a small bug here - I had a use for findrepl today and V2.0 doesn't allow this syntax:

Code: Select all

findrepl /o:1:1 <file.txt

FindRepl: No search part given

Yes, it was a small bug. Thanks for report it. I just fixed it and modified the posted code. I want also to do several modifications based on your suggestions for a clearer descriptions (I like they ). I will post a notice when I completed all these changes, but I think that fixing this bug was an urgent thing!

Antonio

PS - There is a new answer at S.O. that show a very clear example of /O and /B switches usage; perhaps you want to review it. It is here.

[dbenham]

From a few posts ago:

Accordingly to http://msdn.microsoft.com/en-us/library ... 71(v=vs.84).aspx regexp alternation documentation, "The largest possible expression on either side of the pipe character is matched". This means that if the input data can be matched by more than one regular expression in the alternation, the selected one will be the largest possible one; if several regexps are of the same size, they are tested to match in left-to-right order.

I believe you are misinterpreting the MS doc. (I also was puzzled at first). It is simply stating that the entire expression on either side is alternated. For example: "a b|c d" should be thought of as "(a b) or (c d)", not "a (b or c) d".

The regex engine works left to right, and short circuits once a matching alternation is found. It does not pick the largest one when multiple expressions match.
For example:

Code: Select all

C:\test>echo abc abc|findrepl "^a|ab" ""
bc c

The "^a" expression matches the "a" at the beginning of the line and short circuits, thus not allowing the "ab" to match the beginning. But the second a is not at the line beginning, so "ab" gets a chance to match the 2nd occurrence.


Dave Benham

[bars143]

Hi! Antonio,

i have a problem matching uppercase and lowercase in /A switch?

inputfile.txt

View - leading uppercase
view - all lowercase
VIEW - all uppercase
vieW - trailing uppercase

test.bat

Code: Select all

@echo off

echo using leading uppercase with "/i"
echo type inputfile.txt ^|FindRepl /i "View" /a "displayed"
echo.
      type inputfile.txt |FindRepl "View" /a "displayed"
echo.

echo using leading uppercase with no "/i"
echo type inputfile.txt ^|FindRepl "View" /a "displayed"
echo.
      type inputfile.txt |FindRepl "View" /a "displayed"
echo.

echo using all-lowercase with "/i"
echo type inputfile.txt ^|FindRepl /i "view" /a "displayed"
echo.
      type inputfile.txt |FindRepl "view" /a "displayed"
echo.

echo using all-lowercase with no "/i"
echo type inputfile.txt ^|FindRepl "view" /a "displayed"
echo.
      type inputfile.txt |FindRepl "view" /a "displayed"
echo.

pause

ouput in console:

Code: Select all

using leading uppercase with "/i"
type inputfile.txt |FindRepl /i "View" /a "displayed"

displayed - leading uppercase
view - all lowercase
VIEW - all uppercase
vieW - trailing uppercase

using leading uppercase with no "/i"
type inputfile.txt |FindRepl "View" /a "displayed"

displayed - leading uppercase
view - all lowercase
VIEW - all uppercase
vieW - trailing uppercase

using all-lowercase with "/i"
type inputfile.txt |FindRepl /i "view" /a "displayed"

View - leading uppercase
displayed - all lowercase
VIEW - all uppercase
vieW - trailing uppercase

using all-lowercase with no "/i"
type inputfile.txt |FindRepl "view" /a "displayed"

View - leading uppercase
displayed - all lowercase
VIEW - all uppercase
vieW - trailing uppercase

Press any key to continue . . .

result does not replace all case insensitive "view" to "displayed"

do you have script to search string with no case sensitive?

thanks

Bars

windows xp sp3 32bit user

[Aacini]

Well, your program does NOT execute any FindRepl with the /i switch, although the echo messages said so!

After I added the /i switches to the right commands (below the echo commands with the /i switch), I got this output:

Code: Select all

>test
using leading uppercase with "/i"
type inputfile.txt |FindRepl /i "View" /a "displayed"

displayed - leading uppercase
 - all lowercase
 - all uppercase
 - trailing uppercase

using leading uppercase with no "/i"
type inputfile.txt |FindRepl "View" /a "displayed"

displayed - leading uppercase
view - all lowercase
VIEW - all uppercase
vieW - trailing uppercase

using all-lowercase with "/i"
type inputfile.txt |FindRepl /i "view" /a "displayed"

 - leading uppercase
displayed - all lowercase
 - all uppercase
 - trailing uppercase

using all-lowercase with no "/i"
type inputfile.txt |FindRepl "view" /a "displayed"

View - leading uppercase
displayed - all lowercase
VIEW - all uppercase
vieW - trailing uppercase

Presione una tecla para continuar . . .

What happen? Well, the documentation explain this point clearly:

Please note that this multi-string replacement can only be achieved with case sensitive literal strings. If a search alternative match different input texts (i.e. if it is a regular expression), the result will be wrong; for example: FindRepl /I "One|Two|Three" /A "Uno|Dos|Tres". Previous example will replace "One" word by "Uno", but any other word that may be matched as result of the /I switch will be deleted, like "one", "onE", etc. You may use FindRepl "One|one|Two|two" /A "Uno|uno|Dos|dos" as a provisional fix, but the next version of the program will solve this problem.

If you explain what should be the expected output in each case, I could assemble the right version 2 regexp/JScript pair...

EDIT: For example, to replace all case insensitive possibilities by the same replacement string, use:

Code: Select all

      type inputfile.txt |FindRepl /i "view" /a "'displayed'" /J

Antonio

[bars143]

Aacini , its working with "/J" switch but i dont know that your machine produced blanks as compared to my windows xp sp3 32it.

its really easy to replace multiple string in one go rather using 3 or more pipes.

now i will start testing your scripts to be use in extracting webpage title , webpage series number , and other important string, then copy all and paste as part of a file name for easy querying.

i will soon ask if i have a further problem

thank for quick reply.

Bars