JREPL.BAT v8.6 - regex text processor with support for text highlighting and alternate character sets

[dbenham]

Hello, I've been using JREPL to encode/decode game data, (I've made an online batch game, but that's beside the point) and I have ran into a problem. People are able to intercept the calling parameters and therefore get my encode/decode table.

Is there any possible way I can copy/paste JREPL into my game code, and then call a subroutine for it within the batch file whenever I need to use it? That way people can't intercept the parameters when calling JREPL?

I don't like that concept for a number of reasons.

• JREPL is already quite large for a script - adding it to a complete game script will only increase the issue.
• Putting anything in batch is inherently insecure. Sure, you can obfuscate text within a script, but anyone who really wants to crack the code can easily do so any number of ways, no matter what you do. It should be enough to use minimal encryption (obfuscation) to discourage casual cheating. Anything beyond that seems pointless to me. That is the strategy I use with my batch port of Colossal Cave Adventure. And that was easily done with pure batch - no need for a complicated regular expression text processing utility.
• If you are going to use JScript as part of your game, (even if it is embedded within an encryption/decryption utility) you might as well simply write the entire game in JScript. You can easily provide a much richer game experience with a lot less effort. The only reason I used hybrid JScript/batch with JREPL is because I wanted powerful and convenient regular expression text processing for command line and batch environments, and CSCRIPT scripts are inconvenient to call from the command line or a batch script. The batch portion is simply devoted to providing a convenient batch interface. I can't see any such benefit with a self contained game.
• Something just feels wrong about embedding JREPL within some other program - sorry. I don't mind distribution of the complete, stand-alone JREPL.BAT script as part of some larger collection of files, but I prefer to keep the files separate.

Dave Benham

[Mordru]

Thanks for the quick reply... The reason I coded the game in batch is because I mainly work on it at work in my frequent spare time, and notepad is the only program I can use. JREPL is used to encode/decode gamesave data. My batch file is compiled to exe using "Advanced batch file converter" or something along those lines. It does not just place the .bat file in a temp folder and call it, so I am not worried about the bat source code being read.

Do you know any other solution then if I can not call JREPL within the same batch file?

[floppyh4x]

I am trying to convert several tab delimited files to commas. I have attached some examples that need to be converted into the following:

Example:

Code: Select all

1122.777.434  M.Q. TER HORST                           10    20     03     AFV    2018-06-30 22   TOETS HOOFD

this needs to look like this:

Code: Select all

1122.777.434,M.Q. TER HORST,10,20,03,AFV,2010-06-30,22,TOETS HOOFD

in the numbers, the dots need to remain the same, in the customer name, no conversion is needed to comma's. At the end its no problem if theres a comma or a space in between. I have 10.000 of these exactly the same records, different content. I failed to use JREPL because I dont know if those are spaces or tabs. Whenever im selecting white space it looks like spaces. Is there anyone out there willing to help me out of this situation?

Here are more examples of how it could look like:

Code: Select all

    1178.916.332  ZILVEREN PENSIOEN- EN LEVE LANG                40    20     40     SOS    2018-06-30 19   EXPIRATIE GEMENGDE VERZEKERING                                 
                                                                                                                                                                
    1178.917.657  J. HUPOIRN ESO                            40    20     50     SOH    2018-06-00 18   EINDE HOOFD TWEE
                                                                                                                                                                
    1178.928.446  J.J.K.P. JANSEN                           40    32     02     AFV    2018-06-00 22   TOETS HOOFDEN
                                                                                                                                                                    
    1178.928.756  J.G.O. JANSENS                           40    32     02     AFV    2018-06-00 22   TOETS HOOFD     

[Jtpo1983]

Hi there,

A big thank you to the developer for all your work on JREPL.

I am having trouble with the pre and post context for /K

Looking at a previous example, can you tell me how I would use pre and post in order to return x number of lines before and after matches?

JREPL search.txt "" /K 0:file /F input.txt > output.txt

I've tried a large number of variations but I haven't had any results returns (whereas the above example does return the line containing the match - which is awesome BTW).

Thank you in advance

[amitabhob]

Thanks to you all, the Developer for this impressive utility, and everyone of you, for your constant help & and valuable inputs.

I am very novice in the scripting area, and came across this forum while trying to find a solution for one requirement I have. Advance thanks to you all for your time and any help.

I have a big 300-400mb file, with contents something similar to, what is shown below.
The need is:
To replace, Only the 1st 3 characters i.e "ABC" with "00XXXX00-" (excluding quotes), in every line, but, only when, the first 15 chars begin with ABC and ends with DEF.
If this pattern exists somewhere else in the same line, then, no replacement to be done on the other patterns found.

For example: sampleTest.txt file has below content:
ABC123456789DEFAAAB1R$R36ABC123456789DEF-0AAAB123AAAB12EF1TRYU
ABC1234567891234DEF--0AAAB123A124561AA2AEF34DEF111AATRE&*90
ABC671243598DEF34DEF111AAAB12EF111AAAB12#45*999SAHTYUP-0RRLKU

Expected output of the new file resultFile.txt is:
00XXXX00-123456789DEFAAAB1R$R36ABC123456789DEF-0AAAB123AAAB12EF1TRYU
ABC1234567891234DEF--0AAAB123A124561AA2AEF34DEF111AATRE&*90
00XXXX00-671243598DEF34DEF111AAAB12EF111AAAB12#45*999SAHTYUP-0RRLKU

Note: on 1st line, there were 2 matching patterns, but replacement is done only on the 1st pattern on that line.

I was able to construct the command upto this:
type "sampleTest.txt"|jrepl "ABC.........DEF" "00XXXX00-" /B > "resultFile.txt"

With this, I am able to limit search to 1st 15 chars, but, this command will also replace all 15 chars, not just the 1st 3 Chars ABC, which is what I want to fix.

If you can please help, with the right command. Thanks again.

[jpeni]

Is there a collection of example commands and scripts for JREPL.BAT?

I can find many individual examples of JREPL.BAT and many suggested solutions to
search and replace questions, but no combined collections that would make it easier
to understand the many options and functions of JREPL.BAT.

[amitabhob]

Thanks to you all, the Developer for this impressive utility, and everyone of you, for your constant help & and valuable inputs.

I am very novice in the scripting area, and came across this forum while trying to find a solution for one requirement I have. Advance thanks to you all for your time and any help.

I have a big 300-400mb file, with contents something similar to, what is shown below.
The need is:
To replace, Only the 1st 3 characters i.e "ABC" with "00XXXX00-" (excluding quotes), in every line, but, only when, the first 15 chars begin with ABC and ends with DEF.
If this pattern exists somewhere else in the same line, then, no replacement to be done on the other patterns found.

For example: sampleTest.txt file has below content:
ABC123456789DEFAAAB1R$R36ABC123456789DEF-0AAAB123AAAB12EF1TRYU
ABC1234567891234DEF--0AAAB123A124561AA2AEF34DEF111AATRE&*90
ABC671243598DEF34DEF111AAAB12EF111AAAB12#45*999SAHTYUP-0RRLKU

Expected output of the new file resultFile.txt is:
00XXXX00-123456789DEFAAAB1R$R36ABC123456789DEF-0AAAB123AAAB12EF1TRYU
ABC1234567891234DEF--0AAAB123A124561AA2AEF34DEF111AATRE&*90
00XXXX00-671243598DEF34DEF111AAAB12EF111AAAB12#45*999SAHTYUP-0RRLKU

Note: on 1st line, there were 2 matching patterns, but replacement is done only on the 1st pattern on that line.

I was able to construct the command upto this:
type "sampleTest.txt"|jrepl "ABC.........DEF" "00XXXX00-" /B > "resultFile.txt"

With this, I am able to limit search to 1st 15 chars, but, this command will also replace all 15 chars, not just the 1st 3 Chars ABC, which is what I want to fix.

If you can please help, with the right command. Thanks again.

Seems like, I was able to find the solution for this. This is what I have done:
jrepl "^ABC(.{8}.DEF.*)" "'00XXXX00-'+$1" /j /f sampleTest.txt /o resultFile.txt

This seems to be serving my purpose. Thanks again !!

[mrshiseido]

Hi Dave,

I would like to make use of the JREPL to search for a series of strings (PC mac address, seperated by .) on each line in a text file throughout a list of data text files (4Mb) under particular folder (i.e. c:\test), count the total no of matches for each string once from those data text files, then write the total count for each string in a separate text file, and if possible, put the total count next to the related string that have been searched in the output text file.

I am new to dos batch & script and been trying my best to look around for the best tools, I came across JREPL & want to make use of it. Your assistance & prompt feedback will be highly appreciated. Thanks in advance.

[dbenham]

Here is version 7.13

JREPL7.13.zip

Downloaded 760 times in 3 months from the main release post while v7.13 was the current release

(27.12 KiB) Downloaded 873 times

Summary of changes

Code: Select all

prompt>jrepl /?history

    2018-07-19 v7.13: Bug fix - /INC and /EXC regex failed to match any
                      line that immediately followed a prior block.
                      Added ADO code to create XBYTES.DAT in case CERTUTIL is
                      missing.
    2018-07-18 v7.12: Fixed XBYTES.DAT creation cleanup bugs, and improved docs
 <truncated...>

1) /INC and /EXC bug fix

A regular expression /INC or /EXC block would fail to match a line if it immediately followed a prior block. As a trivial example, the /INC "/^/" option should specify a line block that matches all lines, but the bug was causing every other line to be skipped.

I'm shocked and embarrassed that this nasty bug was not detected sooner. It was easy to fix by simply moving a couple lines of code.

2) Added ADO code to create XBYTES.DAT in case CERTUTIL is missing

The /XSEQ option utilizes an XBYTES.DAT file to properly map a given \xnn escape sequence to the correct Unicode code point that generates the desired byte value, depending on the character set. JREPL automatically tries to create the XBYTES.DAT using CERTUTIL, but some XP machines do not have CERTUTIL.

Version 7.13 adds the ability to create the file using ADO when CERTUTIL is missing, thanks to some nice code provided by Steffen (aGerman)

It is possible that an XP machine may be missing both CERTUTIL and ADO, in which case XBYTES.DAT still cannot be created, and \xnn defaults to the Windows-1252 character set for its map.

3) Fixed XBYTES.DAT creation cleanup

During creation of XBYTES.DAT, Versions prior to v7.12 were sometimes leaving behind an intermediate XBYTES.HEX file that should have been deleted. Version 7.12 fixed this minor bug.

Note that the bug fix will not remove any already existing XBYTES.HEX that was left behind by an earlier version. But the file is small, out of the way, and harmless.


Dave Benham

[aGerman]

Would that be a way to work around a missing certutil.exe?

*.js

Code: Select all

var Stream=WScript.CreateObject('ADODB.Stream'),
    Node=WScript.CreateObject('Microsoft.XMLDOM').createElement('e');
Node.dataType='bin.base64';
Node.text='AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4v'
+ 'MDEyMzQ1Njc4OTo7PD0+P0BBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWltcXV5f'
+ 'YGFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6e3x9fn+AgYKDhIWGh4iJiouMjY6P'
+ 'kJGSk5SVlpeYmZqbnJ2en6ChoqOkpaanqKmqq6ytrq+wsbKztLW2t7i5uru8vb6/'
+ 'wMHCw8TFxsfIycrLzM3Oz9DR0tPU1dbX2Nna29zd3t/g4eLj5OXm5+jp6uvs7e7v'
+ '8PHy8/T19vf4+fr7/P3+/w==';
Stream.Type=1;
Stream.Open();
Stream.Write(Node.nodeTypedValue);
Stream.SaveToFile('all.bin');

Steffen

[dbenham]

That works great on my Win 10 machine. I like the code

But I have no way to test on XP. I suppose it depends on what ADO versions it works on.

I didn't think ADO was standard on any version of Windows, let alone XP. So I think there still could be some XP machines where it could fail.

I'm not overly concerned if some XP users do not have access to the XBYTES.DAT functionality. The XBYTES.DAT is only needed when working with escape sequences for byte codes greater than 127, and Microsoft dropped support for XP years ago.

But if someone can verify that your code works on XP, and reassure me that ado is most likely going to be present on XP, then I think the code would be a great addition.


Dave Benham

[aGerman]

I can't verify that it works on XP but I remember that I used ADO streams on XP back then. The stream object was introduced with ADO 2.5 (Windows 2000 already).
https://docs.microsoft.com/en-us/sql/ad ... erver-2017

Steffen

[dbenham]

Great news

I found a StackOverflow user that was able to confirm that his XP machine that was missing CERTUTIL was able to run the ADO code to create XBYTES.DAT.

So I incorporated the code into JREPL and updated the prior release post to v7.13.

I also squashed an unrelated nasty bug concerning /INC and /EXC.

Thanks for your help Steffen.


Dave Benham

[aGerman]

You're wellcome Dave

I still have a suggestion (and sorry that I didn't mention that in the first place)...
Now that you know it works on XP while you also know that certutil might not be available - why not only use the code that will work in each case?
Furthermore: You're struggling to find the right place where to save the file. Then you write a temporary file with hex codes that you need for certutil. Then you need certutil to create the binary file. Then you have to remove the temporary file. Then you have to read the file into the stream object... That sounds all quite complicated and eventually might waste too much time.
Using just the few lines of JScript you already have the binary content in a stream object. Just keep it and use it whenever you need. What do you think, is it still worth to write the XBYTES.DAT file for future use?

Steffen

[dbenham]

Yes, I thought about that. But I was influenced by my memory of reading something about ADO not being native to any base install of Windows, regardless of version. As far as I know, it is possible to be missing from any version of Windows. That being said, I think it is available to most Windows installs. Contrast that with CERTUTIL that is native to all versions of Windows except XP, which is no longer supported. So I decided it is better to preserve both techniques to maximize the likelihood of creation of the file.

As long as XBYTES.DAT has been created once, the code to create it is never executed again, so I'm not too worried about the delay. And the CERTUTIL code is pretty fast any way, though certainly not as fast as the ADO. The only thing that concerns me is the rare machine where neither is available, and the short delay is present every time you run JREPL. But I don't think I should worry too much about operating systems that have long ago lost MS support. At least the utility runs cleanly there, albeit a bit slower.

If I see some documentation that ADO can be expected to be on all machines, then I will probably remove the CERTUTIL code.


Dave