JREPL.BAT v8.6 - regex text processor with support for text highlighting and alternate character sets

[aGerman]

I see. To be honest I really expected to have ADO by default at least on XP and onwards because it was already introduced in the late 1990's and I never worked on a machine where it wasn't installed. I'll keep my eyes open. If I find something about its availability then I'll get back to you.

Steffen

[dbenham]

There may be confusion over ADO vs ADO.NET.

ADO.NET certainly is not guaranteed to be present, and I may have read something that confused the two, or I may have misinterpreted what I read.

I can't find anything that indicates ADO is not standard, even for XP Home. I see articles about it not working in some cases, but they seem to be about compatibility issues that are solved by standard updates.

My StackOverflow tester was on XP SP3 without .NET, and 7.13 worked just fine. But he did not specify Home or Pro.

I'm almost ready to pull the trigger and drop the CERTUTIL code, but not quite there.


Dave Benham

[aGerman]

ADO vs ADO.NET

I think you nailed it.
Finally you need to worry about upward compatibility rather than downward compatibility. Microsoft already tries forcing us to use .NET. Powershell is set as default console on Win 10. They want us to use C#.NET rather than C/Win32 API and thus, it's a logical consequence to have ADO.NET as successor of ADO ...

Steffen

[fdart17]

Hi dbenham,

Thank you very much for writing and maintaining this script.

Within the context of a cmd.exe in Windows 10, I'm calling a batch script which then invokes an exe to do a C++ build. I want to modify the output to the cmd prompt to color warnings in yellow and errors in red. This seems like a good application of this script. Is it possible to invoke jrepl in such a way that it will attach itself to the existing cmd prompt, running continuously to intercept incoming output and modify it before it is displayed to add ansi escape characters, while still allowing the user to interact with the existing cmd prompt to invoke more commands?

Thanks,
Justin

[Squashman]

Hi dbenham,

Thank you very much for writing and maintaining this script.

Within the context of a cmd.exe in Windows 10, I'm calling a batch script which then invokes an exe to do a C++ build. I want to modify the output to the cmd prompt to color warnings in yellow and errors in red. This seems like a good application of this script. Is it possible to invoke jrepl in such a way that it will attach itself to the existing cmd prompt, running continuously to intercept incoming output and modify it before it is displayed to add ansi escape characters, while still allowing the user to interact with the existing cmd prompt to invoke more commands?

Thanks,
Justin

Going to assume this is you: https://stackoverflow.com/questions/523 ... -add-color

[fdart17]

Yes, that is me.

I wanted to post an update here to not waste anyone's time. I've found another solution to the problem which happens to not leverage JREPL.BAT.
https://stackoverflow.com/questions/523 ... -add-color

It's still not as good as I would like, so I will keep searching for a better solution

[Michael.Uray]

I got the message

Code: Select all

JScript runtime error: Nicht genügend Arbeitsspeicher.

(means not enough RAM / memory)

when I did run a regex search/replace:

Code: Select all

JREPL.BAT "^\d+ 0 obj \n(?:(?!endobj $).*\n)*?\/Name \/Paragraph(?:.*\n)*?endobj ?$" "" /F input.pdf /M /O output.pdf

It happens when the cscript.exe process memor runs up to 3GB.

There is actually enough free RAM.
Any ideas?

[Squashman]

You are using the /M option. You are limited to 2GB of ram for 32 bit applications.

[Michael.Uray]

You are using the /M option. You are limited to 2GB of ram for 32 bit applications.

I need to parse multi line files like this https://regex101.com/r/fA6wE2/155 to search for objects which contain "Name /Paragraph".
The file has about 1.5 GB.

Any idea how to get around this issue?

[dbenham]

Unfortunately there is no way to circumvent the file size limit when using the /M option. I'm not sure the exact limit, but it is actually less than 1 GB.

You may still be able to use JREPL if you ditch the /M option and combine the /T option with /JMATCH and custom JScript. You don't state what your end goal is. The code below simply preserves object entries that contain a "/Name /Paragraph" line, and throws everything else away.

Code: Select all

@echo off
setlocal

set "init=var obj=false, keep=false, eol='\n'"

set "find1=^\d+ 0 obj\s*$"
set "repl1=obj=$0+eol;$txt=false;"

set "find2=^/Name /Paragraph\s*$"
set "repl2=if (obj){obj+=$0+eol;keep=true;}$txt=false;"

set "find3=^endobj\s*$"
set "repl3=$txt=keep?obj+$0:false;obj=keep=false;"

set "find4=^.*$"
set "repl4=if (obj){obj+=$0+eol;}$txt=false;"

set "find=%find1%|%find2%|%find3%|%find4%"
set "repl=%repl1%|%repl2%|%repl3%|%repl4%"

call jrepl find repl /t "|" /jmatchq /jbeg init /v /u /f input.pdf /o output.pdf

exit /b

Dave Benham

[dbenham]

Upon looking closer at your original code, it looks like you are trying to remove all the objects that contain "/Name /Paragraph".

Code: Select all

@echo off
setlocal

set "init=var obj='', keep=true, eol='\n'"

set "find1=^\d+ 0 obj\s*$"
set "repl1=obj=$0+eol;$txt=false;"

set "find2=^/Name /Paragraph\s*$"
set "repl2=if (obj)keep=$txt=false; else $txt=$0;"

set "find3=^endobj\s*$"
set "repl3=$txt=keep?obj+$0:false;obj='';keep=true;"

set "find4=^.*$"
set "repl4=if (obj){$txt=false;if (keep)obj+=$0+eol;}else $txt=$0;"

set "find=%find1%|%find2%|%find3%|%find4%"
set "repl=%repl1%|%repl2%|%repl3%|%repl4%"

call jrepl find repl /t "|" /jmatchq /jbeg init /v /u /f input.pdf /o output.pdf

exit /b

Note - It would probably be just as easy to write a custom JScript script as it is to use JREPL as I have done. And custom JScript would probably be a bit faster.


Dave Benham

[Michael.Uray]

Upon looking closer at your original code, it looks like you are trying to remove all the objects that contain "/Name /Paragraph".

This is correct.

I had to replace the variable assignment to the command to run it, but it does not create the expected output.

Code: Select all

call jrepl "%find%" "%repl%" /t "|" /jmatchq /jbeg init /v /u /f input.pdf /o output.pdf

The output file (71MB) gets much larger than the input file (7MB) for some reason.

[dbenham]

I had to replace the variable assignment to the command to run it, but it does not create the expected output.

Code: Select all

call jrepl "%find%" "%repl%" /t "|" /jmatchq /jbeg init /v /u /f input.pdf /o output.pdf

The output file (71MB) gets much larger than the input file (7MB) for some reason.

No, the find and repl must not be enclosed by percents in the JREPL call.

I tested the code against the input that you provided in the link, and it worked perfectly for me. But then I added some additional lines at the top of the input that were not contained in an "object", and it began behaving badly. I did some testing, and it appears there is a newly discovered bug - there are internal str and obj objects within the code that interfere with the user supplied JScript if /JBEG defines a variable named str or obj.

I was able to get the code to work properly if I changed obj to Obj:

Code: Select all

@echo off
setlocal

set "init=var Obj='', keep=true, eol='\n';"

set "find1=^\d+ 0 obj\s*$"
set "repl1=Obj=$0+eol;$txt=false;"

set "find2=^/Name /Paragraph\s*$"
set "repl2=if (Obj)keep=$txt=false; else $txt=$0;"

set "find3=^endobj\s*$"
set "repl3=$txt=keep?Obj+$0:false;Obj='';keep=true;"

set "find4=^.*$"
set "repl4=if (Obj!==''){$txt=false;if (keep)Obj+=$0+eol;}else $txt=$0;"

set "find=%find1%|%find2%|%find3%|%find4%"
set "repl=%repl1%|%repl2%|%repl3%|%repl4%"

call jrepl find repl /t "|" /jmatchq /jbeg init /v /u /f input.pdf /o output.pdf

exit /b

Update - I fixed the bug and released version 7.14. The original code using obj should now work.

Dave Benham

[dbenham]

Here is version 7.14

JREPL7.14.zip

Downloaded 75 times over 5 days from the main release page while v7.14 was the current version.

(27.24 KiB) Downloaded 1180 times

This is simply a bug fix to allow user defined variables named str and/or obj, and to hide the internal xbytes variable from user code.

Code: Select all

    2018-10-15 v7.14: Bug fix - User defined variables declared in /JBEG named
                      str and/or obj were getting clobbered.
                      Bug fix - Internal variable xbytes was visibile to user
                      supplied JScript.

Dave Benham

[Michael.Uray]

I had to replace the variable assignment to the command to run it, but it does not create the expected output.

Code: Select all

call jrepl "%find%" "%repl%" /t "|" /jmatchq /jbeg init /v /u /f input.pdf /o output.pdf

No, the find and repl must not be enclosed by percents in the JREPL call.

Hmm, I had to add that again in the .cmd file.
"find" and "repl" are envirnment variables so I have to use them with "%" on the beginn and on the end, right?
I have to also put it under quotes because the string (^\d+ 0 obj\s*$|^/Name /Paragraph\s*$|^endobj\s*$|^.*$) contains empty spaces, right?

The output is still a 70 MB file, maybe I should send you the input file via PM that you are able to recreate the problem.