A better way to build an HTML Index of files?

[dbenham]

I added % % prior to every line except the following:

Code: Select all

    ) DO (

Can you see if prefixing the above line with % % fixes the XP bug? That very well may be the problem.

Dave Benham

[Acy Forsythe]

Sadly... I tried both the % % and the un-indent methods and they both resulted in the bug.

But I took this a step further and got to a point where the bug does not occurr....

I removed each line from the top down and ran the script.

Here is what I ended up with:

Code: Select all

(
    FOR /F "tokens=*"%%a in ("
%  %"<td><font color=#ffffff><b>Email Address</b></font></td>"%NL%
%  %"</tr>"
%  %) DO (
    ECHO(%%~a
    )
) >> "%WORKDIR%\%NEWDIR%\index.htm"

Now... you should be asking yourself the same thing I asked myself... "WHAT?!?!?"

The last two lines I removed were almost Identical to that next line. But I ran this 30 times and not once did it generate the error.

So freaking out, I added the next line back in... IMMEDIATE error. Again and Again. Take it out, No error.

Now it's getting wierd, so instead of putting the old line back in, I just copied the existing line so now I have two duplicate working lines and guess what? I got the error anyway proving that the content of the line has nothing to do with it.

But in all the testing I did notice something... There file that cannot be found had several control characters and part of an e-mail address that had been stored in %EMAIL% earlier.

Now why that's relevant is... I removed every bit of my code except what was needed to output to HTML, IE the path variable and my setlocal and the code above and that's IT. Also, after running the code, I check SET and none of my variables are there.

So I close the command prompt, open a new one and get some new results... Predictable, repeatable results. Still the bug, but the file it cannot find is cycling through the memory of the current command prompt and also proves that even Endlocal getting rid of variables used in memory and clearing out set, that memory is still there being used and holding past values.

[dbenham]

Sadly... I tried both the % % and the un-indent methods and they both resulted in the bug.

Darn

I have a confession to make - I steered you wrong earlier:

- You could indent with only one space. This will work because the %NL% macro will escape the leading space character on the next line. You don't want the lines to start with the quote because then the quote will be escaped and that will lead to complications.

The above is garbage The %NL% does not end with a caret so it does not escape the 1st character on the next line. The whole line should be moved all the way to the left.

But your experiments with % % show that this is still not working.

I'm glad the simple FOR loop works (as it should!)

Dave Benham

[Acy Forsythe]

Arrrghhh...

A Co-Worker asked what I was doing and as I was explaining it to them, I started getting the bug even with only using the last two lines, which previously and consistently was avoiding the bug...

[dbenham]

I have a theory

jeb - if you are there I think you will be interested in this

If we run code with ECHO ON then we see what the parser sees and it helps to diagnose what is happening.

I modified the code by prefixing some of the lines with various token delimiters and moving other lines all the way to the left. I also modified it to run with echo on.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set LF=^


set NL=^^^%LF%%LF%^%LF%%LF%
cls
echo on
for /F "tokens=*" %%a in ("
"-------------------------"%NL%
,,,"</table> </center> <tr>"%NL%
;;;"<td valign=bottom bgcolor=#ffffff>"%NL%
==="<table border=0 width=100%% cellpadding=0 cellspacing=0>"%NL%
,,,"<tr>"%NL%
==="<td bgcolor=#123759 height=40>"%NL%
;;;"<table border=0 width=100%% cellpadding=0 cellspacing=0>"%NL%
   "<tr>"%NL%
   "<td width=10>&nbsp;</td>"%NL%
"<td valign=center><font color=#ffffff><b>Email Index</b></font></td>"%NL%
"<td align=right><font color=#ffffff>&copy; Site_Name, Inc.</font><br>"%NL%
"<td width=10>&nbsp;</td>"%NL%
"</tr> </table> </td> </tr> </table> </td> </tr> </table>"%NL%
"</body>"%NL%
""%NL%
"</html>"
   ) DO @echo(%%~a
)

Here is the outupt. Note that I have prefixed each output line with : because the forum flushes everything left if a line leads with a single space.

Code: Select all

:D:\utils>for /F "tokens=*" %a in (" "-------------------------"
: "</table> </center> <tr>"
: "<td valign=bottom bgcolor=#ffffff>"
: "<table border=0 width=100% cellpadding=0 cellspacing=0>"
: "<tr>"
: "<td bgcolor=#123759 height=40>"
: "<table border=0 width=100% cellpadding=0 cellspacing=0>"
: "<tr>"
: "<td width=10>&nbsp;</td>"
: "<td valign=center><font color=#ffffff><b>Email Index</b></font></td>"
: "<td align=right><font color=#ffffff>&copy; Site_Name, Inc.</font><br>"
: "<td width=10>&nbsp;</td>"
: "</tr> </table> </td> </tr> </table> </td> </tr> </table>"
: "</body>"
: ""
: "</html>") DO @echo(%~a
:-------------------------
:</table> </center> <tr>
:<td valign=bottom bgcolor=#ffffff>
:<table border=0 width=100% cellpadding=0 cellspacing=0>
:<tr>
:<td bgcolor=#123759 height=40>
:<table border=0 width=100% cellpadding=0 cellspacing=0>
:<tr>
:<td width=10>&nbsp;</td>
:<td valign=center><font color=#ffffff><b>Email Index</b></font></td>
:<td align=right><font color=#ffffff>&copy; Site_Name, Inc.</font><br>
:<td width=10>&nbsp;</td>
:</tr> </table> </td> </tr> </table> </td> </tr> </table>
:</body>
:
:</html>

All of the echoed lines of code within the IN() clause are prefixed with a single space. The multiple token delimiters are condensed into a single space. But more importantly, the lines that started out flush left are indented 1 space. Why?

The clue is the very 1st line that is not preceded by %NL% but is preceded by a <LF>... It follows the initial (" after an intervening space. I think the <LF> is functioning as a token delimiter the same as any other token delimiter, and it is condensed into a space. It is also unescaped and unquoted, so it can lead to the XP bug.

If I am correct, then jeb's FOR /F technique of iterating across multiple strings is INCOMPATIBLE with XP.
Unless jeb can figure out how to escape the <LF> while still treating each line seperately.
I haven't been able to make it work.

Dave Benham

[dbenham]

Arrrghhh...

A Co-Worker asked what I was doing and as I was explaining it to them, I started getting the bug even with only using the last two lines, which previously and consistently was avoiding the bug...

The bug is indeed nasty and appears randomly depending on the internal state of the machine.

I believe the only thing that is not random is that it will not appear if all token delimiters are escaped or quoted. The trick for your case is identifying the offending token delimiter.

I'm hoping my theory in my prior post concerning <LF> is correct.

Dave Benham

[Ed Dyreen]

'

The clue is the very 1st line that is not preceded by %NL% but is preceded by a <LF>... It follows the initial (" after an intervening space. I think the <LF> is functioning as a token delimiter the same as any other token delimiter, and it is condensed into a space. It is also unescaped and unquoted, so it can lead to the XP bug.

I also wondered what that leading space was doing there, corrupting my batch

Post subject: set variable=set variable=was not expected at this time
viewtopic.php?f=3&t=2060&p=9253&hilit=+underscore#p9253

It would be very easy though, only if I knew where this leading space is coming from

Code: Select all

prompt $

@echo on
set FileOpen=(%$n1c%
set RULE=!$ED.Path!^^^^^^^^%$n1c%
some%$n1c%
)

output:

Code: Select all

set FileOpen=(
set RULE=!$ED.Path!^^^^
some
)

set FileOpen
FileOpen=(
set RULE=somepath^^
some
)

(
set RULE=somepath^
_some
)
some wordt niet herkend als een interne
of externe opdracht, programma of batchbestand.

set RULE
RULE=somepath^

echo.endoftest
endoftest

pause
Druk op een toets om door te gaan. . .

The underscore represents a leading space.

I believe the only thing that is not random is that it will not appear if all token delimiters are escaped or quoted. The trick for your case is identifying the offending token delimiter.

Or cheat

Code: Select all

2>nul ( for something in some ,,, offending token do set something ) &for something in "something" do ...

[Acy Forsythe]

I wonder if quoting the LF character would fix anything... I don't have time to continue messing with it right now, but I will run some more tests this evening or tomorrow morning.

[jeb]

:idea: I have a theory

jeb - if you are there I think you will be interested in this

Yes, I'm always interested in new theories

And, yes it seems to be produce the XP delimiter bug.

If I am correct, then jeb's FOR /F technique of iterating across multiple strings is INCOMPATIBLE with XP.
Unless jeb can figure out how to escape the <LF> while still treating each line seperately.
I haven't been able to make it work.

It's not only the first <LF>, each line produces the same problem.

But I suppose, I got the (obvious) solution

@echo off
setlocal EnableDelayedExpansion
set LF=^


set ^"NL=^^^%LF%%LF%^%LF%%LF%"
set "space= "

cls
echo onf
:loop
for /F "tokens=*" %%a in (^""One"%NL%^
%space%"two"%NL%^
%space%"three"%NL%^
%space%"four"^
") DO (
@echo('%%a' --- '%%~a'
)
goto :loop

The loop is only to better examine the XP-Bug.
The %space% is to see where the space has to be (else you can't see beginning spaces in posts).
If you add more than one space at the beginning, or remove one of the carets you got the bug.

Why it works now?
I suppress any "free" delimiter, the remaining spaces are escaped, by the rule that a caret at the end of line starts a multiline and the first character of the next line will be escaped.

EDIT:
And a variation of the code, I prefer this, as it looks nicer

Code: Select all

@echo off
setlocal EnableDelayedExpansion
set LF=^


set ^"NL=^^^%LF%%LF%^%LF%%LF%^^"
cls

for /F "tokens=* delims=_" %%a in (^"%NL%
___"One"%NL%
___"two"%NL%
___"three"%NL%
___"four"%NL%
") DO (
   @echo('%%a' --- '%%~a'
)

I move the line end caret into the %NL% and use ___ as delim instead of spaces.
And now all lines looks the same way.

hope it helps
jeb

[dbenham]

Achh -
I tried putting the caret at the end, but I was hung up on using "delims=" with no leading characters.
Thanks for pointing out the "obvious" solution jeb.

If I am correct, then jeb's FOR /F technique of iterating across multiple strings is INCOMPATIBLE with XP.
Unless jeb can figure out how to escape the <LF> while still treating each line seperately.
I haven't been able to make it work.

It's not only the first <LF>, each line produces the same problem.

Yes, I realized that. But at first I thought the spaces were a result of how the parser formats multiple lines within a () code block. But then I noticed that the 1st line is on the same line as (" with an intervening space. I then realized this was an entirely different mechanism - the normal <LF> characters (those not in the %NL% macro) were being converted to spaces by the parser.

Dave Benham