Page 1 of 1
case study of the 'for %~sk' bug in win7
Posted: 10 Feb 2013 01:15
by Liviu
Background: there was a longstanding xp bug where %~sk could return a wrong/invalid SFN in a for loop. Earliest reference I could locate is foxidrive's usenet post at
https://groups.google.com/group/alt.msdos.batch.nt/msg/254df0764b2bbae8?hl=en. As my luck usually goes, I had to rediscover the bug on my own years later. In the ensuing discussion, it turned out that the original xp bug had been fixed in vista and later. However, a related (but likely different) bug still exists in Win7. Below is a somewhat simpler test case than the one I posted at
http://groups.google.com/group/alt.msdos.batch.nt/msg/85379bd32ede38b6?hl=en though it still requires at least one character outside the current codepage to trigger it.
Assuming the following directory structure exists under the current directory, and for-sk-w7.cmd is...
Code: Select all
@for /d %%d in (*) do dir /x "%%~d"
for /r %%k in (*) do @if not exist "%%~sk" echo "%%~sk" ---???--- "%%~fk"
...the output under Win7 x64 is the following.
EDIT: marked the files with an incorrect %~sk with
***.
Code: Select all
C:\tmp>for-sk-w7
C:\tmp>dir /x "19 €"
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234
C:\tmp>dir /x "20 €"
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1 ***
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234 ***
C:\tmp>dir /x "21 €"
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12 ***
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123 ***
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234
C:\tmp>dir /x "22 €"
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234
C:\tmp>for /R %k in (*) do @if not exist "%~sk" echo "%~sk" ---???--- "%~fk"
"C:\tmp\202075~1\12345_~1.1 \12345=7.1" ---???--- "C:\tmp\20 €\12345=7.1"
"C:\tmp\202075~1\12_4~1.123 \12=4.1234" ---???--- "C:\tmp\20 €\12=4.1234"
"C:\tmp\21716B~1\1234_6~1.12 \1234=6.12" ---???--- "C:\tmp\21 €\1234=6.12"
"C:\tmp\21716B~1\123_5~1.123 \123=5.123" ---???--- "C:\tmp\21 €\123=5.123"
C:\tmp>
What displays as an extra space in the directory part of the failing pathnames above is actually a U+2000 "en quad" space. Don't see any pattern, nor have much of a clue as to what kind of bug could possibly cause it. Just thought I'd mention it here for the cmd fun collectors
Liviu
Re: case study of the 'for %~sk' bug in win7
Posted: 10 Feb 2013 02:01
by foxidrive
Another thread a couple of years earlier.
https://groups.google.com/d/topic/alt.m ... discussionI found that using command.com helped but at the expense of a different issue.
Re: case study of the 'for %~sk' bug in win7
Posted: 11 Feb 2013 00:24
by Liviu
Interesting, thanks for the pointer. Another link in that thread
http://groups.google.com/group/alt.msdos.batch.nt/msg/32a14ae26b10e5cf dates the first XP bug report back to 2002, with a hint that it existed since the original NT.
Off the stated (Win7) topic now, the XP %~sk bug was arguably different, and definitely easier to hit, than the one still left in Win7. What they have in common is that among the necessary trigger conditions are a LFN directory name with a SFN strictly shorter, and a LFN file name with a SFN strictly longer. However, the additional dependency on "unicode" characters in the path applies to the Win7 bug, only. That said, the bug is not new to Win7 and existed in XP all along, just that it was obscured by the old non-Unicode-dependent bug. For the record...
Code: Select all
@dir /s /x
@echo. & for /d %%d in (*) do @for %%k in ("%%d\*") do @if not exist "%%~sk" echo "%%~sk" ---???--- "%%~fk"
@echo. & for /r %%k in (*) do @if not exist "%%~sk" echo "%%~sk" ---???--- "%%~fk"
...gives under XP Sp3 --
EDIT: marked the files with an incorrect %~sk with
***.
Code: Select all
C:\tmp>for-sk-xp
Directory of C:\tmp\1 3 9
05/29/2010 02:51 PM 0 120F6A~1.ABC 12.abcd
05/29/2010 02:51 PM 0 123~1.ABC 123.abcd ***
05/29/2010 02:51 PM 0 1234~1.ABC 1234.abcd ***
05/29/2010 02:51 PM 0 12345~1.ABC 12345.abcd ***
05/29/2010 02:51 PM 0 123456~1.ABC 123456.abcd ***
05/29/2010 02:51 PM 0 123456~2.ABC 1234567.abcd
05/29/2010 02:51 PM 0 123456~3.ABC 12345678.abcd
Directory of C:\tmp\1 3 8
05/29/2010 02:51 PM 0 120F6A~1.ABC 12.abcd
05/29/2010 02:51 PM 0 123~1.ABC 123.abcd ***
05/29/2010 02:51 PM 0 1234~1.ABC 1234.abcd ***
05/29/2010 02:51 PM 0 12345~1.ABC 12345.abcd ***
05/29/2010 02:51 PM 0 123456~1.ABC 123456.abcd ***
05/29/2010 02:51 PM 0 123456~2.ABC 1234567.abcd
05/29/2010 02:51 PM 0 123456~3.ABC 12345678.abcd
Directory of C:\tmp\1 3 7
05/29/2010 02:51 PM 0 120F6A~1.ABC 12.abcd
05/29/2010 02:51 PM 0 123~1.ABC 123.abcd ***
05/29/2010 02:51 PM 0 1234~1.ABC 1234.abcd ***
05/29/2010 02:51 PM 0 12345~1.ABC 12345.abcd ***
05/29/2010 02:51 PM 0 123456~1.ABC 123456.abcd ***
05/29/2010 02:51 PM 0 123456~2.ABC 1234567.abcd
05/29/2010 02:51 PM 0 123456~3.ABC 12345678.abcd
Directory of C:\tmp\1 3 6
05/29/2010 02:51 PM 0 120F6A~1.ABC 12.abcd
05/29/2010 02:51 PM 0 123~1.ABC 123.abcd
05/29/2010 02:51 PM 0 1234~1.ABC 1234.abcd
05/29/2010 02:51 PM 0 12345~1.ABC 12345.abcd
05/29/2010 02:51 PM 0 123456~1.ABC 123456.abcd
05/29/2010 02:51 PM 0 123456~2.ABC 1234567.abcd
05/29/2010 02:51 PM 0 123456~3.ABC 12345678.abcd
Directory of C:\tmp\19 €
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1 ***
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12 ***
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123 ***
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234 ***
Directory of C:\tmp\20 €
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1 ***
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12 ***
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123 ***
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234 ***
Directory of C:\tmp\21 €
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1 ***
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12 ***
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123 ***
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234 ***
Directory of C:\tmp\22 €
02/09/2013 11:39 PM 0 12345_~1.1 12345=7.1 ***
02/09/2013 11:39 PM 0 1234_6~1.12 1234=6.12 ***
02/09/2013 11:39 PM 0 123_5~1.123 123=5.123 ***
02/09/2013 11:39 PM 0 12_4~1.123 12=4.1234 ***
"C:\tmp\139~1\123~1.ABCbcd" ---???--- "C:\tmp\1 3 9\123.abcd"
"C:\tmp\139~1\1234~1.ABCbcd" ---???--- "C:\tmp\1 3 9\1234.abcd"
"C:\tmp\139~1\12345~1.ABCbcd" ---???--- "C:\tmp\1 3 9\12345.abcd"
"C:\tmp\139~1\123456~1.ABCbcd" ---???--- "C:\tmp\1 3 9\123456.abcd"
"C:\tmp\138~1\123~1.ABCcd" ---???--- "C:\tmp\1 3 8\123.abcd"
"C:\tmp\138~1\1234~1.ABCcd" ---???--- "C:\tmp\1 3 8\1234.abcd"
"C:\tmp\138~1\12345~1.ABCcd" ---???--- "C:\tmp\1 3 8\12345.abcd"
"C:\tmp\138~1\123456~1.ABCcd" ---???--- "C:\tmp\1 3 8\123456.abcd"
"C:\tmp\137~1\123~1.ABCd" ---???--- "C:\tmp\1 3 7\123.abcd"
"C:\tmp\137~1\1234~1.ABCd" ---???--- "C:\tmp\1 3 7\1234.abcd"
"C:\tmp\137~1\12345~1.ABCd" ---???--- "C:\tmp\1 3 7\12345.abcd"
"C:\tmp\137~1\123456~1.ABCd" ---???--- "C:\tmp\1 3 7\123456.abcd"
"C:\tmp\19898B~1\12345_~1.1\12345=7.1" ---???--- "C:\tmp\19 €\12345=7.1"
"C:\tmp\19898B~1\1234_6~1.121234=6.12" ---???--- "C:\tmp\19 €\1234=6.12"
"C:\tmp\19898B~1\123_5~1.123123=5.123" ---???--- "C:\tmp\19 €\123=5.123"
"C:\tmp\19898B~1\12_4~1.123\12=4.1234" ---???--- "C:\tmp\19 €\12=4.1234"
"C:\tmp\204C83~1\12345_~1.1€\12345=7.1" ---???--- "C:\tmp\20 €\12345=7.1"
"C:\tmp\204C83~1\1234_6~1.12\1234=6.12" ---???--- "C:\tmp\20 €\1234=6.12"
"C:\tmp\204C83~1\123_5~1.123\123=5.123" ---???--- "C:\tmp\20 €\123=5.123"
"C:\tmp\204C83~1\12_4~1.123€\12=4.1234" ---???--- "C:\tmp\20 €\12=4.1234"
"C:\tmp\21C58B~1\12345_~1.1 €\12345=7.1" ---???--- "C:\tmp\21 €\12345=7.1"
"C:\tmp\21C58B~1\1234_6~1.12€\1234=6.12" ---???--- "C:\tmp\21 €\1234=6.12"
"C:\tmp\21C58B~1\123_5~1.123€\123=5.123" ---???--- "C:\tmp\21 €\123=5.123"
"C:\tmp\21C58B~1\12_4~1.123 €\12=4.1234" ---???--- "C:\tmp\21 €\12=4.1234"
"C:\tmp\2284D4~1\12345_~1.1 €\12345=7.1" ---???--- "C:\tmp\22 €\12345=7.1"
"C:\tmp\2284D4~1\1234_6~1.12 €\1234=6.12" ---???--- "C:\tmp\22 €\1234=6.12"
"C:\tmp\2284D4~1\123_5~1.123 €\123=5.123" ---???--- "C:\tmp\22 €\123=5.123"
"C:\tmp\2284D4~1\12_4~1.123 €\12=4.1234" ---???--- "C:\tmp\22 €\12=4.1234"
"C:\tmp\139~1\123~1.ABCbcd" ---???--- "C:\tmp\1 3 9\123.abcd"
"C:\tmp\139~1\1234~1.ABCbcd" ---???--- "C:\tmp\1 3 9\1234.abcd"
"C:\tmp\139~1\12345~1.ABCbcd" ---???--- "C:\tmp\1 3 9\12345.abcd"
"C:\tmp\139~1\123456~1.ABCbcd" ---???--- "C:\tmp\1 3 9\123456.abcd"
"C:\tmp\138~1\123~1.ABCcd" ---???--- "C:\tmp\1 3 8\123.abcd"
"C:\tmp\138~1\1234~1.ABCcd" ---???--- "C:\tmp\1 3 8\1234.abcd"
"C:\tmp\138~1\12345~1.ABCcd" ---???--- "C:\tmp\1 3 8\12345.abcd"
"C:\tmp\138~1\123456~1.ABCcd" ---???--- "C:\tmp\1 3 8\123456.abcd"
"C:\tmp\137~1\123~1.ABCd" ---???--- "C:\tmp\1 3 7\123.abcd"
"C:\tmp\137~1\1234~1.ABCd" ---???--- "C:\tmp\1 3 7\1234.abcd"
"C:\tmp\137~1\12345~1.ABCd" ---???--- "C:\tmp\1 3 7\12345.abcd"
"C:\tmp\137~1\123456~1.ABCd" ---???--- "C:\tmp\1 3 7\123456.abcd"
"C:\tmp\19898B~1\12345_~1.1\12345=7.1" ---???--- "C:\tmp\19 €\12345=7.1"
"C:\tmp\19898B~1\1234_6~1.121234=6.12" ---???--- "C:\tmp\19 €\1234=6.12"
"C:\tmp\19898B~1\123_5~1.123123=5.123" ---???--- "C:\tmp\19 €\123=5.123"
"C:\tmp\19898B~1\12_4~1.123\12=4.1234" ---???--- "C:\tmp\19 €\12=4.1234"
"C:\tmp\204C83~1\12345_~1.1€\12345=7.1" ---???--- "C:\tmp\20 €\12345=7.1"
"C:\tmp\204C83~1\1234_6~1.12\1234=6.12" ---???--- "C:\tmp\20 €\1234=6.12"
"C:\tmp\204C83~1\123_5~1.123\123=5.123" ---???--- "C:\tmp\20 €\123=5.123"
"C:\tmp\204C83~1\12_4~1.123€\12=4.1234" ---???--- "C:\tmp\20 €\12=4.1234"
"C:\tmp\21C58B~1\12345_~1.1 €\12345=7.1" ---???--- "C:\tmp\21 €\12345=7.1"
"C:\tmp\21C58B~1\1234_6~1.12€\1234=6.12" ---???--- "C:\tmp\21 €\1234=6.12"
"C:\tmp\21C58B~1\123_5~1.123€\123=5.123" ---???--- "C:\tmp\21 €\123=5.123"
"C:\tmp\21C58B~1\12_4~1.123 €\12=4.1234" ---???--- "C:\tmp\21 €\12=4.1234"
"C:\tmp\2284D4~1\12345_~1.1 €\12345=7.1" ---???--- "C:\tmp\22 €\12345=7.1"
"C:\tmp\2284D4~1\1234_6~1.12 €\1234=6.12" ---???--- "C:\tmp\22 €\1234=6.12"
"C:\tmp\2284D4~1\123_5~1.123 €\123=5.123" ---???--- "C:\tmp\22 €\123=5.123"
"C:\tmp\2284D4~1\12_4~1.123 €\12=4.1234" ---???--- "C:\tmp\22 €\12=4.1234"
C:\tmp>
Liviu
Re: case study of the 'for %~sk' bug in win7
Posted: 11 Feb 2013 05:22
by foxidrive
Interesting to read there are differing bags.
It would help if you highlight the lines that error - it's difficult to see what we are looking at on a simple inspection.
Re: case study of the 'for %~sk' bug in win7
Posted: 11 Feb 2013 16:42
by Liviu
foxidrive wrote:It would help if you highlight the lines that error - it's difficult to see what we are looking at on a simple inspection.
Right. Done, with *** since colors don't seem to work inside code tags.
While the legacy XP bug has a discernable pattern, the Win7 one looks quite odd. Maybe someone with more imagination (and spare time
) can guess the hidden failure rule.
Re: case study of the 'for %~sk' bug in win7
Posted: 11 Feb 2013 20:49
by Squashman
quote tags will show color.
Re: case study of the 'for %~sk' bug in win7
Posted: 11 Feb 2013 22:07
by Liviu
Thanks, and right. But quote tags also coalesce consecutive spaces, and use a proportional font, instead. Guess one can't always have it both ways. Which should be a familiar line around here
Re: case study of the 'for %~sk' bug in win7
Posted: 12 Feb 2013 01:02
by foxidrive
Thanks, I see that these generate the wrong output lower down the page.
Code: Select all
05/29/2010 02:51 PM 0 123~1.ABC 123.abcd ***
05/29/2010 02:51 PM 0 1234~1.ABC 1234.abcd ***
05/29/2010 02:51 PM 0 12345~1.ABC 12345.abcd ***
05/29/2010 02:51 PM 0 123456~1.ABC 123456.abcd ***
Such as in here:
"C:\tmp\139~1\123~1.ABC
bcd" ---???--- "C:\tmp\1 3 9\123.abcd"