.cmd/.bat file size limit? Yes. 2GB

[miskox]

Hello all!

Here https://stackoverflow.com/questions/121 ... atch-files and here https://groups.google.com/g/alt.msdos.b ... _H4JWPih-A (for MS-DOS 6.22) that there is not file size limit to .bat/.cmd. But...

Over the weekend I generated (with another .cmd of course) a .cmd 80+ MB in size so I was wondering if there is a size limit to the batch file. I thought of two possible size limits: 2GB and 4GB. It turns out that there is a 2 GB size limit. Batch will not crash - it will start from the beginning. Here www.kockarna.si/test.zip is a test.cmd (.zip is 106 MB, test.cmd when extracted is 4+ GB, 40,000,005 lines). Test.cmd contains

Code: Select all

@echo off
:: XXXXXXXXXXXXXX (actually there are more Xs - total line length is 110 bytes (including CR+LF)
:: XXXXXXXXXXXXXX
:: XXXXXXXXXXXXXX
:: XXXXXXXXXXXXXX
echo line 10 000 000
:: XXXXXXXXXXXXXX
:: XXXXXXXXXXXXXX
.
.
.
echo line 20 000 000

So it returns this:

Code: Select all

c:\echo %time%&test.cmd
16:54:55,92
line 10000001
line 10000001

C:\findstr /N /I /C:"line" test.cmd
test.cmd:10000002:Echo line 10000001
test.cmd:20000003:Echo line 20000002
test.cmd:30000004:Echo line 30000003
test.cmd:40000005:Echo line 40000004

So: echo line 20 000 000 is past the 2 GB limit. So you can see that it returned to the start (when bit 28 (of 28) was set (negative number).

In a .zip there are also two files: test.exe - it will generate test.cmd (on my machine test.exe finishes very quickly (in a few seconds) but then I have to wait for the cache to be written to the disk - so please be patient if test.cmd's size is still 0 bytes) and test.pb - PureBasic source (see PureBasic.com for demo version if needed).

I doubt that there is a line limit - I think the pointer where current .bat is at (when executing) is in the offset so it has nothing to do with the lines (.bat does not use line numbers (remember BASIC?) and because Aacini wrote a SETFILEPOINTER - same thing I would say). And it would not crash because 40 million is very low number if compared to 2GB.

Also: FIND and NOTEPAD report that the file is too big to process. FINDSTR was able to process it.

This test was done on Windows 10, x64.
Test on XP Pro, x86 will follow.

Saso

[dbenham]

Nice investigation. I never dreamed of testing such a thing.

I always thought of the file size as "limitless", actually meaning bigger than I could ever need. But it makes sense that there would be a size limit, given that cmd must keep track of the byte offset within the file, and push that offset onto a stack every time it makes a CALL so that the file position can be restored upon return.

2 GB is a natural limit - the limit of a signed 32 bit integer.

So you can see that it returned to the start (when bit 28 (of 28) was set (negative number).

Don't you mean bit 32

I'm a bit surprised at the observed behavior. Does it really restart cleanly at the beginning of the file? You could change the first line as follows to get a better idea:

Code: Select all

@echo File beginning & echo off

I imagine restarting cleanly requires extra code to detect byte offset overflow. I would have thought the script would simply crash and burn.


Dave Benham

[miskox]

@Dave: you are correct: 32 bits and not 28 bits (looks like we have too much wind and 4 bits were blown away).

I did a test with modified .cmd. First three lines contain:

Code: Select all

@echo START
@echo off
::                    1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Test shows:

Code: Select all

C:\test.cmd
START
line 10000001
START
C:\

My initial .cmd had

Code: Select all

echo line

every one thousand lines so that is why I knew it started from the beginning. I then modified it because there were too many ECHOs.

So: be careful you don't have a .bat exceeding 2GB - results will be unexpected.

.zip updated (BEL (0x07) added to the ECHO command so you know when something gets displayed).

Saso