Detecting a true zero byte file and not a file being written

[doshelp]

Hello this may not be a solveable problem but I wanted to ask. I wrote a batch file to monitor a directory for any files that equal Zero bytes. We have an application that occasionally errors and creates a zero byte xml file. When it does it stops a system routine from importing from that directory. I am unable to get the softeware vendor to fix the zero byte file creation issue so I am monitoring the folder and manually moving the zero byte file.

The problem I am having is that the batch file is recognizing files that are just starting to be written so I am getting lots of false positives. The file = zero bytes as the batch file runs but when the directory opens for me to look the application has written the rest of the XML and it is no longer zero bytes.

is there any trick to program into the batch wich can tell if a file is still being written rather than detecting it as one of the zero byte files that will never be completed?

Code: Select all

@echo off

setlocal enabledelayedexpansion

if EXIST \\File-Exchange\APInvoices\*.xml (
   for %%j in (\\File-Exchange\APInvoices\*.*) do (
   set size=%%~zj
   set maxbytesize=0
   if !size! EQU 0 start \\File-Exchange\APInvoices
   )
)

[Squashman]

I believe this is what you are looking for. If not Dave will correct me.
http://stackoverflow.com/a/10520609/1417694

[doshelp]

Thank you that code works but I am finding out that the application that writes these XML files is not issuing a lock so the script is returning the result as if it is unlocked for writing.

I tried to tweak the code so that it pauses and checks the file again after 10 seconds. It normally takes the application about 2-3 seconds to write the file so if it detects a zero byte file then I wait 1- seconds and try to check that file's size agian. I am still getting false positives though.

The script works each time when I set the conditions manually. Its when the application is in the process of writing the files that I get the false positives. Maybe I need a longer pause than 10 seconds.

Here is the new code

Code: Select all

@echo off

setlocal enabledelayedexpansion

if EXIST \\File-Exchange\APInvoices\*.xml (
   for %%j in (\\ahacct01\File-Exchange\APInvoices\*.xml) do (
   set size=%%~zj
   set maxbytesize=0
   if !size! EQU 0 timeout /t 10 /nobreak
   set size=%%~zj
   if !size! EQU 0 start \\File-Exchange\APInvoices
   )
)

for %%i in (\\ahacct01\File-Exchange\APInvoices\Error\*.*) do (
   start \\ahacct01\File-Exchange\APInvoices\Error
)

[foxidrive]

Just wondering here - I've noticed in some cases that a filesize is not updated all the time when a file is being created.

I don't have a test case to check, but can you monitor a file that is definitely being written to and report the filesize in a loop, to see if the filesize is actually increasing when checked by the for command?

There may be some time delay before a filesize becomes available.

Code like this to check file.xml

Code: Select all

@echo off
:loop
for %%a in (file.xml) do echo "%%~za" & ping -n 2 "" & goto :loop

[Squashman]

Just wondering here - I've noticed in some cases that a filesize is not updated all the time when a file is being created.

I have encountered the same thing but I thought I remember seeing a thread discussing that very problem but I cannot remember if it was here or somewhere else.

[Squashman]

I wonder if WMIC get filesize would work.

Code: Select all

wmic datafile where name='c:\\file.txt' get FileSize

[doshelp]

I will do some testing. I think what is happening is the application creates a zero byte file and then writes the content to it. The file does not seem to get assigned a file size until the write process is completed. so the short window that a file is being written to my script thinks it is a zero byte file.

I belive a looping solution will work. I am going to try and increase the pause time to 30 seconds and see if that clears out the false positives.

I will also try the other features suggested and maybe add some output to the code so I can see how it sees the file as it is checking.

[Squashman]

FORFILES can report the file size as well but I bet it suffers from the same fate as the DIR command or the FOR command.

Code: Select all

forfiles /m myfile.txt /C "cmd /c echo @fsize"

[Squashman]

Would be interesting to see what the output of the TYPE command is when the program is still writing the XML file.

[Squashman]

The synapses are almost all firing.

http://www.robvanderwoude.com/type.php#ShowSize

[foxidrive]

The synapses are almost all firing.

http://www.robvanderwoude.com/type.php#ShowSize

They are firing pretty well I reckon!

That's a neat solution...

[doshelp]

I will look into a more elegant solution for now I learned something. I was watching the file creation today and I noticed one take about 10 seconds to finish. In the past I always witnessed it in 2-3 seconds. I increased my pause time in my loop to 30 seconds so now if it detects a 0 byte file it waits 30 seconds and checks again. If its still 0 bytes then it triggers. This appears to be working for now. Not the best solution but its removed the false positives during write time.

[Squashman]

Delayed execution is always a decent solution. My predecessor used it a lot in our data automation. But it has bitten me in the @$$ a few times over the years and as I find scripts where she did this I find a better way to code the overall script and automation tasks.

[miskox]

This works for me (probably for files with exclusive lock only?):

Code: Select all

move /y a.a a.a >nul 2>nul

returns ERRORLEVEL 1 if file is locked, 0 if not locked.

Saso

[doshelp]

Here is the new code I added some output to better be able to follow what is going on. I needed to add some more IF Exist checks. There is a process that moves these files out of the directory every 15 minutes and if the file was moved during the 30 sec wait then it was creating a false positive.

to use TYPE and DIR on a networked UNC path I needed to add the pushd command to create a temporary virtual drive for syntax.

I am watching the output now to see if the "TYPE" command is catching the bytes written after the file is detected as 0 Bytes.

Code: Select all

@echo off

setlocal enabledelayedexpansion

pushd \\File-Exchange\APInvoices\

if EXIST \\File-Exchange\APInvoices\*.xml (
   for %%j in ("\\File-Exchange\APInvoices\*.xml") do (
      set size=%%~zj
      echo first read for file 
      echo %%j
      echo %%~zj
      if !size! EQU 0 (
      type "%%j"
      dir "%%j"
      echo ZERO BYTE FILE DETECTED
         if EXIST \\File-Exchange\APInvoices\*.xml (
            timeout /t 30 /nobreak
            if EXIST \\File-Exchange\APInvoices\*.xml (
               type "%%j"
               dir "%%j"
               echo second pass
               set size=%%~zj
               echo %%j
               echo %%~zj
               if !size! EQU 0 (start \\File-Exchange\APInvoices)
            )
         )
      )
   )
)

for %%i in (\\File-Exchange\APInvoices\Error\*.*) do (
   start \\File-Exchange\APInvoices\Error
)

popd