Page 1 of 1

ZeroKs 0 KB file compression

Posted: 13 Apr 2022 11:35
by Jim W.
ZeroKs will convert most any file to 0 K bytes,
at least from what you can tell by the file size.
When ZeroKs implodes a file, the data is not
stored IN files. Instead, it IS the files.
And the files are always size zero.
One consequence, in theory at least, is that
when a disk quota is in effect, the operating
system (or perhaps cloud server) may not see
the imploded files as using any space at all.

Code: Select all

[code]
rem save as ZeroKs.bat
@echo off
enabledelayedexpansion
if not exist %~dp0temp md %~dp0temp
if not exist %~dp0implode md %~dp0implode
if not exist %~dp0explode md %~dp0explode
:Option
setlocal
cls
echo.
echo.                     ZeroKs v. 0.00
echo.
echo.             I-mplode  E-xplode  N-fo  X-it
echo.
set /p "option=Select:->"
if %option%==I set option=i
if %option%==i goto :Implode
if %option%==E set option=e
if %option%==e goto :Explode
if %option%==N set option=n
if %option%==n call :Info
if %option%==X set option=x
if %option%==x echo Thank you and goodbye.&exit /b
goto :Option
:Implode
echo.
echo              ZeroKs implode file to 0 bytes
echo.
echo Set input directory (file to implode)
echo ENTER for default [%~dp0]
set /p "ImplodeInputDir=:->"
if not defined ImplodeInputDir set ImplodeInputDir=%~dp0
if not exist %ImplodeInputDir% goto :Implode
echo.
echo Directory of %ImplodeInputDir%
echo.
set num=0
for %%A in (%ImplodeInputDir%\*.*) do (
set /a num=!num!+1
set file!num!=%%~nxA
echo !num!	%%~nxA)
echo.
echo Select file by number
set /p "ImplodeInputFileNum=:->"
set OriginalFileName=!file%ImplodeInputFileNum%!
for %%A in (%ImplodeInputDir%\%OriginalFileName%) do (
set ImplodeInputFilePathName=%%~fA)
echo.
echo Set output directory
echo ENTER for default [%~dp0implode\]
set /p "ImplodeOutputDir=:->"
if not defined ImplodeOutputDir set ImplodeOutputDir=%~dp0implode
if not exist %ImplodeOutputDir% goto :Implode
call CERTUTIL.EXE -encode %ImplodeInputFilePathName% %~dp0temp\%OriginalFileName%.enc >nul
call FIND.EXE /c "==" %~dp0temp\%OriginalFileName%.enc >nul
set el=%errorlevel%
call FIND.EXE /c "=" %~dp0temp\%OriginalFileName%.enc >nul
set el=%el%%errorlevel%
set linenumber=10000
for /f "skip=1 usebackq delims=" %%A in ("%~dp0temp\%OriginalFileName%.enc") do (
set /a linenumber=!linenumber!+1
set "line=%%A"
set "line=!line:/=(!"
set "line=!line:+=)!"
set /a totalfiles=!linenumber!-10001
if "!line:~0,1!"=="-" set /a numfiles=!linenumber!-10001 & set line=!numfiles!_files_%el%&set linenumber=10000
copy /y nul %ImplodeOutputDir%\%OriginalFileName%[!linenumber!]!line! >nul)
del %~dp0temp\%OriginalFileName%.enc
echo.
echo ZeroKs file implode complete.
echo !OriginalFileName! saved as (%totalfiles%) 0 byte files in [%ImplodeOutputDir%\]
echo.
endlocal
pause
goto :Option
:Explode
setlocal
echo.
echo           ZeroKs explode file to original state
echo.
echo Set input directory
echo ENTER for default [%~dp0implode\]
set /p "ExplodeInputFileDir=:->"
if not defined ExplodeInputFileDir set ExplodeInputFileDir=%~dp0implode
if not exist %ExplodeInputFileDir% goto :Explode
echo.
echo Directory of %ExplodeInputFileDir%
echo.
set num=0
for %%A in (%ExplodeInputFileDir%\*10000*.*) do (
set /a num=!num!+1
set file!num!=%%~nxA
echo !num!	%%~nxA)
echo.
echo Select file by number
set /p "ExplodeInputFileNum=:->"
echo.
echo Set output directory
echo ENTER for default [%~dp0explode\]
set /p "ExplodeOutputDir=:->"
if not defined ExplodeOutputDir set ExplodeOutputDir=%~dp0explode
set indexfile=!file%ExplodeInputFileNum%!
call :Countloop
copy nul %~dp0temp\%OriginalFileName%.enc >nul
for /f "skip=1" %%A in ('dir /o /a-d /b %ExplodeInputFileDir%\%OriginalFileName%*') do ( 
set line=%%A
set line=!line:~%startpos%!
set "line=!line:(=/!"
set "line=!line:)=+!"
echo !line! >> %~dp0temp\!OriginalFileName!.enc)
for /f %%A in ('dir /o /a-d /b %ExplodeInputFileDir%\%OriginalFileName%*_*.*') do ( 
set addon=%%A
set "addon=!line:~-2")
for %%A in (%ExplodeOutputDir%\%OriginalFileName%) do (
set ExplodeOutputFilePathName=%%~fA)
if exist %ExplodeOutputFilePathName% (
echo.
echo File %ExplodeOutputFilePathName% exists.
set /p "ow=Overwrite? (Y/N):->"
if !ow!==Y set ow=y
if not !ow!==y (
goto :Option
) else (     
del %ExplodeOutputFilePathName%))    
if !addon!==01 echo ==>> %~dp0temp\%OriginalFileName%.enc
if !addon!==10 echo =>> %~dp0temp\%OriginalFileName%.enc
call CERTUTIL.EXE -decode %~dp0temp\%OriginalFileName%.enc %ExplodeOutputFilePathName% >nul
del %~dp0temp\%OriginalFileName%.enc
echo.
echo ZeroKs file explode complete.
echo !OriginalFileName! restored in [%ExplodeOutputDir%\]
echo.
endlocal
pause
goto :Option
:Countloop
if not defined count set /a count=0
set /a count=%count%+1
set leftchr=%indexfile:~0,1%
set tempfilename=%tempfilename%%leftchr%
if "%leftchr%"=="[" set OriginalFileName=%tempfilename:~0,-1%
if "%leftchr%"=="]" set /a startpos=%count% & exit /b
set indexfile=%indexfile:~1%
goto :Countloop
:Info
echo.
echo Implosion. Collapse. Entropy. Dark matter. Zero.
echo.
echo ZeroKs will convert most any file to 0 K bytes,
echo at least from what you can tell by the file size.
echo.
echo When ZeroKs implodes a file, the data is not
echo stored IN files. Instead, it IS the files.
echo And the files are always size zero.
echo.
echo One consequence, in theory at least, is that
echo when a disk quota is in effect, the operating
echo system (or perhaps cloud server) may not see
echo the imploded files as using any space at all.
echo.
echo Zeroks is written entirely in Windows batch
echo so you know you're getting the best quality
echo code money can't buy. Plus, it's text readable.
echo.
echo Feel free to copy it. You can even ZeroKs it.
echo.
echo NO Copyright 2022 Jim Worley tirespider@gmail.com
echo.
pause
exit /b
[/code]

Re: ZeroKs 0 KB file compression

Posted: 15 Apr 2022 06:33
by jfl
I'm afraid you don't understand that creating a file on disk requires allocating data to describe your file characteristics (pathname, create time, access rights, etc.). This is called "meta-data" about the file.
Here, you're converting N KB of data to X*N KB of meta data, with X >> 1.
So
1) You're using far more disk space after your so-called compression.
2) I'm pretty sure that any file system that has disk space quotas will count the meta data space in your quota as well. So this won't make your situation better, this will make it worse.
Did you really try this on a system with actual quotas?

Re: ZeroKs 0 KB file compression

Posted: 15 Apr 2022 22:38
by Jim W.
Try it? Why would I try it? I only wrote the damn thing!

Re: ZeroKs 0 KB file compression

Posted: 16 Apr 2022 14:01
by aGerman
Jim W. wrote:
15 Apr 2022 22:38
Why would I try it?
Because you wrote ..
One consequence, in theory at least, is that
when a disk quota is in effect, the operating
system (or perhaps cloud server) may not see
the imploded files as using any space at all.
... which implies that it might be useful in such case.

Steffen

Re: ZeroKs 0 KB file compression

Posted: 18 Apr 2022 13:20
by Eureka!
jfl wrote:
15 Apr 2022 06:33
I'm afraid you don't understand that creating a file on disk requires allocating data to describe your file characteristics (pathname, create time, access rights, etc.). This is called "meta-data" about the file.
Here, you're converting N KB of data to X*N KB of meta data, with X >> 1.
So
1) You're using far more disk space after your so-called compression.
2) I'm pretty sure that any file system that has disk space quotas will count the meta data space in your quota as well. So this won't make your situation better, this will make it worse.
All will depend on the filesystem. I assume this is about NTFS (because of "access rights"), so a bit more detail:
The NTFS metadata contains a couple of files that describe the structure of the filesystem. These are not (easily) accessible to humans. One of these files is the Master File Table ($MFT). The MFT contains per file a 1KB record (sometimes multiple records) with attributes (not metadata).
Attributes are things like filename, 8.3 ("DOS") filename, 4 different dates (created/modified, etc),file-attributes (Read-only, Hidden,etc).
"Access rights" are saved in a separate metadata file ($secure). At least: in any halfway modern Windows version (IIRC since XP)


As an MFT record is 1KB, there usually is some space left (not every filename consists of 255 characters, for example). That space is used by Windows to save the content of small files (typically 700-800 bytes max), so they don't "eat" a cluster on the volume.
To check for yourself: Create a file with content "Hello World" and check file properties in File Explorer. Size on disk will very likely be 0 Bytes, even though the file-size itself isn't.


It has been many, many years since I did anything professionally with disk quota, but the "ZeroKs compression" would actually work in a corporate environment where data is on a server with NTFS datastroage.
For example, there is a quota per-user on \\server\users-share\<username>.
(quota are enforced locally on the server, so X:\users-share\<username> would be a little more correct)

As the filenames are stored in the MFT, they will not be counted as part of the quota on users-share\<username>.

(Not that I think all this is a very good idea ... )

Re: ZeroKs 0 KB file compression

Posted: 21 Apr 2022 05:13
by jfl
Eureka! wrote:
18 Apr 2022 13:20
but the "ZeroKs compression" would actually work in a corporate environment where data is on a server with NTFS datastroage.
If it does work, then it's an NTFS quota bug!
That's why I asked if he had really tried it:
I'd be surprised if Microsoft had left such a severe bug in their file system, but who knows, it's worth trying!

Re: ZeroKs 0 KB file compression

Posted: 18 May 2022 12:38
by Samir
A little late to the party, but NTFS also has the concept of file streams so you can store the data in a different stream that won't be visible. I'm actually surprised that malware hasn't used this as an exploit yet...

Re: ZeroKs 0 KB file compression

Posted: 18 May 2022 16:18
by ShadowThief
Samir wrote:
18 May 2022 12:38
A little late to the party, but NTFS also has the concept of file streams so you can store the data in a different stream that won't be visible. I'm actually surprised that malware hasn't used this as an exploit yet...
Mostly because alternate data streams get cleared when you download the file

Re: ZeroKs 0 KB file compression

Posted: 19 May 2022 14:51
by Samir
ShadowThief wrote:
18 May 2022 16:18
Samir wrote:
18 May 2022 12:38
A little late to the party, but NTFS also has the concept of file streams so you can store the data in a different stream that won't be visible. I'm actually surprised that malware hasn't used this as an exploit yet...
Mostly because alternate data streams get cleared when you download the file
Interesting. I guess this is especially true once it goes to a non-NTFS file system, making such an exploited limited in scope.