Split text file into multiple files, delete input file, move split files to another directory

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
bearwires
Posts: 7
Joined: 06 Jun 2017 06:40

Split text file into multiple files, delete input file, move split files to another directory

#1 Post by bearwires » 06 Jun 2017 06:46

I am hoping the wonderful experts here can assist me with my problem. I have searched for hours and tried all the answers I found similar to my problem but cant seem to get any of them to work for this particular problem. I am not a coder but have dabbled with other code and got them to work for other issues.

I need to split a text file containing keywords(1 kw per line, no blank lines) into multiple text files within the same directory, each with 300 lines (except last text file if total input lines not exactly divisible by 300). The input file will NEVER be larger than 100MB and usually less than 50MB.

Ideally, I then need the input file deleted once it has been split and all split text files moved to another directory (there are no other text files to worry about in the original directory) I need it to be a bat file or vbs script called via bat file.

input file:

keyword-file.txt

output files:

keyword-file_1.txt (300 lines)
keyword-file_2.txt (300 lines)
keyword-file_3.txt (300 lines)
etc

To clarify requirements the above:

1) Split input text file (<100MB) into smaller text files, each with 300 lines
2) Delete input text file
3) Move all split text files to another specified directory

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Split text file into multiple files, delete input file, move split files to another directory

#2 Post by aGerman » 06 Jun 2017 10:20

In order to pick a good technique:
How many lines would a file of ~100MB contain and what's the maximum length of a line?

Steffen

bearwires
Posts: 7
Joined: 06 Jun 2017 06:40

Re: Split text file into multiple files, delete input file, move split files to another directory

#3 Post by bearwires » 06 Jun 2017 10:55

A 100MB file would contain roughly 2 million keywords/phrases. Usually, the file is alot smaller at around 30MB though.
A keyword or phrase can be upto about 100 characters which is inclusive of spaces.

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Split text file into multiple files, delete input file, move split files to another directory

#4 Post by aGerman » 06 Jun 2017 11:32

Customize the variables infile, outfolder, and bunchsize. Remove the REM in the last line in order to delete the input file if the code works for you.

Code: Select all

@echo off &setlocal

set "infile=keyword-file.txt"
set "outfolder=C:\test"
set "bunchsize=300"

for %%i in ("%infile%") do (set "basename=%%~ni"&set "extension=%%~xi")
if not exist "%outfolder%\" md "%outfolder%"
for /f %%i in ('type "%infile%"^|find /c /v ""') do set /a "bunches=%%i / bunchsize, remaining=%%i %% bunchsize"

setlocal EnableDelayedExpansion
<"!infile!" (
  for /l %%i in (1 1 %bunches%) do (
    >"!outfolder!\!basename!_%%i!extension!" (
      for /l %%j in (1 1 %bunchsize%) do (
        set "line="&set /p "line="
        echo(!line!
      )
    )
  )
  if %remaining% gtr 0 (
    set /a "bunches+=1"
    >"!outfolder!\!basename!_!bunches!!extension!" (
      for /l %%j in (1 1 %remaining%) do (
        set "line="&set /p "line="
        echo(!line!
      )
    )
  )
)

REM del "!infile!"

Steffen

bearwires
Posts: 7
Joined: 06 Jun 2017 06:40

Re: Split text file into multiple files, delete input file, move split files to another directory

#5 Post by bearwires » 07 Jun 2017 16:41

Thanks Steffan, it appears to work great on a 7MB keyword file. :D
Thanks you very much for your help and prompt response.

Post Reply