Rewrite file with UNIX LF endings

[cookie]

I have a batch file which creates a list of files in a folder (dir) and replaces the windows path (P:\....) to Unix formatted path (/mnt/P/...); this txt file will be used as input to a PeopleSoft process. I need to write each record in the file with LF following at the end of each line so right now I have code to remove the CRLF characters but I can't figure out how to insert LF. In Notepad, all the lines show as one long line but in Notepad++ I can see each line followed immediately by LF and that's what I am trying to emulate. I tried dos2unix on the original file but that leaves 2 black spaces before the LF.This is what I have
echo | set /p newline="/mnt/P/Test_Files/FA/Staging/%isir_idsa%.DAT" > P:\Test_Files\FA\TDClient_Inbound/edit_test.txt
Is there a way to do this? Any help is greatly appreciate it.

[aGerman]

A line feed can be created using

Code: Select all

set lf=^


:: make sure the 2 lines above are empty!

I assume your attempt would work that way:

Code: Select all

<nul set /p newline="/mnt/P/Test_Files/FA/Staging/%isir_idsa%.DAT%lf%" >"P:\Test_Files\FA\TDClient_Inbound/edit_test.txt"

Steffen

[cookie]

Thank you for the reply but I can't get it to work. By using
<nul set /p newline="/mnt/P/Test_Files/FA/Staging/%isir_idsa.DAT%lf%" >"P:\Test_Files\FA\TDClient_Inbound/edit_test.txt" I am losing %isir_idsa.DAT% and the output file is not created.

P:\Test_Files\FA\TDClient_Inbound>set lf=


P:\Test_Files\FA\TDClient_Inbound>set /p newline="/mnt/P/Test_Files/FA/Staging/lf\Test_Files\FA\TDClient_Inbound/edit_test.txt" 0<nul
/mnt/P/Test_Files/FA/Staging/lf\Test_Files\FA\TDClient_Inbound/edit_test.txt
P:\Test_Files\FA\TDClient_Inbound>pause
Press any key to continue . . .
Not sure what I am doing wrong...

[aGerman]

Was it required to redirect the path surrounded with quotation marks?

[Squashman]

I use dos2unix and unix2dos all the time without a problem. Today in fact.

You could also use Vbscript to do the line ending change or Dave Benham's JREPL.bat

[ShadowThief]

Because I'm a sucker for (ab)using certutil as a hex editor, this script takes a file that is smaller than 71 MB and goes into the hex to remove the carriage returns but keep the newlines.

Code: Select all

@echo off
setlocal enabledelayedexpansion

:: Ensure a file was passed in
if "%~1"=="" (
    echo Please provide a file to process.
    pause
    exit /b
)

:: Ensure file is under the certutil input limit
if %~z1 GTR 74472684 (
    echo This file exceeds the maximum file size of 74472684 bytes ^(71 MB^)
    echo Please use a smaller file.
    pause
    exit /b
)

:make_rand
:: Generate a random number to reduce the risk of filename collision
set rand=%RANDOM%%RANDOM%
set "temp_file=%~dpf1_%rand%.tmp"
set "hex_file=%~dpf1_%rand%.hex"
set "new_file=%~dpf1_new.%~x1"
if exist %temp_file% goto :make_rand
if exist %hex_file% goto :make_rand

if exist %new_file% choice /c:YN /M "%new_file% already exists. Overwrite? "
if %errorlevel% equ 1 del %new_file%
if %errorlevel% equ 2 exit /b

certutil -encodehex "%~1" "%temp_file%"

:: The script will break if you have spaces in your file path.
:: This is a feature, not a bug. Names your paths correctly.
for /f "tokens=1,*" %%A in (%temp_file%) do (
    set "line=%%B"
    set "hex_substring=!line:~0,48!"
    set "no_carriage=!hex_substring:0d=!"
    echo !no_carriage! >>%hex_file%
)

certutil -decodehex "%hex_file%" "%new_file%"

:: Temp file cleanup
del /q %hex_file%
del /q %temp_file%

[cookie]

Hello,

I did use dos2unix and it works but my problem is a little deeper. Here is the setup:
1. - I create a list with file in the folder
FOR %%i IN (P:\Test_Files\FA\Staging\I*OP.DAT*) DO echo %%i >> P:\Test_Files\FA\TDClient_Inbound\ISIR_list.txt

P:\Test_Files\FA\Staging\IGCO17OP.DAT
P:\Test_Files\FA\Staging\IGCO18OP.DAT

2. - I need to change the Win path to Unix path, so I replace P:\Test_Files....\file1.dat with /mnt/P/Test_Files/....file1.dat and create a new fil
set txtfile=P:\Test_Files\FA\TDClient_Inbound\ISIR_list.txt
set newfile=P:\Test_Files\FA\TDClient_Inbound\ISIR_list_new.txt
set "search=P:\Test_Files\FA\Staging\"
set "replace=/mnt/P/Test_Files/FA/Staging/"
for /f "tokens=*" %%a in (%txtfile%) do (
set newline=%%a
echo %%a
set newline=!newline:%search%=%replace%!
echo !newline! >> %newfile%
)
set out=P:\Test_Files\FA\TDClient_Inbound\EDI_list.txt
type %newfile% > %out%

/mnt/P/Test_Files/FA/Staging/IGCO17OP.DAT
/mnt/P/Test_Files/FA/Staging/IGCO18OP.DAT
3. - I end up with extra spaces even if I do the dos2unix conversion after .DAT and before the LF character
4. - I need to create a list of these files to pass to a process in PeopleSoft which does not like any extra spaces
Maybe I need a different approach...

[Compo]

To get rid of the extra spaces, you should be able to create the list directly from the for loop like this:

Code: Select all

@(FOR %%A IN ("P:\Test_Files\FA\Staging\I*OP.DAT") DO @(SET "_FP=%%~pnxA"
   SET _DL=%%~dA
   SETLOCAL ENABLEDELAYEDEXPANSION
   ECHO=/mnt/!_DL:~,1!!_FP:\=/!
   ENDLOCAL))>"P:\Test_Files\FA\TDClient_Inbound\EDI_list.txt"

You are adding your own unwanted spaces because you have included them before the > and >> redirectors.

If you then wish to get rid of the LF then at least you are working with a clean list!

[dbenham]

I believe the following simple script will do everything you are looking to do. It requires JREPL.BAT

Code: Select all

@echo off
setlocal
set "fileMask=P:\Test_Files\FA\Staging\I*OP.DAT*"
set "outFile=P:\Test_Files\FA\TDClient_Inbound\EDI_list.txt"
(for %%F in ("%fileMask%") do @echo %%~fF) | jrepl "^([a-z]): \\" "mnt/$2 /" /i /u /t " " /o "%outFile%"

You can change the fileMask to any path and file mask, and the code will properly change the format of the drive letter, as well as change all back slashes to forward slashes. It uses the JREPL /T option to perform two find/replace operations in one pass. The /U option specifies unix line terminators.

If your current directory happens to be "P:\Test_Files\FA\Staging", then you could simply set "fileMask=I*OP.DAT*", and the ~f modifier in %%~fF will list the full path.


Dave Benham

[cookie]

Thank you Compo soooo much for your help!!! I don't have enough words to describe my gratitude, I thought I was losing the little bit of sanity that I have left after trying so many things for 5 days with not success. This is the first time I had to do something like this. Used dos2unix and my process in PS worked as expected. Thank you again
Thank you Dave as well - I will look closely into the JREPL since I noticed that is being referenced quite a lot.