Page 1 of 1

How to read a file in UTF-8 and remove a BOM in batch script?

Posted: 04 Oct 2021 07:11
by PiotrMP006
Hi

How to read a file in UTF-8 and remove a BOM in batch script?

Re: How to read a file in UTF-8 and remove a BOM in batch script?

Posted: 04 Oct 2021 08:33
by aGerman
The BOM is just one multibyte-character. You can read it away using PAUSE commands for the 3 bytes.

Code: Select all

@echo off
set "rmvBomRedir=|((pause&pause&pause)>nul&findstr "^^")"
set "rmvBomFor=^|((pause^&pause^&pause^)^>nul^&findstr "^^"^)"

>nul chcp 65001

:: redirect to console
type "utf8.txt" %rmvBomRedir%

:: redirect to a pipe
(type "utf8.txt" %rmvBomRedir%)|find /v ""

:: redirect to a file
>"utf8noBom.txt" (type "utf8.txt" %rmvBomRedir%)

:: process in a FOR /F loop
for /f "delims=" %%i in ('type "utf8.txt" %rmvBomFor%') do echo %%i

pause
Steffen

Re: How to read a file in UTF-8 and remove a BOM in batch script?

Posted: 05 Oct 2021 01:54
by aGerman
Forget about what I told you about one PAUSE is enough. Updated the code above.