Check MAXIMUM RECORD LENGTH

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
ghostmachine4
Posts: 319
Joined: 12 May 2006 01:13

Re: Check MAXIMUM RECORD LENGTH

#16 Post by ghostmachine4 » 16 Sep 2010 00:49

Not that OP's data should have control characters, but if there is, such as \r (Ctrl-M) , the batch solutions with findstr will not detect them as a character.

Code: Select all

C:\test>more file
CODE1QWERTYYUIOPASDFGHJKLZXCCVVBNM                                             0
CODE2QWERTYYUIOPASDFGHJKLZ1234565MX                                            012
CODE3QWERTYYUIOPASDFGHJKLZX987654M                                             0
CODE2^WER:TYYUIOPASDFGHJKLZ123:565MX                                           0

C:\test>od -c file
0000000   C   O   D   E   1   Q   W   E   R   T   Y   Y   U   I   O   P
0000020   A   S   D   F   G   H   J   K   L   Z   X   C   C   V   V   B
0000040   N   M
0000060
0000100                                                               0
0000120  \r  \n   C   O   D   E   2   Q   W   E   R   T   Y   Y   U   I
0000140   O   P   A   S   D   F   G   H   J   K   L   Z   1   2   3   4
0000160   5   6   5   M   X
0000200
*
0000240       0   1   2  \r  \n   C   O   D   E   3   Q   W   E   R   T
0000260   Y   Y   U   I   O   P   A   S   D   F   G   H   J   K   L   Z
0000300   X   9   8   7   6   5   4   M
0000320
*
0000360                       0  \r  \n   C   O   D   E   2   ^   W   E
0000400   R   :   T   Y   Y   U   I   O   P   A   S   D   F   G   H   J
0000420   K   L   Z   1   2   3   :   5   6   5   M   X
0000440
*
0000500                               0  \r  \r  \n
0000513



Code: Select all

C:\test>more test.bat
@echo off

for /f "tokens=1 delims=:" %%i in (
'findstr /bnrc:"................................................................................." "%~1"'
) do echo %%i

C:\test>more test1.bat
@echo off &setlocal
setlocal EnableDelayedExpansion
for /l %%i in (0,1,%~2) do set "dots=!dots!."
echo Lines with more than %~2 characters:
for /f "delims=:" %%i in ('findstr /n "%dots%" "%~1"') do (set "x=1" &echo %%i)
if not defined x echo No line.
pause

C:\test>test.bat file
2

C:\test>test1.bat file 80
Lines with more than 80 characters:
2


but gawk/sed will detect it

Code: Select all

C:\test>gawk "length>80{print NR}" file
2
4

C:\test>sed.exe -n /^.\{81\}/= file
2
4

amel27
Expert
Posts: 177
Joined: 04 Jun 2010 20:05
Location: Russia

Re: Check MAXIMUM RECORD LENGTH

#17 Post by amel27 » 16 Sep 2010 01:18

ghostmachine4 wrote:but if there is, such as \r (Ctrl-M) , the batch solutions with findstr will not detect them as a character.
hm... it is very specific case, but legal windows EOL is "\r\n"... :wink:

ghostmachine4
Posts: 319
Joined: 12 May 2006 01:13

Re: Check MAXIMUM RECORD LENGTH

#18 Post by ghostmachine4 » 16 Sep 2010 01:38

amel27 wrote:
ghostmachine4 wrote:but if there is, such as \r (Ctrl-M) , the batch solutions with findstr will not detect them as a character.
hm... it is very specific case, but legal windows EOL is "\r\n"... :wink:

it doesn't matter. In general, control characters in files are also characters. I highly doubt OP will have control characters, but in case there are, its always good to take care of them.

darioit
Posts: 230
Joined: 02 Aug 2010 05:25

Re: Check MAXIMUM RECORD LENGTH

#19 Post by darioit » 16 Sep 2010 04:48

Hello everybody,

thanks to everybody for each response.
I agree with ghostmachine4 "findstr is also not batch. Its an external command as well."

Definitivly I chose AWK because has stunning performance, and also is very simply to use.

I have one more question, I wrote this simply code, and it works fine

checkfile.bat file1.txt 80

Code: Select all

gawk.exe "length>%2{print \"at line \"NR\" bad record \"$0}" %1 >> C:\log.txt


But I need some extra command if the condition is true and the log.txt is written, like to call another part of batch program

Regards
Dario

ghostmachine4
Posts: 319
Joined: 12 May 2006 01:13

Re: Check MAXIMUM RECORD LENGTH

#20 Post by ghostmachine4 » 16 Sep 2010 06:16

if your file satisfy the 80 characters rule, then your log.txt will be 0 size. So you can check for 0 size log.txt file, and call your other batch statements.

(As a side note, depending on what you want doing next, your task could well be written as an gawk script , but let's not go into that)

amel27
Expert
Posts: 177
Joined: 04 Jun 2010 20:05
Location: Russia

Re: Check MAXIMUM RECORD LENGTH

#21 Post by amel27 » 16 Sep 2010 06:24

Code: Select all

gawk "length>%~2{print \"at line \"NR\" bad record \"$0;N+=1};END{exit N}" "%~1" >>"%~n0.log"
if errorlevel 1 call :err %errorlevel%
::...
::some code
::...
exit /b 0

:err
echo bad records: %~1
goto :eof

darioit
Posts: 230
Joined: 02 Aug 2010 05:25

Re: Check MAXIMUM RECORD LENGTH

#22 Post by darioit » 16 Sep 2010 07:01

PERFECT !!!!
1 million row in only 3 second !!!!!

Many Thanks

Regards
Dario

!k
Expert
Posts: 378
Joined: 17 Oct 2009 08:30
Location: Russia

Re: Check MAXIMUM RECORD LENGTH

#23 Post by !k » 16 Sep 2010 08:48

with sed :

Code: Select all

set "err="
set /a len = %~2 + 1
for /f %%a in ('sed -n /^^.\{%len%\}/^= "%~1" ^|sed -n $^=') do set err=%%a
if not "%err%"=="" goto :err
::...
::some code
::...
exit /b 0

:err
echo bad records: %err%
goto :eof

darioit
Posts: 230
Joined: 02 Aug 2010 05:25

Re: Check MAXIMUM RECORD LENGTH

#24 Post by darioit » 17 Sep 2010 03:57

It's possible substitute in awk command "length>" with "length NOT EQUAL" ?

Regards
Dario

amel27
Expert
Posts: 177
Joined: 04 Jun 2010 20:05
Location: Russia

Re: Check MAXIMUM RECORD LENGTH

#25 Post by amel27 » 17 Sep 2010 04:03

Code: Select all

Relational expressions are allowed using the relational operators: 

<   less than
<=  less than or equal to
==  equal to
>=  greater than or equal to
!=  not equal to
>   greater than

ghostmachine4
Posts: 319
Joined: 12 May 2006 01:13

Re: Check MAXIMUM RECORD LENGTH

#26 Post by ghostmachine4 » 17 Sep 2010 09:24

darioit wrote:It's possible substitute in awk command "length>" with "length NOT EQUAL" ?

Regards
Dario

use !=

Post Reply