remove duplicate string in txt file

[goodywp]

I have a task to remove the duplicate string as below

input:

T501-08680-0102 0501-08690-0100
T501-08680-0102 T501-08699-0100

I would like to have output like this: the second line T501-08680_0102 is duplicate one should be removed

output:
T501-08680-0102 0501-08690-0100 T501-08699-0100

I tried this code but looks like hard-coded when the string is changing there is an issue

Code: Select all

@echo off
setlocal enabledelayedexpansion
set "string=T501-08680-0102 0501-08690-0100 T501-08680-0102 T501-08699-0100"
set "newstring="
for %%i in (%string%) do (
  echo !newstring!|findstr /i "\<%%i\>" >nul || set "newstring=!newstring! %%i"
)
echo %newstring:~1%

there must be some way not like the above code when I do not know which string is coming

Thanks

[Squashman]

I would recommend writing each string out to a file and sorting it. Makes it easier to remove duplicates. Dbenham created a nice function to remove duplicates in a file. I will have to search for the thread.

[Aacini]

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set "string= "
for /F "delims=" %%a in (input.txt) do for %%b in (%%a) do (
   if "!string:%%b=!" equ "!string!" set "string=!string! %%b"
)
echo %string:~2%

input.txt:

Code: Select all

T501-08680-0102 0501-08690-0100 
T501-08680-0102 T501-08699-0100

output:

Code: Select all

T501-08680-0102 0501-08690-0100 T501-08699-0100

Antonio

[goodywp]

Excellent!!! Antonio!
Thanks both you and Squashman!

[goodywp]

Hi Antonio,

Is it possible always to have the output as this order

T501-08680-0102 T501-08699-0100 0501-08690-0100

instead of
T501-08680-0102 0501-08690-0100 T501-08699-0100

Thanks

[Aacini]

• The output of my code is exactly the same output you posted in your question. If you want not such an output, why you posted it?
• I don't understand why the three elements in the output should have this order: T501-08680-0102 T501-08699-0100 0501-08690-0100. Please, describe the rule used in such an ordering.
• Please, carefully read the first topic in this forum. After that, post a new description of your problem that include all specifications...

[goodywp]

Thanks Antonio! I should focus on one requirement....