SET variable and MKDIR in one line? Automate a process

[TMC]

Dear Forum,

I'm in need of some help with a command line I use to crop point cloud data. The software I want to use provides a very good command line mode (CloudCompare). So I wrote a little CMD one liner which is almost doing everything I want, but now I want to automate this process.

First I'm going to explain my goal, code and then my issues with it.

In an folder I have my original point cloud files "c:\originals\file_1 .. file_n". I need every file to be cropped at specific coordinates So I used a for loop to do this. I want to crop the files at over 800 positions so an automation is my goal. The cropped point clouds are going to be saved in an folder with an unique ID "c:\originals\id\". I have a list (right now in excel) with my coordinates and ID which I want to feed into the cmd line.

set the id to e.g 1 (given by my excel file) I have to run this line stand alone. Should I use /V:OFF to disable delayed environmental expansion?

Code: Select all

set id=*

make the directory for output. Problem: everything after this line is used to create folders. e.g. the for of the next line is going to be a folder, how do I prevent this?

Code: Select all

mkdir %id%

The for loop to iterate over all files in the "originals" folder were the CMD is active (shift right click) %f is a variable for the filename

Code: Select all

for %f in (./*) do start /w "" 

Since there are over 140 files to process I use start /w so that only one file is processed at a time

which program should be use to work with the point clouds

Code: Select all

"C:\Program Files\CloudCompare\CloudCompare.exe"

need to shift the date since they are geo referenced %f in use with -O is opening the file

Code: Select all

-O -GLOBAL_SHIFT -312600 -5693000 0 %f

no auto save and -CROP2D is the command for the cropping tool with 6 vertices

Code: Select all

-AUTO_SAVE OFF -CROP2D z 6

here the verticies for the shape of the cropping tool are defined (eg. 67 860 55 875 .... and so on)

Code: Select all

x1 y2 x2 y2 x3 y3 x4 y4 x5 y5 x6 y6

define export format (.bin) binary format

Code: Select all

-C_EXPORT_FMT BIN

save the cropped cloud into a given folder (needs to be created before) %id% is a unique ID of the cropped cloud %f the filename is the same as the original

Code: Select all

-SAVE_CLOUDS FILE: "C:\originals\%id%\%f"

Right now I change the ID and the vertices by hand but since every crop cycle takes about 30 mins to 1 hour I want to automate those two steps to use the "nights" as well .

I'm struggling to implement the the first to lines of code (set id and mkdir) to work with the rest of my code.

Additionally I don't know how to automatically read out the ID and the vertices from my excel file. I think I can save this excel file as an csv text file and somehow read out the ID and vertices. But how do I do this?
If the text file would be in the folder c:\vertices\vertices.txt (with comma separated).
The form of the text file would be ID,x1,y2,x2,y2,x3,y3,x4,y4,x5,y5,x6,y6.

I guess i need to write a batch or .cmd file for that. But my skills are to low to do that on my own.

So In short:

Is there a way to automatically process my originals and make a batch that would crop the originals using the provided code?

Pseudocode:

Code: Select all

1. read the first ID and vertices from file and store them in variables
2. make output directory using the ID as folder name
3. crop all originals using the code lines above and save them in the right folder
4. read the next ID and vertices 
5. make new directory with ID
6. crop .... and so on

I know this is very much to ask, but this would save me a really great deal of time. I hope one of you is kind enough to help me.
If there are any questions don't hesitate to ask me.

Best Tobias

[pieh-ejdsch]

Hallo TMC,

I have to admit - I had to read the text five times to understand what you want. Now I'm behind it.
You use a for /f loop to go through the table. Go to a sub. Work with the help of a second loop and do not need to define a variable extra.

Code: Select all

for /f "usebackQ tokens=1,* delims=," %%i in ("c:\vertices\vertices.txt") do call :cropFile %%j
echo done
pause
exit /b

:cropFile
for %%f in ("c:\output") do (
  echo file %%i  ...
  md "%%~f\%%i"
  "C:\Program Files\CloudCompare\CloudCompare.exe" -O -GLOBAL_SHIFT -312600 -5693000 0 "c:\originals\%%i"  -AUTO_SAVE OFF -CROP2D z 6 %* -C_EXPORT_FMT BIN  -SAVE_CLOUDS FILE: "%%~f\%%i\%%i"
)

Please note - in Same path: A file can NOT have the same name as a folder. That's why I took a separate directory. I do NOT know the naming convention of the files.

Phil

[TMC]

Dear Phil,

I have to admit - I had to read the text five times to understand what you want. Now I'm behind it.

I'm sorry for the confusion. English is not my native language, but I'm happy that you got what I want to achieve. I hoped, that the pseudocode would explain the most part of what I want to do

By reading your code I think this looks quite like what I want, although I'm not quite sure how you'd managed it to store the vertices in a variable called "%*".

But this is going to be the first thing I test tomorrow morning. And I will report back!

Am I going to put this in a .bat or in a .cmd file? or directly in the command prompt? (I'm guessing .bat since you use double % for the variables - am I right?).

Please note - in Same path: A file can NOT have the same name as a folder. That's why I took a separate directory. I do NOT know the naming convention of the files.

Is this a limitation of the language? The naming convention of the original files are: Scan_01.bin, Scant_03.bin ... Scan_n.bin.
The folders were the cropped files should be saved should be named according their ID, which is in the vertices.txt, so the filename is not the same as the folder.

Maybe if you have the time you could explain a bit your lines of code? Especially the part where you access the vertices.txt and where you "store" the vertices in variables, that would be really great.

Thank you very much,

Best Tobias

[TMC]

Dear Phil,

the script is doing nearly what I want. But it seems that the script is taking the ID as the file name. But filename and ID are different things.
The ID is the "name" of the cropped area, the filename after cropping should stay the same.

Let me try to explain in more detail:

In the folder originals there are several hundred files: Scan_01.bin, Scan_03.bin ... Scan_*.bin.
In the file vertices.txt the ID and the vertices of the cropping area is saved: ID,x1,y1,x2,y2...x6,y6 (the ID is just a number like 1,2,3 or 41.1 ...)
Every file in the original folder needs to be cropped by the first set of vertices and saved in a folder named according to the id, then every file in the originals folder needs to be cropped by the second set of vertices and saved in a folder according to the id and so on. Whereas the cropped version of the original file should be the same as the original file.

More detailed:

Code: Select all

ID,x1,y2,x2,y2,x3,y3,x4,y4,x5,y5,x6,y6
1,63.9887,860.4890,55.3285,875.4890,63.9887,890.4890,81.3093,890.4890,89.9695,875.4890,81.3093,860.4890  //this is the first area to be cropped
2,53.3017,857.0570,44.6415,872.0570,53.3017,887.0570,70.6223,887.0570,79.2825,872.0570,70.6223,857.0570	 //this is the second area to be cropped
3.1,x1,y2,x2,y2,x3,y3,x4,y4,x5,y5,x6,y6									 //third
4,x1,y2,x2,y2,x3,y3,x4,y4,x5,y5,x6,y6									 //fourth
890,x1,y2,x2,y2,x3,y3,x4,y4,x5,y5,x6,y6									 //last area

So the script is creating the output folder correctly according to the ID for example "1". (this is working )
In the output folder a cropped version of every original file should be saved. For example c:\output\1\Scan_01_cropped.bin, Scan_03_cropped.bin, ... ,Scan_*cropped.bin

I think one only needs to change the second for loop to something like for %%f in ("c:\originals") do (....). But I'm not sure how to tell the script to take the filename not from the vertices.txt but from the folder originals.

I hope I'd managed to explain it this time good enough so one can understand this by reading this the first time ^^.

Best Tobias

[TMC]

Well after trying to change the code a little I encountered a different problem.

It seems like that the vertices are stored not individually but in one variable, so %* looks like the following: x1,y1,x2,y2...x6,y6 - so the commas (delimiters are stored as well).
But cloud compare needs them to be delimited by a space: x1 y1 x2 y2 and so on. So I changed the vertices.txt to a space delimited layout and adapted the code - I'm okay with this solution.

The second issue I ran into is that not every file in the originals folder was processed. So I changed the code a little bit.
In the code provided I added:

Code: Select all

for %%k in ("C:originals\/*") do start /w ""

Now every file (stored as a path in %%k) in the folder "originals" is cropped. Now it is cropping the files, but I have trouble saving the file in the right location.

If I use this code to save the cloud

Code: Select all

-SAVE_CLOUDS FILE: "\%%i\%%k"

it is trying to save the cropped cloud into "/ID/C:/originals//Scan_01.bin but it should be "C:/output/ID/Scan_01.bin".
The problem seems to be that the variable %%k is the path of the original file. In what manner I need to change this line so that the output is "C:/output/ID/FileName"?

Code: Select all

for /f "usebackQ tokens=1,* delims= " %%i in ("C:\vertices\vertices.txt") do call :cropFile %%j
echo %%*
echo done
pause
exit /b

:cropFile
for %%f in ("C:\output") do (
  echo file %%i  ...
  md "%%~f\%%i"
 for %%k in ("C:originals\/*") do start /w "" "C:\Program Files\CloudCompare\CloudCompare.exe" -O -GLOBAL_SHIFT -312600 -5693000 0 %%k -AUTO_SAVE OFF -CROP2D z 6 %* -C_EXPORT_FMT BIN  -SAVE_CLOUDS FILE: "\%%i\%%k"
)

[TMC]

Alright, I tryied a bit on my own and I came to a really "dirty" solution. I hope someone is kind enough to help me clean this code a little bit up that would be awesome.

Code: Select all

for /f "usebackQ tokens=1,* delims= " %%i in ("C:\vertices\vertices.txt") do call :cropFile %%j
echo %%*
echo done
pause
exit /b

:cropFile
for %%f in ("C:\output") do (
  echo file %%i  ...
  md "%%~f\%%i"
 for %%k in ("C:\originals\/*") do FOR /f "tokens=3 delims=\" %%n IN ("%%k%%") do start /w "" "C:\Program Files\CloudCompare\CloudCompare.exe" -O -GLOBAL_SHIFT -312600 -5693000 0 %%k -AUTO_SAVE OFF -CROP2D z 6 %* -C_EXPORT_FMT BIN  -SAVE_CLOUDS FILE: "c:\output\%%i\%%n"
)

I implemented a second for loop to split the string to the original file path by "\" as a delimiter:

Code: Select all

FOR /f "tokens=3 delims=\" %%n IN ("%%k%%")

This line splits the file path "c:\originals\Scan_01.bin" at the "\" and only stores the last string into the variable "%%n".
Is there a way to automatically set the tokens argument (in this case tokens=3) to the max value? Since the files aren't always in that folder structure?

This line is now saving the file into the output folder + ID + file name , BUT somehow it adds a "%" to the file name. So the saved file is named Scan_01.bin%. This is no major deal but maybe someone could help me clean this mess up a little bit?

Code: Select all

-SAVE_CLOUDS FILE: "c:\output\%%i\%%n"

I know I posted a lot today, but I tried to solve the problem on my own, I hope this is okay

Best Tobias

[pieh-ejdsch]

The first loop is actually only intended to reuse another variable which is generated in a caller loop.
Since you are now using a loop in the called function yourself, you basically do not need the first loop anymore.
But since you get the message about the index only once, this variable will continue to be used there.
The folder is also created only once.
The loop variables can also be expanded to drive: \ path \ filename.extension, as well as date, time, and other attributes.

Code: Select all

for /f "usebackQ tokens=1,* delims= " %%i in ("C:\vertices\vertices.txt") do call :cropFile %%j
echo done
pause
exit /b

:cropFile
for %%f in ("C:\output") do (
  echo file %%i  ...
  md "%%~f\%%i"
 for %%k in ("C:\originals\*") do start /w "" "C:\Program Files\CloudCompare\CloudCompare.exe" -O -GLOBAL_SHIFT -312600 -5693000 0 %%k -AUTO_SAVE OFF -CROP2D z 6 %* -C_EXPORT_FMT BIN  -SAVE_CLOUDS FILE: "%%f\%%i\%%~nxk"
)

Phil

[TMC]

Dear Phil,

thank you again for your time! Most appreciated!

This is working perfectly, although I must admit, that I have some trouble to understand the changes you made.
Somehow CMD language has a quite steep learning curve, at least for me.

Best Tobias