Making a program process multipe files and........can I ask for assitance

[seanm]

Hello.
Background.
I have several drones, each creates flight logs in the form of a .txt file and a .DAT file.
I have these stored on computer and wish to go 'through' them.
In their raw form they are not human readable and so a program "CsvView" and or "TXTlogToCSVtool_general" is used to 'translate' them. Those programs work fine but it is a manual process in that each individual flight log has to be uploaded and processed, one at a time. Once the individual flight log has been processed I have the option to output the data as a csv.
With over one thousand 'flight log's this is tedious.
It seems to me that it should be possible to automate the process.

In a very concise nut shell, how do I do this?

In more detail, the flight logs contain details such as the drone's serial number, its 'name', the radio controller' serial number (though sometimes it is missing) etc. and it would be useful, as a means of sorting these logs, if at least the drone's serial number and or 'name' could be added to the name of the original .txt flight log.

Secondly I would very much like to 'copy' one non-empty line from the csv output of each .txt flight logs and add it to a common csv file.

Am I fooling myself?
Thanks

[aGerman]

Without knowing the content of your files and the related format specification I'm afraid we can't help out at all. And in case the "not human readable" means that the data is saved in a binary format a Batch script will most likely not be able to process the content. Only plain text.

Steffen

[seanm]

Ok and thanks, can you suggest any forums where this sort of thing might be covered?

[aGerman]

Well, I don't know what software does generate these files. If you know it then you could try to find specifications of the files. The drones or their brand might have specific forums where you could find those information or where you could ask for it.

Steffen

[penpen]

I searched google for "CsvView", "Manual" and "Documentation"and got the following as the first link:
https://datfile.net/Doc/intro.html.

In the "CsvView Manual" (first item there) you can find the link TXTlogToCSVtool.

In the OP there, it states:

Notice 3: Tool can be also run from command line like this, example:
TXTlogToCSVtool "C:\temp\inputFile.txt" "C:\temp\outputFile.csv" J I G

In case that's what you are looking for, you could use a for-loop to process them all:
Everything else depends on how the data you want to use for naming is stored in detail (which you should provide).


penpen

[seanm]

Penpen, major happy dance, I don't know how many times I have looked at that page on Phantompilots and not realised the significance of what I was seeing. What is the J I G ?
From memory there is a bit of the film Armageddon where Michael Clarke Duncan goes "thankyou, thankyou, thank you", if I have remembered correctly it is appropriate to this moment.

Coincidentally today I discovered there was a manual for CsvView but I hadn't got 'into it'.

[seanm]

Yes!!!!!!!!!!!! in 8 minutes this

FORFILES /m *.txt /C "TXTlogToCSVtool /c @file @fname.csv"

processed 846 flight logs, absolutely incredible

[Squashman]

Yes!!!!!!!!!!!! in 8 minutes this

FORFILES /m *.txt /C "TXTlogToCSVtool /c @file @fname.csv"

processed 846 flight logs, absolutely incredible

I am still dismayed how users stumble upon FORFILES to use for tasks like this. While it does accomplish the task, a basic FOR command would have sufficed and is normally more efficient and faster.

Code: Select all

FOR %%G IN (*.txt) DO TXTlogToCSVtool "%%G" "%%~nG.csv" 

[seanm]

From my perspective, as a numpty, it was perhaps because of the pages I found it had the most understandable or best explained help pages etc.
Meaning no offense but I don't speak computer and I get the impression that some help pages are written for those who are fluent in speaking computer, so when a numpty like me comes along we take the first option that is understandable.
Any way I will give your version a go and see if there is a difference and thanks for the code.

Just tried
FOR %G IN (*.txt) DO TXTlogToCSVtool "%G" "%~nG.csv"
and it is quicker but it pegs the CPU at 98% whilst it is running and I can't bring any other windows etc. to the fore, so effectively the computer is locked whilst it runs. It also writes every action to the command window which the former did not.
I assume the double %% were for this being run as a batch file but as a simple single line command it wouldn't run with them present so I reverted to the single %