Our full technical support staff does not monitor this forum. If you need assistance from a member of our staff, please submit your question from the Ask a Question page.


Log in or register to post/reply in the forum.

Recombining .dat files


Otemohu May 28, 2014 08:26 AM

Hi all,

I use several dataloggers with each one with one or more datatables. All datatables are collected by loggernet (19 datatables).
Is there an easy way to schedule a recombination of some variables of each tables into a new one?

I thought using task master and launch a .exe file but I don't now how to proceed (programming)?

Cheers,
Daniel


EarlyBird May 28, 2014 08:35 AM

I use MS Excel to do this kind of operation. Can you do the same? Alternatively import all into a SQL database and use a query to select the required fields, MS Access for example.


Otemohu May 28, 2014 08:38 AM

So you launch automatically MS Excel and a programming Excel?
I don't know how to do that.
We have no SQL database.


EarlyBird May 28, 2014 08:47 AM

What do you do with the data files? How do you process and present the data. Do you send the data files to someone else to process?


Otemohu May 28, 2014 08:52 AM

The 19 files are just copied with Loggernet to a local PC (scheduling) and saved. Each night, a local program download all the 19 files from the field site to the lab through satellite.

I would like to select some variables of each file, combining them into a new one and send it to a ftp server.

Downloading and uploading are easy for me but the part concerning the recombining?!?


EarlyBird May 28, 2014 09:00 AM

Have you ever used Split?


Otemohu May 28, 2014 09:03 AM

No never. I have not thought about that. Is it schedulable?


EarlyBird May 28, 2014 09:18 AM

Yes it is schedulable and I always used it in PC208W days, a long time ago!

Not easy to learn and not easy to set up as it is not very intuitive. Also has a maximum of 8 input files at one time so you will need three iterations to process 19 files.


Otemohu May 28, 2014 10:06 AM

I just take a look on Split and, as you said, not easy to to set up and not very intuitive !!!!
I'm not sure it is applicable with the new file format from the CR1000 or CR3000 with header and timestamp ....


EarlyBird May 28, 2014 10:11 AM

I have not used Split for over ten years but apparently it is compatible.


EarlyBird May 28, 2014 10:16 AM

Here is a link as an example

http://www.campbellsci.co.uk/forum/messages.cfm?threadid=8605FF6B-89D5-4AE7-8E74B2289C09BA55


Otemohu May 28, 2014 12:07 PM

Ok thank you. I took a look on the topic but it remains difficult to use.
I added eight of my 19 files to do a test (through "Add Data File"). On the output File Windows, I don't know what I have to do. I can add Time series Heading but nothing is displaying in the Filename raw and impossible to adress a Filename from the Data File I added before ....


EarlyBird May 28, 2014 12:40 PM

Obviously I can do this but it is very difficult for me to talk you through the process. Where are you in the world?


Otemohu May 28, 2014 02:07 PM

Than you for your help. I think I begin to understand. I put numbers in the Select line. It works !!!

I succeed even with files with data gap.

It remains a question: how to incorporate a header with name of variable, unit, etc..; into the .prn file.

The Line1, Line2 are only dedicated to the report.

Daniel


EarlyBird May 28, 2014 06:12 PM

You are getting there quicker than I did the first time. This is a tedious long winded process, using Split I mean. But once it is set up it will work reliably and for a long time without modification.


Otemohu May 28, 2014 08:36 PM

Thank you. It works for my project but I think I don't still know all the subtlety of Split.


Dana Jun 2, 2014 04:02 PM

Split does work with the newer table-based timestamps. You just use element #1 for all the fields in your timestamp (e.g., date("mm/dd/yy h:nn";1;1;1;1) to format the time stamp). Note that the default column width for Split fields is 8, but you can set the width of each field individually on the Output File tab, Width row of the grid. If the field is not wide enough to accommodate the data, you will get a Bad Data error when you run the PAR file.

You will need to timesync the files (using an entry on the Start Condition line) so that the times stay sync'd up for the files. There is information in the Split help on this.

There is also information in the Task Master help on setting up a Split PAR file to run as a task.

Dana


Otemohu Jun 3, 2014 02:58 PM

Thank you Dana.
It working well now.


patrick110 Aug 30, 2014 06:48 AM

What do you do with the data files? How do you process and present the data.


bazenservis Sep 16, 2014 12:38 PM

I am in the same situaltion. Thank you Dana, you show me the way how to make it out.

Log in or register to post/reply in the forum.