Skip to main content
C
ClaudioPedrazzi12-Amethyst
12-Amethyst
July 21, 2015
Solved

Decimation (downsampling) of ASCII (txt) input files

  • July 21, 2015
  • 11 replies
  • 4833 views

Hi everybody,

although I looked a little in past questions, I don't seem to find anything suitable.  I am confronted with the problem of reducing the sampling rate of a set of files, if possible with a variable reduction factor. For example setting reduction := 60 I would then have a value every minute, instead of a value every second, and so on. All my attempts are baffled by error: "insufficient memory for the required operation".

I built an example with a limited amount of data, in order to show what I am trying to do. But the real files that I have got can reach 1.5 Millions of lines. They are also not readable with Excel.

Moreover, the procedure seems to me very slow.  There has to be a better approach, but at the moment I cannot see it.

Thanks a lot in advance for any hint!

Regards

Claudio

PS: a small collateral question would be if there is some easy Mathcad-way to convert the time stamp "hh:mm:ss" in a normal number (like for example seconds from midnight or so).

Best answer by AlanStevens

Does the attached help at all?

Alan

11 replies

A
AlanStevens19-TanzaniteAnswer
19-Tanzanite
July 21, 2015

Does the attached help at all?

Alan

C
ClaudioPedrazzi12-AmethystAuthor
12-Amethyst
July 21, 2015

Hi Alan,

thanks a lot!!!

I can understand what you did. The idea with an index (j) having an index itself (ir) is really interesting, I never would have thought of it!

I have tried your version with one of my production files, one with 1.2 million lines. it works!  If I then change the file name and try to execute the Worksheet again for a new file, I get "insufficient memory"... but this is a minor inconvenience, I have just to close Mathcad and restart it, and then it works.

Thank you also for the function for computing the number of seconds. I have a question concerning that one, probably it has nothing to do with the function itself, but I still would like to understand.

When I use WRITEPRN to write out the file, although the data seem correct in Mathcad, they appear wrong in the file, precisely they appear wrong as soon as the time goes back to 00:00:00 ... what in my actual production data can happen. I just cannot understand that!  This is certainly an error in WRITEPRN function.  I include a picture of the problem. The "yellow line" is the first wrong one: the value 91 seconds should go in the first column!

Unbenannt.JPG

If you (or anyone) have any suggestion what I am doing wrong here, it would also be appreciated!

Best regards

Claudio

C
ClaudioPedrazzi12-AmethystAuthor
12-Amethyst
July 21, 2015

I understood the problem with WRITEPRN.  In my opinion, it is not intended for writing out files with a given number of columns. It is just intended for writing out a file that is readable with READPRN. I tested this rereading the file and subtracting the two matrixes.  They match.  I was confused because I wanted a file similar to my input one, with a constant number of columns.

I guess if I want that I will have to use other functions.  Still no idea which one .

Regards

Claudio

R
RichardJ19-Tanzanite
19-Tanzanite
July 21, 2015

PS: a small collateral question would be if there is some easy Mathcad-way to convert the time stamp "hh:mm:ss" in a normal number (like for example seconds from midnight or so).

See here: Date Calendar and Time functions.mcd - PTC Community

    C
    ClaudioPedrazzi12-AmethystAuthor
    12-Amethyst
    July 21, 2015

    Thanks a lot Richard,

    was not aware of that one.

    Terms & ConditionsAccessibility statement