Community Tip - You can Bookmark boards, posts or articles that you'd like to access again easily! X
Does anyone know how I can read a binary comtrade data file. In this case "File.dat"
What is comtrade? See https://en.wikipedia.org/wiki/Comtrade.
I am interested in displaying the File.dat The others are easy to open in mathcad.
Once I can view it then I can do further processing to it in mathcad
Thanks.
Solved! Go to Solution.
The data file information is specified in IEEE standard C37.111, which has become an IEC standard in 2001.
See here:https://webstore.iec.ch/preview/info_iec60255-24%7Bed2.0%7Db.pdf
Success!
Luc
The .hdr (header) file is an XML formatted file.
The .dat (data) file appears to be a binary data file.
The .cfg (configuration) file is a comma separated values (.csv) file.
While Mathcad can 'read' all these files (the first two with the READBIN function, the last one with READPRN), it cannot interpret them.
You'll have to know the format and meanings of fields, then you can write Mathcad programs to actually interpret the files.
Alternatively, if you have c-code that can interpret these files, you could wrap that into a User-DLL that adds the specific functions to Mathcad.
Success!
Luc
The data file information is specified in IEEE standard C37.111, which has become an IEC standard in 2001.
See here:https://webstore.iec.ch/preview/info_iec60255-24%7Bed2.0%7Db.pdf
Success!
Luc
It's done.
Results:
Let me know if you like it.
Luc
Looks great. But where do you actually give the full path to the files. When I try it sees the "." cfg then does a subscript. Could you give a screen capture of where you actually point the mathcad to the actual files. Thanks for your effort. You have put a lot of work into it. Genius!
I get this error:
You have an exclamation mark '!' in your path. That (apparently) is an illegal character.
Instead of using CWD, define the you path string explicitly, like:
file:= "D:\....\H035"
Success!
Luc
Ah, better still.... assuming you have the mathcad file(s) in the same directory as the .cfg and .dat files, just define:
file:="H035"
That should do it.
Success!
Luc
P.S. I see that you're using Prime (and probably used Mathcad 15 to open my source file).
Does the utility function in Mathcad 15?
Hi The conversion to Prime 4 looks fine. The "!" Is a sub folder. But it also failed to open in the d: root drive. I'll give it another go tomorrow. I really appreciate your work on this. When I get the prime 4 working I'll post it here.
Even if you don't get it to work, I may be able to help to find the problem.
Though I cannot execute programs in Prime 4, I can read the file and inspect it.
Success!
Luc
Hi
Errors I got in Mathcad 15 and prime 4.0.
Hope you can shed some light. Much appreciated.
I can confirm the error in Mathcad 14, M045.
Its the vec2str(W) statement in the READCSVI routine which throws the error because it obviously can't convert a vector containing a zero into a string.
@Luc
Haven't digged into the logic of your program and can't tell why you set W to a 1x1 matrix containing a zero from time to time.
How does vec2str handles a zero byte in MC11? Maybe as a workaround we could write a new vec2str in MC11 which simulates that behaviour.^
!!! EDIT:
Solution found!
Simply replace the two occurrences of W <-- (0) by W <-- 0 and it seems to work OK. At least it works in MC15
I can't give it a try with P4, but I guess the problem there is the very same.
Below you see the difference between vec2str(0) and vec2str of a 1x1 matrix containing 0. The first returns an empty string (which is needed) but the second throws an error. Guess this is different in MC11
Hi The Cfg now works but Dat file says : "can't open the data file"
I can't replicate that error using Mathcad 15) - just changed the two program lines mentioned and it works.
Are you sure you have both files, "file.cfg" AND "file.dat" in the same directory as your Mathcad file? "file.hdr" seems not to be used by Luc's sheet.
The .hdr file is not part of the 'critical' comtrade data, but the .cfg file is needed in order to be able to interpret the .dat file.
Werner,
I remembered once having had a bad experience with supplying an integer to the vec2str() function.
(The error message says that the argument must be a vector)
But I've checked vec2str() can live with a zero, also in Mathcad 11:
So I could change single element vector containing a zero into a simple zero.
A message to the OP:
In Prime you might not need the READCSV function, because Prime has READCSV, which does the same in this application.
Luc
Yes, it works,
here is the adapted version that should also work in versions of Mathcad above 11.
Luc
Yes! Mathcad 15 works but Prime 4.0 has errors:
I also corrected an error, which means that the data file is now read including the very last sample
(I had to omit the last sample because the function was apparently reading past the end of the data file).
Attached are corrected Mathcad 11 and Prime 4 files.
I don't think that the corrected Prime 4 file will solve the problem you show above.
Luc
Does prime have a trace function, and can you use that to pinpoint the location of the error inside the RaedComtradeDat function more precisely?
I don't think so. I could not see any option.
There IS a trace option in Prime. Not so convenient as in Mathcad, but its there -> "Calculation ribbon" -> "Error Tracing"
Ah, I see from your screenshot that this
is the difference to later Mathcad version which throw an error (maybe we could call that a bug).
Still not sure why the OP now has problems reading the *.dat file.
Only explanation I can thinks of is, that he forgot to copy it to the same locations as the cfg and MC file. Also possible that the file is open and in use by another program - not sure if MC would choke on hat.
Prime 4 does not choke on opening (built in READCSV) the .cfg file while that is open in MsExcel.
And it's got no trouble opening first part or entire .dat file while that is also open in MsWord.
Luc
Working fine in Mathcad 15. Problems when i converted to Prime 4.0
I'm afraid I can't solve the problem that Prime has with my (translated) function. I've started my 30 day evaluation period in order to be able to run programs in Prime 4. It appears that Prime has a big problem with one of the statements in the READCSV program.
I've checked all tricks that the statement uses, and verified that they work in (other) Prime test programs that I made. So that is not the problem. The values of r, c and W just before the statement are as expected, and Prime does allow such assignments to R, effectively creating the matrix. So I cannot see why this should be a problem.
I think that simply Prime 4 is the problem: "system error"...
Anyway, it works in Mathcad 15, I suggest you use that to analyse the Comtrade data.
Success!
Luc
Thank you. I will take it up with the vendor. Time to get some value out of the maintenance and support fees.
I agree that the problem seems to be buggy Prime.
Same error here with Prime 3.0.
Crazy enough that the following workaround(?) solves the problem:
At least it works in Prime 3.0:
Certainly odd, because Prime4 says:
Luc
Yes, same error as in Mathcad 15 as I noted above.
OK, maybe its not to be seen as bug as a string containing the NUL character is not the same as a string containing no character at all, but the error message is misleading and it would be quite convenient if vec2str would accept a scalar as argument.
But this vec2str behavior does not explain why my workaround with the if statement is necessary. It should work as it was written by you because vec2str(0) returns the empty string anyway. First idea was that it isn't en empty string but rather a string containing an invisible NUL character, but Mathcad told me otherwise
So I believe that the strange error message we get in Prime is the result of a a Prime bug.
Anyway, our "Mathcad User" should now be able to use your program in Prime, too. He seems to have another Problem (in MC15) when using a different input file.