Is there a faster way to create an Array using a program loop?

[asw391]

I'm not sure what I'm doing wrong but Mathcad is taking far longer than I would expect to create simple arrays.

Goal: create an array with 86,400 rows, all values are 0

For example, I can make an array like that immediately with B[86400 := 0.

However if i try to do the same thing using a while or for loop it takes nearly a minute. 

C:] t<--86400

   ] j<--0

   ] while j<t+1

   ]  ] C[j <--0

   ]  ] j<--j+1

   ] C

Now the program I'm actually trying to write is a little more complicated but I need to write arrays/matrices using programs.  My current file takes 20 min to run which is far to much considering that the calculations are not intense.  I figure I must be making mathcad work harder than necessary.  What am I doing wrong?

[ValeryOchkov]

[asw391]

This is why I'm confused.  That matrix takes no time to generate.  However if I try to do it within a program block it takes 10000000x longer.  I need to create arrays within a program block.

[terryhendicott]

the sum of an interval of PowerMatrix is defined diferently in a reply compared to your jpeg.

[PowerMatrix(1)](i)i  in other words it must extract the column of PowerMatrix for each iteration in the sum.

PowerMatrix( i,1)  is direct access and faster in one of the replies.

It is also better to define the upper boundary of this sum as a number rather than calculate it each time for each item in the sum.

[Werner_E]

---

@terryhendicott wrote:

the sum of an interval of PowerMatrix is defined diferently in a reply compared to your jpeg.

[PowerMatrix(1)](i)i  in other words it must extract the column of PowerMatrix for each iteration in the sum.

PowerMatrix( i,1)  is direct access and faster in one of the replies.

---

I would also expect it to be a little faster but have no proof for it as we don't have access to the internal implementation.  As Fred wrote Mathcad was written aoptimized for matrix calculations and I assume that PTC did not destroy everything that was good in Mathcad when they squeezed out prime. A test with the routine in question here showed no significant difference - I tried. After all the matrices here are not that big so even Prime should have no problem with.

---

It is also better to define the upper boundary of this sum as a number rather than calculate it each time for each item in the sum.

---

Not sure what you mean. The upper limit is "interval*k" and I can't image that its calculated for each summand from new as it would be if you replace the sum operator by a self written loop. I would expect the implementation of the sum operator a bit more efficient but again - we don't have access to the source code and so can't know for sure.

You may give it a test drive and report back on this, if you like.

Only thing which could be done a bit better is to replace k*interval by a variable which is interval adde to for each round. Background is that multiplication is slower than addition. Drawback may be a cumulation of round off errors. But I would not expect any significant change in calculation time by doing so.

So after all it still remains a mystery why the routine took 20 minutes for asw391 and just 20 ms for me (and also him now, I guess). We won't find out but just watching his picture without haviing his original sheet to play with.

[Fred_Kohlhepp]

Mathcad was  (I'm not sure if Prime kept it) optimized for vector/matrix operations.  The "vectorize function" that creates an arrow above the operation is far faster (for example) than multiplying two vectors element by element.  When you write a program to populate an array you force Mathcad to throw out that optimization and do element by element.

Without a more specific example I can't offer better guidance.  Investigate stack, augment, and vectorize to see if there is any way to reduce the amount of programming.

[asw391]

Thanks for the reply.  I'll be more specific with what I'm trying to accomplish.  I've essentially modelled power (kW) generated for every second of a day (86,400 seconds).  I have to calculate it at this scale. 

I want to compress the data into 30 min intervals (1800 seconds).  1st column will be the time in min (30,60,90..) and 2nd column will be the average power generated within 30 min interval.  I've done this successfully, it just takes an excessive amount of time. 

I read somewhere that it was faster to create a matrix with all 0's then with a loop replace the values rather than just create the matrix with loop.  It didn't affect calculation time that way.  Attached is the block that takes 15-20min to run. 

[Fred_Kohlhepp]

Rather than create the large vector (86400 seconds) can you

• calculate the first 30 seconds, take the average, write the first row of your "compressed" array,
• calculate the next 30 seconds,
•  
• rinse and repeat

Filling a matrix with zeros then overwriting it sounds like a memory storage trick.

[Werner_E · Accepted solution]

I guess your Powermatrix is a 86401 x 2 matrix. Thats not really big, indeed.

And you want to create a 49 x 3 matrix by averaging some values.

I retyped without doing any optimizations - just used a syntax to access matrix elements which I am more used to.

I even omitted the preallocation of the three vectors. Usually when dealing with big matrices thats a good idea but given that your three vectors are just 49 element vectors I guess the benefit will not be noticable.

As you can see in the screenshot it took about 30 ms to calculate your compressed matrix.

The calculation was done on a very old and lame notebook.

How long does it take youu to do the calculation? Or did I miss something?

[asw391]

I'm not entirely sure why yet, but your  HRes runs perfectly on my computer with my PowerMatrix.  So thank you very much!  I just started using Mathcad this week and was impressed with the software but that excessive calc time was starting to drive me insane. 

I'm very curious though as to what specific subtle edit made such an exponential difference.  

[asw391]

My original program took 20 mins to run

[Werner_E]

---

@asw391 wrote:

My original program took 20 mins to run

---

OMG!

I can't imagine that anything I changed is responsible for the significant difference in speed, though. Simply turned the calculation into a function (just gave it a try - even if I don't the result appears in a glance) and used the usual matrix indices M[i,j instead of M(i,j). There should be no difference in calc time doing so.

I also am curious as to what slowed down calculation on your side.

[terryhendicott]

Your implimentation could take advantage of the submatrix command

b<-submatrix(PowerMatrix,(k-1}*interval+1,interval*k,1,1) to extract the respective rows into a column vector.

Power(k)<-you can then use Ctrl+4 to get the built in summation of a column vector b and divide by interval.

Both can be used in a program.

[Werner_E]

I haven't tried in this case, but from my experience I can tell the the submatrix command usually slows down calculations.  As the routine as it is calculates in a few Milliseconds I guess its not worth trying. It remains a mystery as to why it took 20 minutes for the OP in the first place but the cause seemed not to be due to the used methods to loop, sum up, etc.

[Werner_E]

OK, I was curious and gave it a try and you are right!

submatrix outperforms the sum operator even by a factor 40!!

I made the Powermatrix 100-times larger to get more reliable results:

[Fred_Kohlhepp]

Try:

Prog(N) := | A[N,N <--0

                  return A

Works up to 16000 x 16000, and it's fast.