Best technique for determination of the bend in a curve

[FDS]

Dear Collaboratory,

I have a problem that I cannot resolve. I would like to determine the bend point in a curve. Strange enough my approach seems to work quite well for some curves but fails completely for others. I have to questions: (1) is there a better method to define the bend point? and (2) what is the cause that it sometimes works fine and fails another time (although the curves do not differ too much)

Thank for your help and input with this problem.

[Werner_E]

Its interesting that you get a better results for set1 if you don't trim (but it gets worse for the second set).

Shouldn't the trimming be done automatically? Which algorithm? How would you decide which is the part of the curve thats of interest.

Have you looked at the spline interpolations - they tend to oscillate sometimes, Maybe a linear interpolation is better.

Its also noticeable that the interpolated function is just using the value of set1, but is used for both. Think this shouldn't be that way.

Another thing I spotted is that the trimmed data is not derived from the set data - the values are different!

[Fred_Kohlhepp]

Werner's right.

Adding the second (correct) function solves this problem.

[Werner_E · Accepted solution]

Find attached a version which does not rely on interpolation (which after all is just feigned accuracy).

If some outliers are avoiding a correct result you may use medsmooth on the second derivative. in your dataset there are points at approx t=9.5 very close and this results in a very small negative second discrete derivative which may be found as minimum in other cases. But maybe you know enough about your data to trim it even more than I have done and so you are able to cut away those values completely. My trimming function will return the data between the absolute minimum and the absolute maximum, assuming the minimum is to the left of the max. Obviously it would be possible to trim even more.

[FDS]

Dear Werner & Fred,

Thank you both for your help. The reason why the trim data and the original data are different is because I normally use MC 15 together with an addon called Kornucopia which has many additional functions. These in between steps I did not mention in my post. I first regularized the data with a Kornucopia function, this function regularizes (maps) the data to a common "X" increment based on various options defined through the argument of the function. I then trimmed the data again using a Kornucopia function.

The datasets actually represent the different stages of blood clotting going from liquid to stable clot. Unfortunately, depending on your health and some other factors many different profiles can be generated. So I will try out your modifications on more complicated curves. I hope you allow me to come back on this item if I experience new problems.

Unfortunately I cannot merit you both with the correct answer due to limitations of the collaboratory.

[Werner_E]

Ah, Kornucopia. Unfortunately I never had access to it but from what I read it seems to be a valuable addon with a lot of useful functions.

I guess you followed in your sheet this article http://www.ptc.com/appserver/wcms/resourcecenter/mathcad.jsp?im_dbkey=135085 and I think it would be a good idea to replace my derivation routine for discrete data (diff()) with the appropriate Kornucopia routine, probably followed by some filtering As shown in the aforementioned artice). I am in no way convinced that my derivation routine will cope with outliers in all cases and so it may happen that the absolute minimum is not at the position you are searching for. In fact my first derivative routine (a simple deltay/deltax) suffered from that problem somewhere at t=9.5, where two datapoints are very close together. I was able to fix this by smoothing the second derivative using medsmooth() with a window of 3, but window=5 would smooth out the searched for minimum, too, so I considered this appraoch suboptimal and came up with a different derivation routine.

If the data sets aren't too large (yours are very small which doesn't make things easier as we hardly can smooth) and you don't run into problem with oscillations introduced by the spline interpolation, the interpolation approach may be the more stable one. You have to try out with a bunch of sample data how it works out.

[FDS]

Dear Werner, if you are doing data analysis quite often I can really recommend you Kornucopia. It also has a great support (swift and to the point). If you want to try it you can download a 30day trial version at www.bodietech.com.

I will play a little bit further with my data sets to see what are the most optimal settings.

Thank you again for your help!