@Fred_Kohlhepp wrote:
Man! That's a lot of data!
First, note that Terry has successfully trimmed this.
Second, Peirce (takes the log of the inequality I'm using. I haven't been able to do that successfully. The large data set is(I think) keeping my solution from working (using the root function.)
Third, my first pass simply treated the entire set as measurements to be averaged and analyzed. Clearly there's a sinusoidal function that might reduce scatter and standard deviation.
Yes, sure a huge amount of data - slightly less than 175000 data points.
As far as I understood Terry guessed(!) a sine as being the upper limit and eliminated all data above it.
I thought about automating that process by using an outlier function (without success).
If we zoom in (see picture below) we can clearly(?) see which data should be considered an outlier/spike. I thought about some kind of windowing and applying an outlier function peu à peu instead of treating all the data in one go ....?
I had not played around with that idea any further as the OP in that thread seemed to be happy with Terrys solution anyway.
The thread just came to my mind when I read your posting.
