I took some time out from the blog to submit the videos, "ATeaser" and "A Teaser in 3D" to the Chemical Engineering space. So far, the process depicted in the videos has been identified, but I have posted some new questions. If you are familiar with chemical reactors, please join in the challenge!
Now for the subject of this posting, model validation. Whenever possible, the predictions of a model should be compared directly to the raw data obtained from experiments instead of compared to calculated experimental results. If calculated results are used for the validation, then there is the possibility that they are incorrect. Another possibility is that the method of calculating the "validation data" may actually be part of the model. An example will show what I mean by that last statement.
A university project was studying the deposition of wax in an oil pipe that was being cooled by glycol in an outer pipe, i.e. a pipe-in-pipe heat exchanger. The objective was to develop a model of the rate of wax deposition as a function of the oil properties and the operating temperatures. The double pipe experiment circulated oil through the center pipe and glycol was fed either co-current or counter-current in the outer pipe. Inlet and outlet temperature of both fluids were continuously recorded, along with flow rates and pressure drop of the oil through the pipe. In addition, temperatures were measured in the oil and glycol at numerous locations along the pipe. The thermocouples for these locations were paired...for every oil temperature there was a glycol temperature at the same location. These paired temperatures were used to compute the local wax thickness based on heat transfer resistances. The predictions of wax thickness from a model was then compared to the calculated thickness from the data.
Do you see the problems with that approach? The models being developed also included the heat transfer calculations used to generate the "data". Thus, the heat transfer calculation was in a sense a subpart of the overall model. So two wrongs (i.e. an incorrect heat transfer model being used twice) could make a right!
In order to eliminate these possible errors, I modeled the entire experiment. Instead of using the paired oil/glycol temperatures along the pipe, I used the outlet oil and glycol temperatures as the raw data. The model used the inlet temperatures and flow rates for input, predicted the deposit thickness, and also predicted the oil and glycol outlet temperatures. Pressure drop was also predicted by the model. The model validation program required the symbolic parametric solve block capability of Mathcad for a solution of the countercurrent heat exchanger portion of the model. This symbolic solution was then used in the ODE solution of the wax deposition model.
The comparisons of model vs. data for a single run are shown in the figures below for two of the models. The curves all represent outlet temperatures as a function of time. The large changes in the curves are due to changes in inlet temperatures or flow rates.
It's not hard to determine the better model. The original approach using calculated "data" could also show "B" as the better model, but the "raw" approach eliminated the possible errors mentioned above.
As I said above, I ignored the paired temperatures along the pipe. In order to use that data, I could have predicted the local oil temperature based on the model's prediction of the wax thickness using the local glycol data as a given input. This would also allow me to compare model vs. raw data. Since there is more relative error in the small temperature differences at a local site, I chose the integrated, outlet temperature approach.
Next: Using a cubic function for smooth transitions
"As I said above, I ignored the paired temperatures along the pipe."
It seems a shame to ignore real measurements. After all, it is possible to get the local distribution wrong while still getting the overall integrated value right. I guess it really depends on what was perceived to be important, but, presumably, because local measurements were made, local distribution was important. I agree though, that it's the predicted and modelled temperatures that should be compared, not the predicted and "measured" thicknesses.
You are absolutely right, we should use as much of the data as possible. My approach was an example of what could be done. There are several modifications that could have been made to my approach while still comparing to the raw temperature data.