Why is polynomial estimator variance so large?
Anyone with experience in energy selective imaging is struck by the terrible performance of polynomial estimators discussed in my
last post. This is most likely due to the fact that in the past the number of spectra was almost always equal to the dimension of the A-vector. In this case, as I showed in the last post, any estimator that solves the deterministic, noise free equations is the maximum likelihood estimator (MLE). With equal number of spectra and dimension, the polynomial estimator is accurate for low-noise data so it provides an ’efficient’ estimator. That is its covariance is equal to the Cramèr-Rao lower bound (CRLB). In this post, I examine the reason for the poor performance with more measurements than the A-vector dimension.
Polynomial estimator when number of spectra equal the A-vector dimension
We can see the root cause of the problem from the coverage of the calibration data for noisy data. Fig.
1↓ shows the two measurement spectra—two dimension A-vector case. The black dots are the calibration measurements and the red dots are the measurements with noise for a single A-vector. Notice that the noisy data fall within the calibration region.
Fig.
2↓ shows the estimates for data on the blue line in Fig.
1↑ The blue curve in Fig.
2↓ is the iterative MLE output and the red curve is the polynomial estimator results. In this case, the outputs are equal verifying the theoretical result from my last post that the polynomial estimator is a maximum likelihood estimator. There are many ways to implement the MLE but, as long as they solve the deterministic equations, they will give the same results.
Polynomial estimator with number of measurement spectra greater than A-vector dimension
The three measurement spectra case is shown in Fig.
3↓. Now the calibration data fall on a two-dimensional surface in the three dimensional measurement space. Deterministically there is no solution for measurements outside the surface. For more measurements than the A-vector dimension, the noisy data, again shown by the red dots in Fig.
3↓, can be off the surface. See the edge-on view in Part (b) of the figure.
Polynomial fit outside the calibration region
Fig.
4↓ shows what happens when we try to use a polynomial fit outside the data region. The figure shows a ninth order fit to the rect function. The fit, plotted as the red line, was calculated with the data from [-1,2]. It gives a fairly good approximation within this region but it diverges rapidly when we try to use it outside this region.
Compare MLE and polynomial estimators away from calibration surface
Fig.
5↓ shows the analogous effect for the polynomial estimator. Again the polynomial estimator is the red line and the MLE is the blue line. The polynomial estimator gives the correct value on the calibration surface but rapidly diverges away from it. As a result random measurement data that are not on the surface will produce large values resulting in a much larger variance than the MLE, which gives reasonable results off the calibration surface.
Discussion
An important difference between the MLE and the polynomial estimator is that the MLE uses the known probability density function of the measurement data. With this information, the MLE is able to perform reasonably with data that are off the calibration surface and therefore have no deterministic solution. The polynomial estimator does not use this information and based on its mathematical properties it diverges away from the calibration surface. This is responsible for its variance being much larger than the MLE and therefore much larger than the CRLB.
—Bob Alvarez
Last edited Oct 18, 2013
Copyright © 2013 by Robert E. Alvarez
Linking is allowed but reposting or mirroring is expressly forbidden.
References