I think there is a systemic problem here, in that I believe more missions suffer calibration difficulties than people would like to admit. Of course, if it's hard to get and use the data and one limits funding for follow-on research, then it's hard for those outside the projects' communities to really know what happened.

Sometimes instruments don't perform exactly the same in situ as they did on the ground, so calibrations have to be redone after the data come down. It's much better to wait awhile to release properly calibrated data than to release improperly calibrated data that might confuse or mislead.

The data from such missions are often still being (re-)calibrated years after launch as more data are processed, instruments better understood etc. Even though the images are being released quickly, their interpretation will also be ongoing for several generations of grad students :)

Calibration and interpretation take time and I think it's fair enough if NASA wants to work with the best possible data before making conclusions public.

Of course even then those conclusions could be overturned next week. And that's the great thing about this dynamic, living beast we call science!

"Long delay for releasing the science data?" Phoenix has been on Mars for less than four months and has a limited lifetime. I would imagine that the science team is working double-time to get done what they have to do. If they have any difficulties with their instrumentation, and almost all missions do, then exceptional discipline is required to find the path to credible results. Add on the pressure to get it right at the time of release, especially if they find something interesting, and the time squeeze becomes immense. Leave them alone and let them do their job. We will see the goodies soon enough.

Calibration issues are quite common for science experiments, so it's not implausible, but I'm not sure what in the original letter makes the listener think that the "problems" mentioned are with calibration, and not the mechanical problems we've heard about.

I don't know what putative "long delay for releasing the science data" is being discussed- typical schedule for NASA science missions to give the science team six months to validate and calibrate data; and the mission has only been on the ground a little over a hundred days so far.

I’d like to respond to the NASA Watch story titled "Nilton Renno Seems to Think That Mars Phoenix Has Calibration Issues."

I gave a seminar at MIT last week about the evidence of highly saline liquid water on Mars. I made it clear that the idea was mine and of various Phoenix Co-Investigators. I also clearly stated that the science team as a whole was still debating that subject. Indeed, I had a slide with the following disclaimer in big letters: The ideas and conclusions reported in this presentation are somewhat controversial and do not represent the opinion of the entire Phoenix team. However, I did mention that one Co-Investigator was not happy that I was giving the presentation because he did not agree with the idea, but that debates were an important part of the scientific process. Indeed, I was at MIT to get the feedback from the MIT community.

During the question period, someone stated that after seeing Phoenix press releases and presentations, that they had not seen any TEGA data. I told that individual that I had seen some data and could state that TEGA detected water. I also answered that it was because the data may not be fully calibrated. In no shape or form was any language communicated to me by anyone that would give the impression of consequences or retribution if I talked about any Phoenix problems or my somewhat scientific controversial results.

--Nilton O. Renno
Professor of Atmospheric, Oceanic and Space Sciences
University of Michigan
Ann Arbor, MI

Even in the best case, calibration is hard. Yes, there have been some missions with severe 'calibration problems', but every mission I can think of has needed years to get the calibration properly sorted out. The instrument in space is never quite the one you thought you had built, you have to amass a good bunch of data and get to be intimately familiar with its quirks and foibles before you can have confidence in what the data are telling you. That is not 'calibration problems', that's the reality of doing science.
The more nuanced version: a 'ten sigma result', one that
is leaping out of the data at you, doesn't need good calibration. A 'three sigma result', one that is just clear enough to be detectable, needs the calibration to be reliable. And the good stuff is always at three sigma - because you always build the spacecraft to be *just* good enough to do the job, or else you could have done it lighter and cheaper.
So you launch, you announce the first-cut top-of-the cream results a couple months later, and then the real work begins, which takes years.

Speaking as a scientist (and a friend and colleague of Nilton Renno), not on behalf of JPL or NASA, there are two really interesting issues here.

The first is scientific disagreement. That's what keeps the process healthy, and working out these disagreements is how we approach (but never achieve) certainty. Like the rest of the world, we exchange email on these subjects, sometimes heated. But it's important to distinguish this essential exchange of ideas from "warning emails telling him of consequences."

The second is calibration. It's one thing to calibrate, for example, a camera with exact performance specifications. It's another thing to calibrate a chemical instrument, a process that requires some knowledge of what is going to be measured. That's sort of a Catch 22 when we're going to an unknown place. We deal with this by calibrating as best we can before we launch, doing a calibration check on Mars, then doing post-experiment calibration in our labs back on Earth. For example, when perchlorate was discovered in the MECA chemistry experiment we really didn't have a calibration curve ready for it, so we had to create one after the fact by doing experiments with the flight spare hardware. This isn't unusual, and it takes time. For those of us who have been working on this experiment for a decade, that's a small price to pay.

Cheers,

Mike Hecht