I really dislike the current method used on orbital and lander
spacecraft for color imaging. It consists of taking separate images
through three different visible light color filters representing Red,
Green, Blue light and combining them into a single color image.
The problem is calibrating the combination of these images taken
separately. So we have a spacecraft in orbit about Mars in Mars
Odyssey supposed to be able to take color images but there is so much
uncertainty in the combination of the colors that we've only had a few
visible light color images released.
And we have two lander spacecraft on Mars supposed to image in color
and each color image release creates controversy in the accuracy of
the color combinations used. It makes you long for the simple color
video cameras used on the Apollo moon missions.
The reason this method is used is that by using all the pixels in the
camera for a single color range you can gather more data in that
frequency range. However, there has been a method developed that
allows you to collect the same amount of data using fewer numbers of
pixels that might finally allow us to collect the full color range
simultaneously as with color video cameras:

Spider Eyes For Martian Robots
by Anil Ananthaswamy
San Francisco - March 28 2001
"The vibrating eyes of jumping spiders have inspired a new breed of
vision sensors that could give the next generation of Mars rovers
sharper eyesight, say researchers in California."
http://www.spacedaily.com/news/mars-general-01d.html

Visual sensor with resolution enhancement by mechanical vibrations
Koch Lab
Ania Mitros and Oliver Landolt
http://www.klab.caltech.edu/~ania/re...ib_retina.html



Bob Clark

Ooops. The Apollo color cameras were also field sequential color cameras. They just had one single tube and a color wheel in front with three color filters. The camera transmitted sequentially filtered images which were again combined into color ones on earth.

Not the best example for your case ;-)

Harald

PS: The ALSJ is currently collecting up lots of old documentation on the Apollo cameras. Have a look at http://www.hq.nasa.gov/alsj/alsj-TVdocs.html

__________________
"Flying in space is risky business, but just staying on this planet is risky business too." - John Young, astronaut

All digital color cameras, even consumer cameras, work this way. Digital cameras by definition take black-and-white images. Unless you're using film, you're going to have combine images.

Hmmm... It's a scanning imager with overlapping scans?.. Scanning imagers have been around since before array based imagers, but the weight, power need and complexity would favor arrays. There are of course some scanning imagers used, like the one used in Mars Express. But this is a passive system, it uses several 1D arrays, and the movement of the craft provides the other scanning direction.

Of course the circular scanning has some interesting possibilities, but the question is if the higher weight, larger size and power need might still prohibit it's use. Anyway, it will not have much to say for color quality, as this system would have to use one of the conventional approaches to color filtering. No camera will create images exactly like how you would see it, in fact our eyes are extremely nonconsistant in their imaging, a scenery may look colorful or bleak, redder or bluer all depending on how your eyes levels of photosensitive chemicals are, still it is as true color as it can get...

__________________
Game over, you lose, we hope you enjoyed playing the exciting game of Thermodynamics...

Hrmm, since this has been brought up...

I've always wondered why they don't have some sort of white calibration reference in the photos. Through the filters, it'd look red, blue, or green. Combining the three images and adjusting until the reference was white should give you true colour, shouldn't it? Of course, if it were so simple, it'd be that simple.

__________________
"I'm making wheatloaf. It's like meatloaf, only with wheat"
"Isn't that just...bread?"

Analog video cameras have separate pixels for R,G,B arranged in triplets so they are all exposed at the same time.
Are you sure digital video cameras take separate filter images and then combine them?


Bob Clark

Hmmm... Well, White balance is used to calibrate digital and video cameras, but I wouldn't say it gives true color. When I see a sheet of paper, I perceive it as white, but it doesn't necessarily look pure white to me, it will have a certain color temperature, it looks orange under low power incandesants, while it looks slightly yellow under higher powers. If you were to balance the color levels so that it was perfect white, it would no longer be true color. In environments where the light has different color, it may be different. And even if you had a perfectly calibrated image, you would have to calibrate your computer too, as there are differences in different screens and video cards. I guess you would need colorimeter, densitometer, spectrophotometer or some such device... But unless you do photo developing or printing, I guess that is a bit overkill :wink:

In addition to what I mentioned in the earlier post, the eye also seems to use a relative approach to imaging, concentrating on differences, not absolute color values, so what your eyes see might not be what a color really is...

__________________
Game over, you lose, we hope you enjoyed playing the exciting game of Thermodynamics...

I guess I could try to answer this... It really depends, some cameras(digital and analog) uses three separate image sensors, each with its own filter. another approach is to use a single imager, but moving filters, this is similar to the system used on the rovers. The most common in digital cameras though, is to create the sensor with a RGBG filter(some use CMYG, but it is the same principle), that is, coating the surface of the sensor with filter pigments like this:

RGRGRGRG
GBGBGBGB
RGRGRGRG
GBGBGBGB

There are many different ways of doing something of course, you could make a camera that used one imager to create the brightness channel, and another to create a red-blue color map, and combine them to make RGB images. The RGB image tubes used in some cameras was likely to be something like a reversed CRT screen, with three different electron beams scanning the light sensitive surface through a shadow mask.

But no matter how you filter the light, you will have a few images that must be combined, encoded and stored on medium, though not necessarily in that order.

__________________
Game over, you lose, we hope you enjoyed playing the exciting game of Thermodynamics...

People do stick colour calibration charts on these beasties. (The Damian Hurst one on the ill-fated Beagle 2, for example.)

What your method does is correct to apparent reflectance, which is more directly related to material properties than "true colour", and hence in some ways, more useful.

Hmm, they were able to do that even though the images were transmitted semi-live?

Bob

Where do you see a problem?

__________________
"Flying in space is risky business, but just staying on this planet is risky business too." - John Young, astronaut

With the astronauts moving and the Lunar Rover moving I would have thought that would have created blurring.


Bob

That's exactly what happened. When something moved quick in front of the camera, you got color borders.

__________________
"Flying in space is risky business, but just staying on this planet is risky business too." - John Young, astronaut

In my opinion, the best way to take accurate color images would be to photograph the object with a high bit depth grayscale camera through as many filters with known properties as possible. That way you can reconstruct a more or less accurate spectra of the entire image. Once you have the spectra, you can always render it in whatever color space you want.

Oh, and as it happens, that's how it's done on the MERs...

The uncertainty is there because the full data hasn't been processed to pull out the relative exposure times and combine the images properly. So the real problem is they are releasing the images too early. ;-)

The reality is it will take time for them to sort the data and publish the exposure times, but they will. When they do, then it will be a lot easier to combine the grayscales for "true color" images. Of course, there's still the matter of which filters are used. One of the filters that is often used for Red actually extends into infrared, so that skews the image content slightly.

If you want to take the image samples simultaneously, that's fine, it would eliminate the step of waiting for the exposure time data to be processed and made available. But that still doesn't eliminate the problem of picking your filter ranges.

You could have a camera capable of taking RGB pictures, but you would still have to find out what to think of as true color for calibration... And of course, it would have many disadvantages... The most important would be that the channels of the imaging would not utilize the full grayscale range; one would have to maintain either the absolute intensity of all channels or the relative, so you would loose detail compared to grayscale optimized images. This reduction in grayscale resolution is not easy to undo by post processing the image... This has been discussed on several earlier threads.

Some people may think that sticking something like a standard digital camera on a rover would be the solution to this... But of course, consumer digital cameras are calibrated to lighting conditions faced here on earth. Also they make a picture that is not true color at all, though it is within the brains tolerance, and it seems easy to calibrate the images further. But for images of an alien environment, it is not that easy, you don't have personal experience(or the camera developer teams experience rather) to lean on. If someone shows you a picture of an environment you are familiar with, you may get a feeling of how close it is to reality in color. But if you are not familiar with it, you couldnt do this, you might even feel that the colors in the image must be wrong, even when they aren't...

__________________
Game over, you lose, we hope you enjoyed playing the exciting game of Thermodynamics...

That was the purpose of suggesting the vibrating sensors method.


Bob Clark

As far as I understood the text, this is a system for improving the X-Y resolution of the finished picture, so that you with a 32x32 pixels array could create pictures of equal or better resolution than using a conventional array based imager, for example the 1024x2048 arrays used in the PanCam system of the rovers(they only use 1024x1024 for actual imaging though, the other half is used to buffer the image during frame transfer). The imager on the site you linked to is a variation on the scanning imagers used in some space probes, some probes used a single light sensitive element, a photomultiplier tube, so it imaged one pixel at a time, and used mirrors and stuff to rapidly scan the area. It is not so popular today, because it had higher mass and used more complex mechanics than arrays, though some probes do use a variation, single line arrays, and let the spacecrafts movement provide the other axis, this is likely why some probes create images that is of fixed width along one axis, but much longer along the other...

But this is not really the problem with color rendition in the final image, that would still have to be done in the same way as the system used today, with some sort of filters. Scientists would still want to use the full grayscale range of each channel, as that improves the detail you can get, so you would still need to post process each of the grayscales if you want something near true color. And you still have the problem of defining how to calibrate the imager system so that it gives true color.

And you'll need some way to calibrate your screen/printer, something like a densitometer, spectrophotometer or colorimeter, to ensure optimal color rendition. Still, it may not be true color, as the range of colors you can make with a RGB triplet is smaller than what the eye can see.

As the eyes are not stable in how they see colors either, true color can be a quite wide spectrum, it is hard to say...

__________________
Game over, you lose, we hope you enjoyed playing the exciting game of Thermodynamics...

I'm not so sure about that. The reason why they don't have 3 different pixels for each of R, G, B is because that reduces the resolution. If this system made the reduction in resolution a non-issue I'm not so sure imaging scientists wouldn't opt for this. Note that with the current system there are days before you can release a full color image. With simultaneous exposure you could release it as soon as you could release any of the single color image.
The ideal solution though would be a system that could change each and any pixel to any frequency range you chose on command.


Bob Clark

Actually, I would say using an array with a RGBG filter you would need a CCD array with higher resolution, but that is not the real problem, these arrays are quite common, almost all digital cameras use them. But they introduce artifacts along the edges of objects in the picture, since the sensor for each color will be hit by light coming from a slightly different angle. Also the problem is that you would want more than three frequencies for scientific reasons(it can help you identify minerals, for example), and want to use filters with tight band pass characteristics that is stable over the entire sensor surface, it is much easier to do in a single filter.

Using a digital color camera to take your holiday pictures, for example, is not a problem, you use them to help your memory of it, not to analyze what everything is made of, the loss of information due to the fact that the camera must maintain the relative levels of the red, green and blue channel is not usually an issue. But sending a lander to another planet is not a holiday excursion, you don't do it to make eye candy images, you do it to learn, to study that world, and so you would want as much information packed in to the data as possible, but use the minimal amount of bandwidth possible. Reduced detail due to limited use of the grayscale, can not be undone in software, if you have a dark picture using only 128 levels of 4096, it will only use 128 levels no matter how you try to increase its brightness...

Anyway it does not matter, as the problem with generating color images have nothing to do with taking three simultaneous or three sequential images, both these methods have been used in cameras on earth for decades. The two ways are essentially the same, you get three separate grayscale images the electronics must integrate into one image.

Seeing as you can not use multiple(more than 3, I mean) filters easily with arrays where the filters are on the surface of the sensor, and that using a separate sensor for each channel and using dichroic mirrors to split the light makes the system rather heavy, using a single imager with exchangeable filters is the best trade of, that you can not easily image something moving rapidly across your field of view, is not a big problem.

__________________
Game over, you lose, we hope you enjoyed playing the exciting game of Thermodynamics...