Sony NEX-5N monochrom conversion results

[Alex H]

It actually seems ironic that UV photographers who really could do with a boost in sensitivity (possibly 600%) have not experimented in this direction whereas the astrophotographers...

 

 

Shane, this statemet of yours gave me a push to finish my severely delayed experiments, which I started many moons ago. Thank you! I finally have semi-functional NEX-5N with "de-bayered" sensor (CFA removed).

 

Over past few days I have been testing and comparing my NEX-5N (monochrom) and NEX-6 (full-spectrum) under different conditions (clear sky, overcast) and with different objects (urban landscape, flowers) in order to access the exposure difference between the two. I used both quartz singlet and UV-capable enlarger lens with Baader-U2 filter. Internet sources (not SONY!) suggest that both cameras use the same sensor IMX080, but I have no way to confirm that. Nonetheless, while shooting full-spectrum modified NEX-5N amd NEX-6 in the past, I have not noticed any considerable difference in UV response between the two. Therefore, I hope that my comparison is rather informative.

 

After comparing the NEX-5N (monochrom) and NEX-6 (full-spectrum) I can say that the increase in UV sensitivity for the monochrome camera will be about 5-6 times more, or 500%-600%, or 2.25-2.5 EV (depending on the conditions, and the quirks of the RAW processing software). My findings, thus, agree with what Shane wrote in his post quoted above, and with this statement of Dan from MaxMax - "Our UV-VIS-IR cameras with the CFA have about 1/6 the UV sensitivity of monochrome UV cameras." (see here: http://www.maxmax.com/our_solution.htm).

 

Unfortunately, I have no means to properly analyse the information preserved in RAW files, and have to rely on the files (and histograms) produced after the RAW conversion. This is what I did: I opened RAW files using same exact software, set WB separately for "full-spectrum" and for "monochrom" cameras, de-saturated the shots and then compared the histograms. I know it is rather crude, but the RAW converters I have are not designed to properly "understand" pictures taken with modified cameras in UV. Therefore, you must treat this information with caution. Also, if anyone wants to analyse RAW files themselves using specialized software, I can provide such - just ask here.

 

And now, two example shots. Both taken at the same time (with few minutes interval), under the same lighting condition (sunny clear day), same lens, same aperture, same ISO. Please note that histograms presented in the pictures are based on the shown processed and cropped TIFF files, not on the RAW files. Nothing was adjusted in the files. Difference in contrast you can see is caused by the difference in acquisition of information by R, G and B sensils between sensors with and without CFA.

 

NEX-6 (full spectrum) - shutter speed = 0.6 second

http://www.holovachov.com/img/s6/v134/p51681238.jpg

 

 

NEX-5N (monochrom) - shutter speed = 0.1 second

http://www.holovachov.com/img/s6/v142/p462520787.jpg

[colinbm]

Thanks Alex for doing this.

I am surprised that the 'full spectrum' photo is sharper ?

The 'monochrome' photo shows some dust bunnies, that are not a problem in themselves here, with this testing. But I am wondering if there is some smearing showing on the sensor that is taking the sharpness off the image ? Perhaps a solvent clean of the sensor will brighten things up ?

You have shown that the exposure time is very much faster.

Cheers

Col

[nfoto]

What aperture/lens is used here? That will set the observations in a better context.

 

Both images appear quite sharp as shown here.

[Alex H]

Colin,

 

As I mentioned in original post above, the major difference is the contrast between the two images. The image taken with "monochrome" camera has lower contrast, which results in lowed acuity of the downsized JPG images posted here.

 

Bjørn,

 

Both images are taken at F/8. But I would not judge sharpness based on these downsized shots. Comparing sharpness and resolution between "monochrom" and "full-spectrum" cameras was not part of the test. One needs to do that under different conditions. I have not had a single windless day since I moved to Stockholm. If there is an interest to compare sharpness between the two cameras, I have to do it indoors, with controlled lighting.

[colinbm]

Thanks Alex

I just felt something else was holding the monochrome image back.

Cheers

Col

[Alex H]

These are test shots, made to be as comparable as possible, with no unnecessary adjustments made. Increasing contrast and sharpness in the post-processing will make them look "nicer", but will devalue their usefullness for comparison.

[Alex H]

Sorry, Bjørn, I missed your question about the lens. It was Focotar-2 50mm F/4.5, with rear-mounted Badder-U2. According to old post by Klaus on NG it transmits down to at least 320 nm. It is very sharp lens in UV.

[Shane]

Alex

great effort! Nice to see you can confirm the 500-600% increase in UV sensitivity. If you get a chance could you compare monochrome 830nm IR with visible, I suspect there will be little improvement at that end of the spectrum, since the Bayer dyes are already transparent to IR.

 

The lower contrast of the monochrome converted image compared to the Bayer image appears consistent with the finding of others who have produced similar tests, specifically Leica M8 vs MM.

 

A couple of things spring to mind and it depends on what RAW converter was used.

 

Did your RAW converter demosaic and interpolate the monochrome image - this would degrade resolution and may produce classic de-Bayering type artifacts

 

WB should really be UniWB otherwise the RAW converter will be multiplying luminance values with WB correction factors.

 

In visible light photography, you will likely regain your contrast through the use of classic B&W filters, after all you just removed the Bayer array "micro-filters" that were inadventently helping with this.

 

Another issue I can think of with regards to not demosaicing/interpolating is that the image will likely "fall apart" quickly if the post processing is pushed too hard e.g. stretching the contrast. If your image was demosaiced then this should not be a problem but at the loss of resolution instead.

 

In addition, exposure based on the histogram should be more predictable since all channels are responding the same (unless your WB factors are coming into play) however, unlike RGB images, overexposure will not be recoverable.

 

Not sure I can view true RAW data for these Sony images but is there any chance I could obtain the two RAW files (or 4 RAW files if you get the IR comparison) to try?

[Shane]

Alex if you use a Mac you could try Accuraw Monochrome software, to process you images. Actually, what did you use?

[Alex H]

Thanks Shane,

 

For this simple comparison I used Sony IDC RAW converter. It does interpolate the data, and does not offer UniWB. But it is the only RAW converter I have that will preserve in-camera pre-set WB for the UV shots and that is why I used it. That is why I had warned to treat this data with caution. I would be happy if someone else can verify my preliminary results using different, more suitable software.

 

I also have Capture One, but there I have to correct WB for UV shots after importing pictures - it automatically adjusts WB of all pictures during import. I also have RPP (Raw Photo Processor) and AccuRaw Monochrome. Supposedly, RPP can process images without demosaicing but I do not seem to be able to make it work like that.

 

AccuRaw Monochrome works fine with my files, but I would not judge the resolution based on these shots - it was sunny but windy, and flower movement was unavoidable, especially for the "full-spectrum" shot. If I have time, I will compare both cameras, and may be add unmodified NEX-6 in the "studio", using controlled light (blue?). Interestingly, AccuRaw has a box "Sensor CFA removed", marking which lowers the exposure of processed file considerably.

 

I do not know how much "cooking" every single of these programs do "under the hood", so the best would be to analyse RAW data with dedicated software, which I do not have. Shane, PM me your email and I will forward you UV files. IR will have to wait - I can not find my B+W 093 filter anywhere...

[Alex H]

I just remembered that RPP has UniWB. Applying UniWB to "monochrom" shot does not change much from the picture under "Auto WB", setting. Applying UniWB to UV shot produces strange colors and pushes the exposure up. Is it normal with UV shots???

[Shane]

UniWB is better suited for looking at the RAW histogram data to see what it really looks like.

[Shane]

It would be interesting to here of your de-bayering technique and the key tough issues you encountered.

[Alex H]

Purely mechanical. One has to have a very sturdy hand. Years of work with microscopic animals help a lot.

[nickspiker]

FYI, I have a raw converter for most raw files that will output straight 16 bit un-debayered data.

And, do you have any more details on how you remove the CFA? And I'm assuming the microlenses come off too.

[Alex H]

FYI, I have a raw converter for most raw files that will output straight 16 bit un-debayered data.

 

RawDigger? I am using it right now and will update the post with the results from it. But for many uses, AccuRaw Monochrome or even dcraw will suffice.

 

And I'm assuming the microlenses come off too.

 

Aren't microlenses on top of CFA in a consumer-grade mass-produced sensor? I do not see any way to remove CFA without affecting microlenses in any way.

[colinbm]

"un-debayered data".........what does this mean.......normal Bayer output ?

 

DIY CFA & microlens removal........***not for the faint hearted*** :)

 

 

Cheers

Col

[Alex H]

"un-debayered data".........what does this mean.......normal Bayer output ?

 

I assumed he meant "un-interpolated", but now I am not sure...

 

Here are exposure histograms produced by RawDigger http://www.rawdigger...nual/histograms

 

NEX-6 (full spectrum)

http://www.holovachov.com/img/s5/v126/p661612776.jpg

 

NEX-5N (monochrom)

http://www.holovachov.com/img/s5/v129/p712923441.jpg

 

As you can see, green channels have practically identical exposure, but the red and the blue channels are different.

[Andrea B.]

Alex, an excellent effort and report on the UV Monochrome versus UV Colour topic.

 

Was the desaturation step a straightforward desaturation with no weighting of R,G and B to mimic the human eye's luminosity function? I expect so, but just want to confirm.

 

How did you have your full-spectrum NEX-6 set in its in-camera Picture Controls (or whatever they are called for a Sony NEX)? Did you use an in-camera colour setting or a monochrome setting? This can make a difference in exposure times *if* using the (inaccurate) histograms to judge exposure because a colour channel can hit the right wall (oversaturate) before the luminosity histo does. And when we see a colour channel hit the right wall we tend to stop pushing the exposure for fear that we cannot pull it back successfully in an editor.

Anyway, the base of this question is how an in-camera colour versus mono setting might affect results?

 

Applying UniWB to UV shot produces strange colors and pushes the exposure up. Is it normal with UV shots???

Changing white balance in some editors may indeed change the "exposure" sliders because white balance occurs on the combined luminance & chrominance layers. This is of necessity because of the way the human eye sees. In ordinary visible colour shots a white balance step typically changes exposure very little. With some UV shots the change may be much larger depending on what initial white balance one starts with in-camera. Some editors compenstate for the change in white balance with a simultaneous exposure change, some do not.

 

Specific example: make a UV shot with Nikon D200 full-spec cam using an in-camera Incandescent white balance, and you get lots of reds and magentas in the shot. Perform white balance on a magenta area in Nikon Capture NX2 to produce blues and greys, and you will see the exposure darken considerably. NX2 does not alter exposure when white balance is changed. Do the same experiment in Photo Ninja and you will see less change in exposure because - apparently - PN tries to compensate for the white balance change.

Reference: http://en.wikipedia.org/wiki/Luminance_%28colorimetry%29

[nickspiker]

"un-debayered data".........what does this mean.......normal Bayer output ?

 

DIY CFA & microlens removal........***not for the faint hearted*** :)

 

 

Cheers

Col

Oh dear, that looks terrible.

Is that the way you did it Alex H?

[nickspiker]

"un-debayered data".........what does this mean.......normal Bayer output ?

 

DIY CFA & microlens removal........***not for the faint hearted*** :)

 

 

Cheers

Col

And, yes, I mean straight raw data that hasn't been interpolated, gamma corrected, or anything. dcraw- which most of you are probably familiar with.

[Alex H]

Andrea

 

Was the desaturation step a straightforward desaturation with no weighting of R,G and B to mimic the human eye's luminosity function? I expect so, but just want to confirm.

 

Desaturation was straightforward for the original histograms posted in the first post. However, the histograms produced by the RawDigger in the second post suppose to represent real exposure values of the raw data.

 

How did you have your full-spectrum NEX-6 set in its in-camera Picture Controls (or whatever they are called for a Sony NEX)? Did you use an in-camera colour setting or a monochrome setting? This can make a difference in exposure times *if* using the (inaccurate) histograms to judge exposure because a colour channel can hit the right wall (oversaturate) before the luminosity histo does. And when we see a colour channel hit the right wall we tend to stop pushing the exposure for fear that we cannot pull it back successfully in an editor.

Anyway, the base of this question is how an in-camera colour versus mono setting might affect results?

 

The "full-spectrum" camera had pre-set WB with PTFE target saved, but, again, it does not matter. I did exposure bracketing. First I took an exposure with the the "monochrome" camera, than set the same exposure value on the "full-spectrum" camera and took a series of shots increasing exposure one step at a time until it was way overexposed. Than I matched the exposures of the shots by comparing the histograms produced by the RAW software. Please compare the histograms I produced with RawDigger - exposures in green channels are very close between the two shots, red channel is overexposed 1 stop, while the blue channel is underexposed 1 stop in the UV-picture taken with the "full-spectrum" camera..

 

Applying UniWB to UV shot produces strange colors and pushes the exposure up. Is it normal with UV shots???

Changing white balance in some editors may indeed change the "exposure" sliders because white balance occurs on the combined luminance & chrominance layers. This is of necessity because of the way the human eye sees. In ordinary visible colour shots a white balance step typically changes exposure very little. With some UV shots the change may be much larger depending on what initial white balance one starts with in-camera. Some editors compenstate for the change in white balance with a simultaneous exposure change, some do not.

 

(EDITED) Andrea, I know how white balancing works, and how it changes exposure values in different software. It can even be different when different camera models are processed wit the same software. (EDITED) The question was addressed to Shane who suggested to use UniWB to process RAW files and than compare the histograms.When I applied UniWB to the monochrome shot, the exposure values did not change, but when I applied it to the UV picture shot by the "full-spectrum" camera, histograms changed considerably. Therefore, I was first not sure if UniWB is actually an optimal way to judge the exposure. But after I compared the histograms produced by RawDigger and histograms produced by RPP using UniWB, they turned out to match closely.

 

Again, it does not really matter, if we trust that the RawDigger produces histograms based on raw data from the file, because the WB settigns set for the picture in RawDigger do not affect histograms. To confirm that, I did try to change preset WB settings in the RawDigger and see if it changes the histograms, and it did not. Contrary to other RAW converters, which produce histograms based on WB-adjusted and interpolated data.

 

I hope I was clear, but if not, please ask.

[Alex H]

Oh dear, that looks terrible.

Is that the way you did it Alex H?

 

No. But I will not give the details, as I do not want to be blamed later on if someone tries to do the same and destroys the sensor. It is a bit more precise job than converting the camera to "full-spectrum". Besides, as it has been discussed on the internet, different camera brands, and different camera models within the same brand, have microlenses and CFA of different hardness, and require different tools. There was nothing new in how I did it. In fact, other people de-bayered their cameras in the similar way and published it on the internet. Google search will bring up all those discussions. All I can say is that having a good microscope is extremely useful for this type of work. And steady hands.

[Andrea B.]

Alex, thank you for the details on your exposure procedure. That helps me understand the variables of the experiment better. :)

 

Can you set your NEXi to uniWB in-camera? UniWB is indeed useful for judging exposure.

(I missed the import of your original question.)

 

(edit: added "via NX2")

UniWB via NX2 is unfortunately no longer possible with newer Nikons. They do not want to read a file which has been uni white balanced in the editor for using as a wb choice in-camera. A while back Shane and I in separate efforts were trying to create a uni wb file which a Nikon would read by tweaking a file's RGB coefficients via editing the hex so that there would be no edit flags. I kinda gave up on that. Shane any updates there?

 

The point of the experiment is made however. A mono conversion for UV gives shorter exposure times. Surely useful when shooting UV.

[nickspiker]

 

 

No. But I will not give the details, as I do not want to be blamed later on if someone tries to do the same and destroys the sensor. It is a bit more precise job than converting the camera to "full-spectrum". Besides, as it has been discussed on the internet, different camera brands, and different camera models within the same brand, have microlenses and CFA of different hardness, and require different tools. There was nothing new in how I did it. In fact, other people de-bayered their cameras in the similar way and published it on the internet. Google search will bring up all those discussions. All I can say is that having a good microscope is extremely useful for this type of work. And steady hands.

Sounds good!

I'll probably wait until http://www.sonyalpharumors.com/sony-to-launch-a-full-frame-black-and-white-sensor-camera/ mostly because it'll probably have micro lenses and be a bit cleaner and less risky, assuming it materializes.