In camera processing RAW v JPG comparison
 

As part of my investigation into missing EXIF information I have been processing images using Nikon Capture, Paint Shop Pro versions 7 & 8. I have been using PSP7 a lot as I am familiar with it. On a couple of occasions I have used PSP8 having needed to do a straighten operation.

I have discovered that any image I do a ' save ' or ' save as ' in PSP7 looses the EXIF. I am going to upload a test image from today to my gallery, that has been cropped and saved in PSP8 to see how much of the EXIF has survived. Nikon Capture suggests some may have gone.

As an aside to that little exercise I thought I would compare how the camera, Nikon D100 processes JPG Fine in comparison with RAW.

On the market today I bought and old mic for a couple of quid ( £'s ). As it has an engraved scale I thought it would make a suitable subject for a sharpness check. After converting it into a fancy paperweight, the mic I use proved a better bet for the comarison, so the attached image is a composite of a RAW converted to JPG and a straight JPG. Apart from cropping NO changes have been made, to give a realistic comparison. A full frame image is also attached.

In the case of the D100 Raw converted to JPG proves to be significantly sharper than JPG fine.

Don

Don thanks for posting the photo's, it really shows up the difference of the in camera jpg against the raw converted jpg. As separate photo's I would have said there is very little difference but side by side like that it really shows. I am glad I shoot raw.

Much is made of the RAW vs JPEG debate. I think the results are very much down to what parameters are used in the raw convertor and the co-efficients used for the jpeg encoder. These you may or may not have control over in your software or camera.

A simple test. The eye of a small rusty sewing needle was photographed with the 20D set to produce RAW and Fine Jpeg from the same shot. No fancy lighting just the builtin flash. I used Canon EOSviewer to convert the RAW to an 8 bit tiff. It has no setting you can play with really. I also tried PS CS to convert but no real difference.

A 100% 300 x 400 crop was taken from jpeg and raw versions and put side by side in a new image. No sharpening was applied to either.

So the question is which of the 2 needles is the RAW, left or right ?

Just to prove there is a small difference the second image is the 2 merged together in exclusive mode.

In the case of Don's Nikon test is it that the fine jpeg is poor or the raw convertor sharpens ?

Nikon Capture RAW processing software should process images in exactly the same way as a Nikon camera. If a Nikon camera has five sharpness modes then these same shaprness modes and terminology are present within Nikon Capture (NC). Without any changes I would expect the same results from NC as the camera.

Unlike other RAW processing software NC has no default, it simply reads all in-camera settings and applys them to the 'RAW' image itself.

The difference in the two techniques is procesing a 12-bit image (NC) and perhaps the camera converts to 8-bit first for faster processing with less memory requirement, hence a sharper RAW converted image.

So your not going to hazzard a guess which is the raw version then Stephen ?

This experiment I have done several times with the 20D and 300D and the difference between fine jpeg and raw is marginal if you have a well exposed image. In my case I don't think it is worth the investment of extra storage to process every shot as raw. The point I am making is that those who only use jpeg should not feel like second class citizens or they are missing out on a big improvement. Raw does have a bigger advantage if the shot was taken with incorrect settings to correct the error or you generate 16 bit tiff for printing. However I have found that raw is not bullet proof in saving bad images.

Rob the reason for the post was that I simply did not recognise that there was a difference. Obviously as technology advances things change.
In this case the D100 is several years old and therefore one can expect things to have changed in the mean time.

As I do not have a modern computer ( 5 years old laptop ) 128mbs ram 30gig hard drive, I chose to use the fastest photo editing program and workflow without totally understanding the consequences of that decision.

Although I went digital when the D100 came out, photography was a long way behind third hobby at the time. So I was mainly happy snapping on program mode at Steam Rallies.
This forum has changed things elevating photography to current no.1. This has made me relook at how digital works and just what the camera is capable of.

Events over the last couple of weeks have put any kit upgrades on the back burner.

Knowing that really good deals can be had on cameras like the D100 I thought this worth posting. I did a print test using Genuine Fractals, of the F2AS in my gallery and printed a slight crop over 6 A4 sheets ( 21 1/2" x 24" ) so I have no quality issues with it.

And no, I did not know which was which in your post.

Don

In the case of the micrometer it is obvious the raw image has had more sharpening because when you start enlarging it it has more edge halo than the jpeg so this particualar test is inconclusive...most people convince theirselves of what they want to see :)
By the way if you turn off or reduce the contrast and sharpeness parameters and set your own post process in PS the difference between raw and HQ jpeg lessens even more and you still have the bonus of more images on a card. Saying that I mostly shoot raw then save as HQjpeg after all processing is done.
You have to convert to 8 bit for printing so this is not really a reason to shoot in raw, the advantage of raw and 16 bit is where extreme post processing will take place and where banding postrization may take place.:)

I keep forgetting when I refer to print folk take this as meaning printing to an inkjet. Yes I did mean top end comercial work.

30 Gig Hard drive now I call that luxury Don :)

A year ago I was using a 333Mhz P2 processor. Now using a 1.5 GHz AMD 20 Gig Drive. ( Donated by younger son who upgraded his motherboard ).

Prospect of another upgrade to 2Ghz ( Donated by elder son ).

I think it is worth trying Raw to see what difference it does make for you in terms of improved quality.

I'll let you know in a few days time which of the two is the raw version.

After reading and viewing the photo of the needle I am still not convinced.
I still feel comfortable shooting Raw and don't mind all the processing involved. When I got my first digital slr I battled with trying to decide what to use for the best then came the final decision to use raw as I felt I had more control.

As an inexperienced photographer, I have convinced myself that shooting RAW gives me a better chance of rescuing some of my poor attempts, particularly shots taken in poor light such as we have been experiencing lately in the UK.

Or am I deluding myself?

I have found this to be quite interesting and decided to do another image with colour in case any differences showed up there.

The chosen subject was a toasted bacon and tomato sandwich ( Duncan influence here ) .... but .... I ate it during the set up. :D
Moral appears to be have a good breakfast before using food as a subject.

So I decided to look a DP Review as now no bacon left.

As the timeline of digital development I looked at and compared the D100, D70, and D50 as they are all around 6 megapixels.

There is in each case a difference between RAW and JPEG but as time has moved on so the differences have become less but it is still there.
In the case of the D100 ( Announced Feb02 ) the sharper Raw conversion is put down to the greater processing power of a PC.
The D70 ( Announced Jan04 ) converted RAW image shows greater contrast and colour saturation than straight JPEG.
The D50 ( Announced April05 ) shows little difference.

The D2X ( dream machine ) converted RAW appears to show better contrast and colour saturation than JPEG.

D100 review http://www.dpreview.com/reviews/nikond100/page11.asp
D70 review http://www.dpreview.com/reviews/nikond70/page11.asp
D50 review http://www.dpreview.com/reviews/nikond50/page12.asp
D2X review http://www.dpreview.com/reviews/nikond2x/page13.asp

Don

I tried my own test but just for sharpness. I used a recent Grays of Westminster price list as it was the first thing printed to hand.

Using a tripod in conjunction with self timer mode to avoid any shake, no flash and Nikon Capture (NC) as RAW convertor.

50mm f1.8 on D2X set for Aperture priority f8 and ISO 100 to produce RAW and JPEG fine (opimum quality)

Pic 1 shows full EXIF from NC and the focus point.
Pic 2 shows JPEG conversion of full photo in fine (otimum quality)
Pic 3 shows NC Conversion of full photo NC4.04
Pic 4 shows JPEG 100% crop around focus area
Pic 5 shows NC 100% crop around focus area

Please note, in camera sharpness set to medium low and full photo shots are set to JPEG level 9 to get under 341kB filesize.
Continued...

Here is a composite of the two 100% crops, are the labels correct? My eyes are getting tried after a long day in Central London, again!

I thought there was a big difference but not so sure - tired eyes!

I'd put the jpg ahead? Not much in it anyway... I'll stick to jpg for the time being.
cheers,
Andy

When I did the composite I thought JPEG left and RAW right but got dazed and swapped them over. I am now convinced RAW left and JPEG right, contrary to the labels!

There is not much in it but when you compare the relevant images on DPreview differences look about the same.

Please note, photos on post 12 are faithful and correct.

So it does appear that as digital has moved on the differences have as good as disapeared. Would probably get the same result from the new D200.

Curses this new kit is good.

Don

Thanks for confirming my results. Many have made the point that Raw is clearly sharper than jpeg. I've been puzzled by this as my tests always showed hardly any difference. On another test I did the jpeg showed a slight cluster of a few pixels on a bright highlight.

FYI the needle on the left was the RAW version.

The logic of the jpeg encoder is to first compress colour and preserve brightness and resolution. Then progressively reduce the number of brightness levels and resolution as the compression factor is increased.

It would be interesting to compare the difference in colour on RAW and fine jpeg.

Does that mean it's back to a toasted BLT test then ???:D
I'll show your post to the management !!

Don

LOL Don

See if you can get the shot before sinking your teeth into it.

Or you could do a "before and after". :D

Today after a good breakfast
 

The BLT image was taken with Nikon D100 in RAW and JPG to check for any effects on colour. The full image is in my gallery.

The JPG fine image has a file size of 2.91mb.
The RAW converted to JPG has a file size of 6.11mb.

The JPG is slightly lighter than the RAW, but side by side in uncompressed form there does not appear to be any loss of detail.
Attached is a combined image where the only processing has been the raw conversion.

As the combined image had to be compressed I have included a combined image that is a small section of the board and has not had any compression hence the small image. I was quite surprised after my thread starter image how little there is to choose between them.

Not quite up to the standards of DP Review but a good result none the less.

Contributions from anyone with a modern bit of kit would be interesting. After this though, the subject would have to have fine detail that may be lost in highlight areas.

Don

An interesting comparison Don. The toast is a good subject for teasing out the differences in this test. The texture of the bread is distinctly fuzzier in the jpeg, at least it is on my monitor. The wood texture doesn't seem to show as much difference, which is surprising. Overall, there's not much difference but the bread really looks quite different to me.

Duncan.

For in-camera JPEG to be 50% (approx) than that of RAW converted to JPEG suggests the in-camera JPEG must apply more compression than its RAW converted variant. More compression equates to a lower quality image in this case a little softer.

In my test I let NC convert and pass 16-bit TIFF to photoshop then resized whilst just opening in camera JPEG and resizing. The filesize difference was negligible <10%.

I'll do my own colour test using size priority and quality priority JPEG from the D2X tomorrow evening and post the results.

Don, Thanks for posting you and your sandwich, gave me a laugh when stuck in a hotel room yesterday evening :D

Steven, from my earlier look at DPR's review the softness difference between JPG and RAW conversion was put down to the greater processing power of converting RAW in a PC. Given the advances in technology I can understand that. As the JPG file was smaller and the colours lighter I thought that the highlight detail would have suffered. Unfortunately at the time of this shot it was not something I had considered so did not include something suitable in the scene. That is why I did a composite from a section of the board - lightest area in the pic. With both images of the BLT up side by side in uncompressed form, I did not find detail to be missing.

My question to you would be ( as you have the latest kit ) what differences are the between Best Quality JPG and RAW converted straight to JPG. That was the route I took. I did not go RAW through TIFF to JPG.

My interest is that if I know the quality you can get in JPG relative to RAW. Then an upgrade to D200 without an additional PC upgrade may be possible. My current kit could handle the JPG file sizes but not the RAW. I feel from my tests that the differences if there are any, will be in the fine detail in highlight areas.


Don

PS Duncan the JPG is softer its not your screen.

A Quick Colour test with 20D

Colour bars made up in PS and displayed on monitor. Close up of monitor Screen focusing on the green stripe.

Raw file was 9.43Mb
Fine Jpeg was 7.21Mb

Attached is a composite made from 100% crops of jpeg and raw image, no other processing.

Again not a hugh difference, the raw may be slightly better.

Sorry about the patchwork but had problems getting the combined image under 400Kb

Could not sleep for thinking about it then Rob .... Last edited by robski : Today at 02:22. :D

Not up on Canon kit so do not know the release date of your camera, but by comparison with the D100 the file size of your JPEG is BIG. Fine Jpeg was 7.21Mb v D100 JPG fine image has a file size of 2.91mb.

Interesting, thanks for you efforts burning the midnight oil.

Don

Canon 20D was released September 2004.

A D2X in optimum quality JPEG mode does not produce JPEG's as large as 7MB. Filesize is nearer to 3-4MB D2X compressed RAW are 10MB each and uncompressed 20MB!

Edit: The CRT screen image is a challenge to JPEG compression as many edges and rapid spacial changes. I think a reallife JPEG from 20D would produce a filesize much lower.

Yes reallife 20D fine JPEGs are more like 2-4Mb dependent on subject detail.

A little known fact about compression is that with some compression methods it is possible to end up with a bigger file than the original if fed with very rapidly changing information.

You have to remember jpeg in fact uses two forms of compression. A degree of Filtering that throws away image information based on your jpeg encoder setting. After Filtering Huffman compression is then used. Huffman is a lossless compression method.

Another Colour Test
 

I was repairing my big diffuser today and thought a test was in order.

I though a more technical subject was in order this time round. The two composites have only been cropped and compressed to post them here. Softness in the JPG Fine is therefore evident. The larger image had 20% compression and the smaller one 5%.

I have viewed these two side by side at 150 magnification and cannot find any detail that has been lost. Not understanding the technicalities as Rob does I was really pleasantly surprised by the result. So for the D100 there appears to be no percievable quality loss when using JPG.

Don

WOW my back hurts after falling off my chair!

I thought I would repeat my test changing a few factors. Firstly we are looking at 100% crops, on dpreview they use 200% to show difference between JPEG and RAW. I altered in-camera sharpening to 'normal' and did a size priority jpeg and quality priority jpeg. Finally I set RAW compression to OFF. I hoped to give RAW a better comparison.

I looked at the two jpegs and thought, yeah well I will not show the test tonight as its not square, I'll have another go tomorrow. Out of curiosity I converted the RAW and looked at them all 200% then.....BANG - legs in the air and sore lumber region!

I could not believe what I saw, two JPEG's have little in it but look close there is a difference. RAW seems to be picking up the print detail from the label much more than JPEG but when you realise its there, it can be seen. When I looked with my eyes at the tomato's label the printing lines are there, but your own chopped tomatoes from J. Sainsbury's and see.

Shown below is the composite of 100% and the original resized only. To see clearer use photoshop or similar to enlarge the image 200%!

I am considering using uncompressed RAW from now on, and fit thicker carpet and underlay in my office :D

Puzzled ??? Are you saying Nikon are using a lossy compression method with their RAW format ! This does not make sense.

The only reason not to compress when using a lossless method is normally is to save the processing overhead of compressing and decompession.

Very impressive results Stephen.

I noticed the difference on the rim of the can at 1:1 magnification. The label was there at 150%.

This to me clearly shows how digital image processing has improved over the years.

The other thought when seeing this is to compare it with good old film and wonder at the ISO equivalent. I think the thread ' Thoughts on DSLR's future developments ' has some relevance as would you want more resolution than this appears to show. It may be interesting for you to do a similar pic of everyday objects and post it in that thread.

Doing so could of course generate personal expense for me. :D

Don

We shall see...

I should have posted four images taken over two shots, two RAW and two JPEG's.

I intend to do test version three using the same can of tomato's tomorrow evening or over the weekend.

I see it this way -- you can shoot raw and convert them all to tiff or jpeg, but you can't shoot jpegs and convert them to raw, it's too late. I've decided to put up with the extra work involved working with raw files for that very reason. Once you allow your camera to process the raw data instead, you've much less room to maneouvre at the editing stage.

Nikon compressed RAW
 

Rob,

Your question really got me thinking and as a result I have spent a good slice of today trying to find an answer.

The only info on the subject I can get is from Thom Hogan reviews. From these the D100 RAW is not compressed but the D70, and D50 are. As you are the tech. type I will link both of his reviews. The info is half way down the page. In his review of the D2X there is no mention of Compressed RAW but it would seem logical that it is the same as the D50 & D70.

We now need a couple of tests from Stephen when he has some time.

Thom Hogan D50 review http://www.bythom.com/D50REVIEW.htm
Thom Hogan D70 review http://www.bythom.com/D70REVIEW.HTM

Don

I'll do a couple of tests but I have a busy weekend ahead so might not be for a few days.

Here is what Nikon have to say in the tail end of a piece regarding JPEG v's RAW and compressed RAW. Read all they way through as it add more weight and information regarding compressed NEF toward the end.

Compressed NEF
Compressed RAW format available in Nikon cameras (and which can be processed by Nikon Capture) employs a strategy that uses the subjective nature of human perception to reduce the overall data volume while maintaining its quality.

Compressed RAW format algorithm claims to be visually lossless because it treats data differently according to the areas there effects of reduction are masked. It therefore applies compression only to areas which will not be critically visible in the final RGB result. In this respect it resembles part of the compression concept of the widely accepted MPEG layer 3 (MP3). Using Compressed NEF, the data to which the process is applied has reduced significance to the naked eye – Whereas JPEG compression, on the otherhand reduces data based on the orientation and degree of detail, and therefore produces effects that are immediately clear.

The NEF compression strategy is based on the fact that the human visual system has differing sensitivity to tonal variations depending on whether they are highlight, mid-tone or shadow.

Compression is applied selectively so that the effect is minimised and in most cases undetectable to the human eye at normal viewing distances.

In extreme cases the effect of compression may become visible, particularly in areas of an image where there is a combination of high levels of detail and brightness, or extreme saturation.

As the compression process is a mathematical operation carried out on subjectively accessed data, it is impossible to predict with accuracy the few types of image that might produce unfavourable results.

Nevertheless the compressed NEF format will offer the benefit of reduce data size with minimal sacrifice of quality compared with JPEG, and allow maximum flexibility to be maintained.

As a useful tip, it may be helpful to note that, as it is only areas of high saturation or brightness that may be affected, shooting one stop under will reduce the amount of data in the sensitive areas.

The decision on when and where to use the compressed NEF format is therefore dependent upon the experience of the user.

So far, Nikon’s tests* have failed to reveal any clear trend in subject type that maybe affected by the NEF compression. In fact it has been impossible in practical use to identify any artefacts when using the compressed format.

The main identifiable disadvantage is in the longer processing time required to create and decode a compressed NEF image, compared to an uncompressed NEF. However in cases where storage space or transmission bandwidth is at a premium compressed Raw remains a valuable option in the armory of today’s versatile photographer.

Reference
'NEF: No loss of data', Nikon Pro, April 2005, Cedar Communications.

It seem uncompressed is only to be used when blown highlights are to be expected in Y or any R, G and B channels.

That was a very thorough explanation Stephen. Thank you.

A bit more detailed than in the two reviews.

From my looking around today it does suggest that uncompressed is used to wring every last ounce out of the sensor but with obvious consequences on file size. Not something that the average user is likley to require.

In film terms I can see it a bit like using Technical Pan at 12 ISO with a more complex development process, when a standard 25/50 ISO would surfice.

Don

Are you sure its not more to do with the Abbots ale! :D

LOL, I was waiting for that....:o

....and a sore head in the morning! :D

I would love to have a try with RAW but need a simple prog for conversion.I tried the Rawshooter essential from Pixmantec,but was told to paste a long website into a browser bar,so fell at the first hurdle,so to speak.Is there an easy download prog I could try?.