The consolidated CRAW compression thread

Originally posted by pegelli Originally posted by gm4jjj Be careful - bit depth and dynamic range are not the same thing. Exactly !: Dynamic range is the difference between the darkest part that gives the smallest signal and the brightest part before it "blows out". Bit depth is how many levels this range between darkest and lightest is devided into. Also be carefull to say that compression allways throws away data. There are compression algorithms which are scientifically proven to be lossless (eg. LZW compression of Tiff files).

Sorry, yes I can see how this can be. I asm still not any wiser whether the compression is lossless (like ZIP etc) or lossy (like Jpeg).

Had a reply from David to my question...

"DK replies - it is definitely 8-bit. Sony describe it as ‘lossless bit depth compression’. I do not understand that fully, but maybe it works like ATRAC and their audio technologies by using perceptual algorithms - compressing insignificant information only. I can see no difference in practical use, because when opened, the 8-bit compression is restored to 12-bit range."

ATRAC is from what I know is not lossless. Will admit I don't really understand much about compression (a little), but I thought that Jpeg compression was also considered (when used on high quality) to just compress the "insignificant information only".

Think I'll just switch to standard RAW and forget that I asked this question.

I am shocked.

I must admit at the same time, that I have't found any difference while peeping at RAW and cRAW, but I WASN'T LOOKING for this kind of difference either. I was looking at highlight and shadow clipping, noise patterns, the like. No difference there.

But difference in bits/channel IS noticeable, and something I'm pretty annoyed with. Having 8 bits per channel, gives you 256 gradients per channel. In a monochrome graduation, this means 256 different gradients in total.

12bit/channel is 4096, and in larger prints, this is noticeable without any further ado.


But when you post process, the problem with the limited bitresolution becomes even more evident.


Why didn't they just do a normal ZIP-like compression? Why a fix-length 8-bit conversion? YEAH, sure it LOOKS like the 12-bit version - but when manipulating, it doesn't behave that way!

Luckily I've only shot in cRAW the past 6 days. Before that, I used plain RAW. Which I will now do again.

*sigh*

Gabriel, thanks for the explanation of how exactly this compression is achieved, most interesting and at the same time disappointing that Sony took this lossy approach. I'm now undecided whether to continue with cRAW + JPG.

I used to be indecisive, but now I'm not so sure

Don't be that picky... I work as a imaging engineer, and that's a very common way of compressing data. In each line, you save the first value, and then save only the diference to next pixels. Since from one pixel to the next you can't have a whole 12bit jump, you won't suffer at all from this compression scheme. A lens can't resolve one pixel as "4096" value and next as "0", because of the antialias filter, the bayer pattern... In the extrange case when it really is a level jump which can not be represented in 8 bits, you can do two things: put a flag and start the sequence again (put the real value and not the difference and go on...) or saturate the 8 bit difference, because it will probably corrected by the next pixel.

Let's make all this clear with a numeric example:

Imagine we have this sequence coming from sensor:

120 - 250 - 324 - 400

We can represent it as:

(120) 0 - 130 - 74 - 176

With no loss at all.

If we had something like that:

120 - 300 - 800 - 830 - 880

it would be compressed as:

(120) 0 - 180 - 256 - 256 - 68

which will be decompressed as:

120 - 300 - 556 - 812 - 880

Which will be a matemathical loss, but in a very extrange case and with minimal visible loss. Even if we apply special processing to saturated level, we could average saturation, and have this result:

120 - 300 - 749 - 846 - 880

Which will be almost lossless, or if we want really lossless:

(120) 0 - 300 - (800) 0 - 30 - 50

Not all compression are as destructive as JPEG ;)

So, ok cRAW is fine, so why do we have RAW?

"cRAW compressed format uses 8-bit depth????"

I think someone said this already, in more abbreviated format, but there appears to be some inconsistency of use of the term bit depth. Color bit depth as used referring to jpeg or tiff or computer monitors isn't necessarily the same the same as assigned bits used in data storage and compression. The usage i the original article is not clear to me.     

Talking about color bit depth of a raw file makes no sense. Raw files have not been demosaiced, so are inherently monochrome. Each sensor pixel records only a brightness level. Yes, sensor pixels have different color filters in a Bayer array, but each pixel records only the brightness through its individual filter.    

Color for each pixel location is calculated in the demosaic process using the intensity recorded for that pixel plus intensity from surrounding pixels combined with the known location and color filter of each pixel. For raw files that happens in the computer, not the camera.

I have posted this before but here is a good summary from Adobe.
http://www.adobe.com/products/photoshop/pdfs/understanding_digitalrawcapture.pdf


          

EgonVLC - many thanks for your excellent explanation with example, all makes much more sense to me now.