FAQ FAQ  Forum Search   Events   Register Register  Login Login

a700 ARW - Findings so far...

Page  123>
Author
my-spot View Drop Down
Senior Member
Senior Member

Joined: 24 January 2006
Country: United States
Location: United States
Status: Offline
Posts: 504
Post Options Post Options   Quote my-spot Quote  Post ReplyReply Direct Link To This Post Topic: a700 ARW - Findings so far...
    Posted: 03 October 2007 at 16:31
I have spent the last couple of days looking into the a700 raw files I have and thought I would share my meager results so far...
I do not have any good cRaw ARW files, so I have not bee able to decode the compressed Bayer pattern data at this time but, I DO have plenty of uncompressed a700 ARW files and have managed to read the Bayer pattern data...
- The data is 12 bit. Period! (not 14 bit as some had hoped).
- The data is arranged in a weird "nibble" (half byte) order and needs to be "decoded" two pixels (3 bytes) at a time.
- The old "MLT0" Minolta makernote seems to finally be gone.
- The old "MRI" block lives on however.
- White balance data seems to be recorded in more then one place.
- These new ARW 2.0 files follow the Tiff/EXIF standard a bit more closely and can be previewed by most software that can read a tiff file (change the ARW to TIF). This is NOT viewing the raw data but showing an embedded jpeg.
- The Sony maker note has been expanded and has many new tags. (i'm still working my way through them).
- Camera rotation seems to (finally) be recorded out in a TIFF tag where it belongs.
- The LensID is still there. Interestingly, Among the LensIDs in the new ARW files I had were 51 & 255. I cannot find reference to these IDs and so I can't include a name in the database.

I did most of this to get my RawHide converter to read the a700's files. I now have it "working" but need to fix the white balance and the color matrices. I hope to publish it as soon as I work around some nasty bugs in Microsoft's .Net environment.
 



Back to Top
kiklop View Drop Down
Admin Group
Admin Group
Dyxum owner

Joined: 14 July 2005
Country: Croatia
Location: Rovinj
Status: Offline
Posts: 10563
Post Options Post Options   Quote kiklop Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 19:21
Thanks for sharing, very interesting.

btw, do you have a list of lens ID's that is different from the one on dyxum?
http://www.dyxum.com/dAdmin/lenses/MakerNoteList_Public.asp?stro=makr

I would like to update this list
Back to Top
David_S View Drop Down
Senior Member
Senior Member

Joined: 20 May 2006
Location: United States
Status: Offline
Posts: 2594
Post Options Post Options   Quote David_S Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 19:24
Frank what type of subject matter is good for images to be decoded? I'm going to shoot some RAW+jpeg & cRAW+jpeg to send someone who is then forwarding them to a software company and I am curious if there is a type of image that is preferred for this type of work.

Thanks,

DS
Back to Top
my-spot View Drop Down
Senior Member
Senior Member

Joined: 24 January 2006
Country: United States
Location: United States
Status: Offline
Posts: 504
Post Options Post Options   Quote my-spot Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 22:01
Originally posted by David_S David_S wrote:

Frank what type of subject matter is good for images to be decoded?


What I really could use is two nearly identical images where the only difference is one is RAW and cRAW. I would suggest tripod mounted to minimize the differences. Also some detail in the extreme upper left had corner (first few pixels) would help. So far, with the one cRAW image I have been able to get, I have found there is at least some funky encoding going on.
Back to Top
vnatchu View Drop Down
Newbie
Newbie

Joined: 02 March 2007
Location: United States
Status: Offline
Posts: 13
Post Options Post Options   Quote vnatchu Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 22:01
cRAW unfortunatly is not lossless.

The tiff file stores 8 bits instead of 12 (hence a 1/3 drop in file size compared to full RAW)

The 8 bits are then mapped to a full 12 bits using a scheme that i'm still trying to understand. of course there is more than 8 bits of information since the mapping will benifit from some sparce spaces.

I have the full code for decoding cRAW files provided by dave coffin (dcraw), its still in beta but it works.

I'm a little disappointed, i'm sure a lossless compression for this amount of file size reduction could have been accomplished.
Back to Top
Shaocaholica View Drop Down
Senior Member
Senior Member

Joined: 14 July 2006
Location: United States
Status: Offline
Posts: 1881
Post Options Post Options   Quote Shaocaholica Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 22:15
Originally posted by vnatchu vnatchu wrote:

I'm a little disappointed, i'm sure a lossless compression for this amount of file size reduction could have been accomplished.

Well converting a full RAW file into Adobe lossless DNG yields a 30% smaller file with all the same data.

Also, is there any official documentation/source code for decoding these files or does Sony only supply that to the big RAW converter developers? Or is it something that can be reverse engineered pretty quickly?

Anyway, I will supply some test RAW and cRAW files from a tripod next week if someone doesn't beat me to it.
 



Back to Top
Martijn85 View Drop Down
Groupie
Groupie

Joined: 29 March 2007
Location: Netherlands
Status: Offline
Posts: 143
Post Options Post Options   Quote Martijn85 Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 22:15
A raw file stores 12 bits info in a 16 bits space. So ther is room for at least 25% lossless compression. (or am i thinking in shortcuts?)
Minolta 5D + 18-70 f/3.5-5.6 + 50 f/1.7 + 70-210 f/3.5-4.5 + Sony HVL-F35AM
Back to Top
mikef View Drop Down
Groupie
Groupie

Joined: 25 August 2007
Location: United Kingdom
Status: Offline
Posts: 149
Post Options Post Options   Quote mikef Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 22:43
No, the Raw file stores 12 bits in 12 bits, not 16. See the first post.

Nevertheless, it should be feasible to write a compression algorithm to get an average 30% reduction in size and is lossless. It might just be too slow with the in-camera processor.
Back to Top
my-spot View Drop Down
Senior Member
Senior Member

Joined: 24 January 2006
Country: United States
Location: United States
Status: Offline
Posts: 504
Post Options Post Options   Quote my-spot Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 23:03
Originally posted by vnatchu vnatchu wrote:

cRAW unfortunately is not lossless.

The tiff file stores 8 bits instead of 12 (hence a 1/3 drop in file size compared to full RAW)

The 8 bits are then mapped to a full 12 bits using a scheme that i'm still trying to understand.


This is also what I found, The scheme is fixed length and needlessly complex IMHO. It almost has to be lossey, but I'm not a compression expert. The a100 used a compression scheme that worked fairly well, was lossless, but was susceptible to damage (flip one bit and you would screw up the whole image). I haven't asked David Coffin for his new code yet - maybe I should...
Back to Top
Entropy View Drop Down
Senior Member
Senior Member

Joined: 02 January 2007
Location: United States
Status: Offline
Posts: 979
Post Options Post Options   Quote Entropy Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 23:18
If cRAW is really lossy this is a big disappointment for me. I was under the impression that Sony claimed that it was lossless - 18MB vs 12MB for a file makes quite a big difference, especially when JPG quality is so poor. I'm spoiled by nice 3MB sharp JPGs out of the a100 so I really don't want to go to 18MB files for similar sharpness out of the a700.
Yuck, and I had just accepted the annoyance of going to a cRAW workflow.
Back to Top
my-spot View Drop Down
Senior Member
Senior Member

Joined: 24 January 2006
Country: United States
Location: United States
Status: Offline
Posts: 504
Post Options Post Options   Quote my-spot Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 23:45
Originally posted by Entropy Entropy wrote:

If cRAW is really lossy this is a big disappointment for me. I was under the impression that Sony claimed that it was lossless...

I haven't figured out the scheme yet, but I know the DNG thingy converts "cRAW" into 12 bit data (stored in a 16 bit space). It might be lossless or at least mostly so, as soon as I know for sure I will let everyone know.

Edited by my-spot - 03 October 2007 at 23:46
Back to Top
vnatchu View Drop Down
Newbie
Newbie

Joined: 02 March 2007
Location: United States
Status: Offline
Posts: 13
Post Options Post Options   Quote vnatchu Quote  Post ReplyReply Direct Link To This Post Posted: 03 October 2007 at 23:52
I expect it to be mostly lossless.
my-spot, you should ask dave for the code.
I have his beta version, which he most likely will merge with his code soon anyway.
if he doesnt mind i'll send you a copy.
Back to Top
my-spot View Drop Down
Senior Member
Senior Member

Joined: 24 January 2006
Country: United States
Location: United States
Status: Offline
Posts: 504
Post Options Post Options   Quote my-spot Quote  Post ReplyReply Direct Link To This Post Posted: 04 October 2007 at 00:02
Originally posted by vnatchu vnatchu wrote:

I expect it to be mostly lossless.
my-spot, you should ask dave for the code.
I have his beta version, which he most likely will merge with his code soon anyway.
if he doesnt mind i'll send you a copy.


I have an email into him and a PM to you... I'm also a proud $upporter of dcraw. If I ever get RawHide off the ground I hope to send him some more $$$. In the mean time, I actually enjoy trying to figure this stuff out - to a point. A year ago June, I worked out the a100's compression scheme and passed my findings on to Dave but he beat me to actual code by about 3 hours. Dave Coffin is truely amazing!
Back to Top
kiklop View Drop Down
Admin Group
Admin Group
Dyxum owner

Joined: 14 July 2005
Country: Croatia
Location: Rovinj
Status: Offline
Posts: 10563
Post Options Post Options   Quote kiklop Quote  Post ReplyReply Direct Link To This Post Posted: 04 October 2007 at 00:32
At the press event i asked about cRaw and i'm told that is lossless (the rep proudly stated that Sony has a lot of knowledge about lossless algorithms from other products). But .. don't take that as certain .. i got other answers that have proven not to be fully accurate.
Back to Top
Dyxum main page >  Forum Home > Equipment forums > Camera Talk > A-mount APS-C Page  123>

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.078 seconds.

Monitor calibration strip

Dyxum.com - Home of the alpha system photographer

In memory of Cameron Hill - brettania

Feel free to contact us if needed.