Open RAW, just my two cents. First, let...
Open RAW, just my two cents.
First, let me remind you that almost all the raw file formats follow basic TIF formatting. That is the primary reason that some folks have been able to “hack” the contents. They do have different file extensions and there is some manufacturer unique data included. And, oh yes, I the photographer own the image no matter what format you choose to save it in.
Most raw formats also conform to IPTC and EXIF standards as extensions to the TIF standard. The controversy here is rooted in a subset of the EXIF data called Makernotes. This area is left to each manufacturer to uniquely define and implement.
But actually, the problem is deeper than just the Makernotes. The individual sensor values have no real meaning unless you understand the spectral response of each channel and the layout of the sensors. Much of this is buried in mathematical algorithms in the editor that are not even in the raw file. All sensors are not created equal. Without these algorithms, you compromise color fidelity. And, I for one do not want to see someone dictate a single format, let alone a single response curve. That would essentially result in a standard TIF or JPG image that I already have through the support of most digital cameras.
Most camera manufacturers support some kind of extended dynamic range. Kodak calls theirs ERI. Fuji uses some additional unfiltered luminosity sensors and calls it CCD SR. Nikon and Canon don’t have fancy names but they are participants. Adobe software has it’s own independent implementation called HDR and unique algorithms in their Camera Raw support. .
Most manufacturers provide a raw file editor for their specific image data. Some folks seem perplexed that these images just seem to look better than in the generic editors. It should be obvious to the casual observer that someone is just guessing about the real image specifications.
Even if most manufacturers were to publish their specific implementation of the Makernotes, it would not necessarily mean that the data would be useful for all generic raw editors.
A common raw format does make sense though. A neutral standards body should define it. It could easily be a legitimate extension of TIF. It should never be owned by only a single software vendor. It needs participation from manufacturers, software vendors, and users alike.
This new TIF extension, call it “Rawnotes” for lack of a better word, should include the minimum information needed by generic raw editors to process the raw image data. That is what has to be worked out between the manufacturers, vendors, and users. The current battle is over white balance. Should that be two, three, or four numbers and how are they scaled? Don’t we also need the spectral information for each color filter? Should that be in one nanometer, five nanometer, or ten nanometer increments and how should it be scaled? Do we need better specifics about how the color filter arrays are laid out? I’ve seen evidence that the Fuji layout is not correctly mapped in some generic raw converters. Are there other shooting metrics such as ISO and shutter speed that might dictate special algorithms? I think that is the case with some Kodak cameras. This is just scratching the surface.
If future needs suggest a change to these “Rawnotes” there could be a new revision level of the standard. Again when and how would be a community decision. Just as we have done with JPG, EXIF, IPTC, HTML, XMP, and other real standards.
Aldus and Microsoft developed the TIFF format in 1986. The specification was owned by Aldus, which in time merged with Adobe. Consequently, Adobe Systems now holds the Copyright for the TIFF specification. The file extension has been hijacked at various times by Kodak, Canon and others. Since it is not owned by a formal standards body it is by default a de facto standard only. The Adobe DNG format has not even approached a de facto stature. Ownership is really the first problem that has to be addressed.
The standard needs to address the minimum data and formats needed for generic image processors to extract a reasonable RGB representation. It does not need to accommodate every image feature that a manufacturer would like to employ. Compliance would be completely optional, but necessary for a seal of certification. Manufacturers should be free to deploy their own proprietary features within the same file without fear of piracy.
Just my two cents.
--
http://www.rags-int-inc.com
Barry, The reason I don’t embrace DNG at this time is...
Barry,
The reason I don’t embrace DNG at this time is not technical. It is simply and only because it is owned exclusively by Adobe. And yes, Adobe also owns TIF.
Some pro cameras allow you to shoot both raw and JPG at the same time. Leica and Hasselblad have announced intent to support the Adobe DNG format directly. I was unable to verify any other cameras though they probably exist. Choices.
Every new industry standard I have ever seen was based to a large extent on previous work. I wouldn’t care if the new standard was actually DNG as long as an independent body owned it.
IMHO, ACR doesn’t always do justice to some camera models already. DNG is based on the metrics that ACR wants. Neither seems to address specific spectral response curves. These are basically addressed with user calibration sliders. There are many camera features that are also not addressed. Nikon curves is one. Kodak has a long exposure shooting mode that Kodak software seems to handle much better than ACR. The list goes on. This is why I suggest that the standard needs to address the minimum camera metrics that would be needed to extract a reasonable image from the raw data. It is apparent that white balance is one of these. I would rather see an NEF format that Adobe can access than a choice that says I have to shoot in NEF or DNG only.
That is why I would rather see civil negotiations than public fist shaking. Fist shaking is an attempt to coerce agreement. And, the Japanese culture does not respond to fist shaking. This is a classic example of why the standard belongs with an independent body.
Cheers, Rags :-)
Rags, thanks for that response. There are still a few...
Rags, thanks for that response.
There are still a few questions - here a 3:
1: Are you saying that you don't embrace DNG as a component within digital photography? Or just as a Raw format to be output by cameras?
DNG can have a good future even if no camera manufacturer uses it for their Raw format. Photographers can make good use of it, by using other ways of turning Raw files into DNG. (As I do). Users of photographs will embrace it for obvious reasons. (I expect to see users of high-quality photographs, such as National Geographic, accepting, and perhaps even asking for, DNG).
I suspect that in a few years, cameras will be the only things NOT using DNG! Except that a few of them MAY be using DNG. I hope LOTS of them will be using DNG.
2. Why does it matter who owns the specification? What PRACTICAL difference does it make?
Every argument about this that I've seen has been a "hand waving" argument. I haven't seen a single reason why it actually matters to a camera manufacturer whether Adobe or ISO own the Raw specification. It is JUST a specification! No big deal.
Suppose that you were a Nikon camera product manager, developing the D3X. Why would it matter to you whether you picked up a document called "DNG version 1.1.0.0", or an internal document called "proposed D3X variant of NEF version D2X", or an ISO standard called "TIFF/EP version 2005"? They are just words, and all would serve your purpose.
(DNG version 1.1.0.0 is freely available. The internal specification may be bound up with internal politics. The ISO standard would cost you 150 euros).
I keep seeing words such as "it mustn't be owned by a company", but no one says why not! (Your website suggests that you and I may share similar engineering skills and experiences. I'm sure we both subject such statements to scrutiny).
3. Who is shaking their fists?
Adobe isn't. They have published a well-engineered specification at their own expense, published a global licence for anyone to use it free, and then worked round every obstacle that the camera manufacturers have thrown at them, within legal boundaries.
Obviously Nikon have been shaking their fists! We all know that. The only reason to encrypt anything is to force their customers to buy their software - what else? And, to "justify" this, they published their derisory statement that photographers don't really need Photoshop!
http://avondale.typepad.com/rawformat/2005/04/nikon_says_phot.html
I suspect that Nikon's problem isn't that they are being forced into anything by another company. It is that they have screwed up, and they know they have screwed up, and they haven't worked out where to go next. (It is now some time since Adobe said anything on this topic. I suspect they are giving Nikon the chance to find a way out of their dilemma).
Barry, 1) Neither. All I said is that DNG is not really...
Barry,
1) Neither. All I said is that DNG is not really an industry standard. Time will tell if it becomes a de facto standard.
2) This should need no further comment.
3) Adobe fired the opening salvo. Nikon’s response has been described as timid by most folks. You could make a “passive aggressive” argument. Hence I noted the cultural differences.
I just read the following excerpt about the Nikon D2Hs white balance. It is similar to the D2X. [quote on]
Precise, easy-to-use Auto White Balance / Auto Tone Control System: The D2Hs features an intelligent white balance system that accurately measures and calculates color temperature and white balance settings for complex conditions..
It achieves this by employing Triple Sensor metering, and determines values from three different sources: a "third eye" incident meter (Ambience Light Sensor) located on the prism of the camera measures environmental light, Nikon's exclusive 1,005 pixel RGB color meter sensor continuously measures reflected light from the viewed image, and the DX image sensor measures color temperature in real-time during the actual exposure. Advanced algorithms process the perfect white balance setting for any given scene, using analysis of values from each of the three sources.
* Nikon's acclaimed 1,005-pixel RGB Exposure/Color Matrix Metering Sensor performs direct through-the-lens (TTL) metering of the subject.
* The LBCAST imaging sensor calculates the lighting characteristics of the actual image to be shot.
* The new external Ambience Light Sensor meters ambient light without being affected by the color of the subject, and distinguishes artificial light from natural light.
An ideal white balance is even possible in situations where white cannot be detected because, for example, only greens or reds are coming through the lens. In such instances, data regarding the available light is collected from the Ambience Light Sensor. The system also features new easy-to-use preset WB controls.
[quote off]
My comments:
Nikon is providing some interesting and innovative new technology addressing white balance. Unfortunately, they did not also emulate this in a backward compatible Makernotes tag. The Makernotes tags are not public. Unfortunately, there is no TIF tag for white balance (that I am aware of). Unfortunately, this did present a problem for Adobe.
Distressing to me, no one seems to care if this new technology works or has any value. Just the obvious and unfortunate impact on their workflow. IMHO there are more diplomatic ways this might have been addressed. Hence I do agree with the need for an OpenRAW standard. Again, WB is not the only issue.
Cheers, Rags :-)
Rags: a follow-up to your answers: 2. I asked: "Why does...
Rags: a follow-up to your answers:
2. I asked: "Why does it matter who owns the specification? What PRACTICAL difference does it make?"
You responded: "This should need no further comment". Are you saying "it doesn't matter"? Or "the answer is obvious"? Or what?
I believe it doesn't matter. I have yet to see any evidence to the contrary. I think "ownership" is simply an irrelevant diversion. It would be interesting to know what Nikon thinks. (Several companies now support DNG - do they care who owns it?)
3. I asked: "Who is shaking their fists?"
You responded: "Adobe fired the opening salvo".
The opening salvo, of course, was from Nikon, by encrypting some of the Raw data. (And it really was encryption, using, I believe, part of the camera serial number and the image number, which has no plausible other justification). This was an action against their customers, to try to force them to buy Nikon software instead of giving them a free choice. THAT latter is the way to find out "if this new technology works or has any value".
All Adobe, (in fact, Thomas Knoll), did, as far as I know, is give a technical statement on 17th April of what the problem was:
http://www.fredmiranda.com/forum/topic/212425
That wasn't a "salvo". It was a legitimate clarification to some of Adobe's customers, who wanted to buy a D2X and feed the images into Photoshop. There has been a lot of discussion worldwide since them. Adobe don't appear to have been involved in it. Nikon has made some rather silly statements, and lots of photographers have expressed their displeasure at them. This website exists partly to counter such actions from camera manufacturers.
Had the D2X supported DNG, at least as an option if not as the native Raw format, Nikon would have been spared the bad publicity, and their customers would have been delighted. (Nikon probably had sight of the DNG specification, at least at an early stage, nearly a year and a half ago). I hope they, and other camera manufacturers, learn something from this.
This was a dispute between Nikon and its customers, not between Nikon and Adobe.
Barry, Obviously we are not in full agreement on some...
Barry,
Obviously we are not in full agreement on some specifics. I have no problem with that. But I would not like the discussion to turn into a quagmire.
I sincerely hope that I made my comments clear. But I could be suffering from Aphasia.
Cheers, Rags :-)
Rags: Dng was engineered for the purpose you describe. So...
Rags: Dng was engineered for the purpose you describe. So why don't you endorse DNG?
It is clear from your article above, and from your web site, that you know a lot about this topic.
What is your technical evaluation of DNG? Is it technically the right sort of specification? If there are technical flaws with it, are these flaws that can be corrected, or are they fundamental?
If the problem you see with DNG is not technical, what is it? Can it be fixed somehow?
It took a lot of time and effort to develop DNG. According to an interview with Adobe's Kevin Connor at the end of last September: "Adobe has been working on this format for almost a year, based on expertise we have built up over the past two years doing support for raw formats within our applications". And now it is 7 months beyond that.
If a standards body were charged with developing a formal standard for Raw files, they would not start from scratch. No credible standards body would ignore that investment. I would expect them to start with DNG, and do the minimum necessary to it.
Some myths surround DNG, such as: "it would inhibit innovation"; "it ties consumers into Adobe products"; "it would enable Adobe to control the camera manufacturers". It may simply be best to dispell the myths.
I don't know at what point something is judged a de-facto standard. But there are perhaps 30 non-Adobe products that read and/or write DNG, and the number increases month by month. It has been widely endorsed by photographers and users of photographs. The "only" problem is that the camera manufacturers don't appear to be taking it seriously. How can that problem be overcome? (At the end of September, Adobe's Kevin Connor said: "We have been sharing the specification with manufacturers since the beginning of the year and soliciting their input").
http://www.digitalmediadesigner.com/articles/viewarticle.jsp?id=28283