OpenRAW

Here! Here!
Standards within any industry is always good for the consumer as well as the players within in that particular industry. It is the next logical step in the digital revolution. Just think where we can go. Manufactuers would sell more cameras, software developers would sell more software, processors would develop more fine quality prints for less money. Photographers could exhale; knowing the images they capture today will be around for our grandchildren.
If this is to be, It will be up to both you and me to see this thru to fruition.

Show me a camera manufacturer who will support DNG as a standard and my children will support them a lifetime of business
Respectfully
Jim Mulholland
Milwaukee, WI

The problem with DNG is that while it is licensed by a publicly listed company, even if todays management have the best intentions to keep it open, there is nothing to stop an intellectual property "land grab" in the future by unscrupulous management. Well documented precedents exist for this in the Linux space, where even the most carefully worded agreements are reinterpreted.

The only protection we have as consumers are standards that are open, patent-free, and in the public domain. Adobe should foster the standard, and then hand it on. The continuing lock-down of our appliances and data by companies who want a business "model" that locks the consumer into a chain of purchases, is one of the big threats of the digital age.

If we give up these standards to private companies or let governments support cartels of companies in the name of "intellectual property protection", it is the equivalent of allowing car companies to build cars that only run with one gas station's fuel, and only take a particular brand of tyre. And having the engine locked away from the owner of the car.

I wholeheartedly agree. This is the 21st century equivilant of the jigsaw problem. Years ago all jigsaws were designed to accept the same blades. Users went along happily using the brand of jigsaw that they liked best and using 3rd party blades that they preferred. Kind of like when photographers used whatever brand of camera they wanted and then used their preference for film and everything was good. Then one of the jigsaw companies that made a really good saw realized that if they changed the saw so that it only accepted their blades, they could sell blades too. Consumers didn't like that because although they liked the saw a lot, they really preferred someone elses blades.

The whole RAW format issue is really the same thing. IMO the camera companies are just trying to preserve more sales of cameras accessories, and software downstream. If I had my druthers I would like to see all the camera companies adopt a universal format and then use their R&D $ to get the best results out of that format. Leave the processing to whomever the shooter wants. Ultimately the camera that makes it the most convenient for the photographer and gives good results will win, but in the current situation it could take many years for one to establish itself as the defacto standard. Let's see the camera companies step up to the plate and agree upon something for once.

I could be mistaken but I always thought that a single standard was better than no standard at all, which seems to be what your proposing.
I'm not really worried about camera manufacturers keeping their RAW formats a 'secret' as they have spent huge sums of money developing their products.
My first need is to manipulate RAW data and I can already do that with Photoshop. My second need is for long term storage of the images I create. This is a bit more difficult as there is no guarantee that the camera manufacturer will be around forever; and there is even less guarantee that they won't change their RAW format at some later date.
If you read the documentation from Adobe you'll see that DNG is an attempt to address such issues and provide access to RAW data in the future. With the growing support of camera manufacturers for DNG, it's looking highly unlikely that you're going to see Open RAW this side of the grave. Having spent vast sums developing their products, I think it's a bit naive expecting manufacturers to hand over proprietary information for the end users convenience.

Nicholas, I hadn't really thought of it that way but I guess that is really what I'm saying, at least for the short term. I sincerely believe that the best way to preserve my RAW files and eventually have a single standard RAW format is to convince the camera makers to document their formats. Both Nikon and Canon have made it pretty clear that they are not willing to cooperate, let alone collaborate on DNG. Part of that is their attempt to protect their RAW formats (that's the part we are trying to change.) I think another reason is that it is Adobe that has created DNG who is just as bent on dominating their market as Nikon and Canon are. I have read the DNG spec and the licensing document. DNG is most certainly an attempt to standardize RAW and address the issues you mention. I guees the best way to describe my view is that I see one universal RAW format (DNG or something else) as 'the whole enchilada' and OpenRAW as somewhat less (though still a very big bite.)

Thanks for the comment Nicholas

I like to think of DNG and OpenRAW as Scylla and Charybdis to the camera companies.

Oooh. Don't want to document.
Oooh. Don't want to cooperate.
Oooh. What to do ? What to do ?
Must do something. Natives are restless.

-- stan

I agree paramountly on the point of Adobe owning the license to DNG. If a lasting general format is to be established, it must be independent.

If a truce is to be made amongst the warring companies, a neutral standard must be made independent and open to all requirements and beyond them, to future potential for if it is to last.

Hello!
I enjoy this site and their intention very much. I am shooting with a Nikon D70 and I am feel concerned by Nikon WB-politics, even when my camera is not affected ba thiy issue up til now. But Nikon announced an firmware update in May. Who can be sure, that they won't update the WB encryption in my camera by this occasion?
Nikon did not answer this question, when I put that on their support.

I am thankfull to Nikon in the point, that it sharpens the awareness of us photographers in the world about the problem of proprietary RAW-data.

Concerning the discussion about DNG and company-independent standardization formats, I do state the following:
Standardization processes are very long term things. Too long for the fast developing technic. We in Germany are specialists to set up rules for nearly every fart and so we are blocking our developement, technical as well as social and political.
Even if it is critical, that big companies will set big standards, I think, that only big companies do have the knowhow and the power to set up indstrial standards. So I am very PRO for DNG-format.

But ist is of maximum importance, that these company-standards are open in any detail to the public! And they should be license free!
It is unethical behaviour of companies, to create proprietary standards in order to get rid of their customers and make money.
As well I am not generally against software patents, as far as they do not concern standardisation. All standards have to be open and free for every one! If a software routine is basically in the means, that they allow basic funtionality, it should be open for all.
This openess is ethical correct.

In this sense, I wish you good success with your OPEN RAW project.

Clemens M. Hürten - Germany
www.ideecreativ.de

I'm in favour of DNG standard and commend Adobe for trying to bring some sanity to the proliferation of RAW formats.

Some respondents seem concerned that DNG will be owned by Adobe and this somehow make it another closed standard but I would point out that Adobe has a very good track record for open standards.
Arguably the three most common file standards (Postscript, PDF and TIFF) are all owned by Adobe and yet have become industry standards.

Perhaps the camera companies would be willing to cooperate with Adobe on DNG if Adobe was willing to share some percentage of revenue from DNG.
It is not necessarily in the camera companies' intrest to adopt a universal, documented format; it makes the camera more of a commodity - takes away something that might be a competitive difference. Of course, on the other hand, encrypted formats annoy customers. I think if Adobe could encourage their participation monetarily it might just tip the balance, they might decide it was more worthwhile to cooperate.
Sounds ridiculous? Why should Adobe do that? I know, but don't you think Nikon and Canon feel they would lose something by cooperating? Why would they do that, for Adobe's gain?
-- Joe S.

Joseph, could you identify this "revenue from DNG" that you want Adobe to share?

At the moment, DNG is a pure give-away. The specification and Converter are downloaded free, without registration. The global licence grants free use, again without registration.

If DNG makes Raw much more commonly used by photographers, then all sorts of companies will make extra revenue. Anyone selling Raw processors and photo-editors, for example. But also, camera manufacturers whose top-end cameras will have greater peceived value, because RAW can bring out the best from cameras.

I would be interested to learn what you think the camera manufacturers would lose for Adobe's gain? What money are you talking about?

Do you believe that, in fact, all camera manufacturers are going to sell, rather than give-away, their Raw processing software? Perhaps Nikon is just the first. But how much money can they get from this? Do they expect photographers to stop using Photoshop, so that they can spend that money with the camera manufacturers instead? Or will the photographers divert some of their camera budget into camera manufacturers' software?

If camera manufacturers intend to get into the software business, they may have to look outside their own customer base. And the easiest way to do that might be to accept DNG!

It is fascinating to look at the two issues: OpenRaw and DNG. I believe that DNG is a pure commercial play by Adobe to strengthen their position in the digital photography chain. A question: How many people buy PDF creation software from anyone other than Adobe?

I believe that DNG would commoditise all software before the photo editor. Adobe is interested in maintaining and growing its domination of that lucrative market. The camera companies can never hope to move into that market if DNG is used in-camera.

I expect that we have not yet gone 10% down the digital photography development path and I am sure that the camera companies want to keep control of as much of the technology as they can. A totally new imaging technology will probably need a new and different raw format. It is in our interests that the camera companies can develop this as they wish. Let's hope the next "foveon" will be a major success with Canon and/or Nikon.

I believe that to make OpenRaw a success, we need to focus solely on getting camera companies to publish their raw formats and not cloud the issue with DNG, something that I am sure they see as not in their commercial interest. Even publishing the raw formats will give away some trade secrets but no doubt the really good ones will be patented and hence no longer secrets :-)

Andrew

Andrew, I think that misrepresents the nature of DNG.

Surely it is TIFF, not DNG, that tends to commoditise what comes before the photo-editor? Editing photographs is typically done on 3-colour (or perhaps 4-colour) arrays of pixels, such as TIFF, not on Raw data. DNG will tend to (further) commoditise what DELIVERS that Raw data, not what PROCESSES it. This is the dilemma that the camera manufacturers face. Even if they do try to sell Raw processing software, their "camera + Raw processor package" will tend to be commoditised by photo-editors that accept TIFF (or JPEG). A Nikon camera + their Raw processor will be competing with a Canon camera + their Raw processor. But also with any future cameras that don't even bother to provide a Raw processor but simply ensure that suitable software is available.

Raw processors & photo-editors will tend to commoditise cameras delivering Raw, whether those cameras deliver DNG or some other Raw format. Adobe will have no major problem building in support of all those Raw formats, even if their formats have to be reverse-engineered rather than being published. It has demonstrated this over the last couple of years. It is smaller companies for which this is a relatively higher cost. For example, PSP (Paint Shop Pro) has much more limited Raw support. If PSP accepted DNG, it would immediately increase the number of cameras it could support with Raw. So use of a publicised format will make it a bit easier for smaller companies to compete with Adobe, and use of DNG will make it easier still for smaller companies to compete with Adobe.

The exception to this arises if the camera manufacturers encrypt their Raw formats. That appears to be the only way they can stop Adobe, and other scrupulous companies, doing all the Raw processing. Obviously, OpenRAW won't condone use of encryption in order to prevent Adobe dominating the value chain! So perhaps it is simply better to accept that Adobe will do so for the next few years at least, with or without DNG. Publicising formats will make things easier for Adobe, and relatively a lot easier for other companies.

I believe that the only camera manufacturers who have anything to fear from using DNG are those who intend to SELL their own Raw processing software, with a lock-in such as encryption. All the others are no worse off, and will sometimes be better off. (And, of course, no one can force them to use DNG. Which is a pity, because lots of photographers, including myself, would be much better off if they provided it as an option!)

That "totally new imaging technology" may only need relatively minor changes to DNG. And those changes would certainly be incorporated into a new version of the specification pretty fast. DNG is already able to support Bayer format, Fujifilm SR sensors, Sony 4-colour sensors, Foveon sensors, etc. But such changes are actually relatively rare. The last one was a year and a half ago.

There is a limit to the amount of change that occur before the Raw processor is unable to turn the results into TIFF or JPEG. For example, perhaps a holographic sensor couldn't. But such a sensor would need a completely different processing enviroment anyway, and lots of things would have to be developed to make that possible. Until then, I doubt if DNG will inhibit anyone.

The goals of OPenRAW are not satisfied merely by specifications. Photographers need software, not just specifications, and preferably an ever-growing choice of such software. DNG is self-contained - software does not need special knowledge of the camera model in order to access the image. There is a fear that camera manufacturers could put such things as look-ups in their Raw formats that access tables in their Raw processors, so that even with a knowledge of the format, without the look-up tables this wouldn't help much. DNG is designed to avoid that lock-in.

As long as Adobe is in charge of DNG it is not OPEN. The market pressure should be applied to Adobe to come up with a plan that is compatible with the other stakeholders: camera makers, software vendors (including competitors), and end users. Adobe has risen to a position of responsibility through their market position to serve the industry unselfishly. It will be a long struggle!

Ken: why does it matter whether DNG conforms to any particular definition of "open"?

What matters is whether it can be used successfully. DNG already is "compatible with the other stakeholders: camera makers, software vendors (including competitors), and end users".

Here are some useful references:

"Open Standards - Principles and Practice" (Bruce Perens)
http://perens.com/OpenStandards/Definition.html

This identifies 6 principles:
1. Availability
2. Maximize End-User Choice
3. No Royalty
4. No Discrimination
5. Extension or Subset
6. Predatory Practices

According to these, DNG is open. There are other definitions that say it isn't.

"Why use open formats?"
http://www.openformats.org/enShowAll

It asks: "Which open formats can I use"?

.... "If you need to present or print high quality documents, PostScript (PS) and Portable Document Format (PDF), are solutions of choice".

But, of course, both of those belong to Adobe.

"When standards don't apply"
http://techrepublic.com.com/5100-22_11-5251441.html

"On the other end of the "de facto" spectrum is Adobe's PDF (portable document format), widely used for electronic distribution of documents. Adobe still owns the specification, which originated in the early 1990s and was considered for submission to a standards body. But it's so freely published that hundreds of non-Adobe tools for generating PDF documents are on the market, including components built into Apple Computer's Macintosh operating system and the Openoffice.org productivity package. Even standards purists such as Perens and Bray say there'd be little to gain from turning PDF over to a standards body at this point. "PDF is an example of a proprietary standard that has achieved so much inertia in the marketplace that it's hard to see where standardizing it would benefit anyone much," Bray said.

Melonie Warfel, director of worldwide standards for Adobe, said Adobe participates in numerous standards bodies, including specialist groups creating open-standard extensions of PDF for archival and advertising uses. But the main PDF specification remains under Adobe's control so it can be quickly adapted to meet new needs, such as the bar-code capability recently added. "If you start developing a standard through a standards body, it takes forever," Warfel said. "My concern is we wouldn't be able to keep up with technological changes. But Adobe achieves most of the goals of open standards by publishing the PDF specification under liberal terms that allow other software makers to use it as they please".

----------

Note the last comment, which applies to DNG too. It is likely that ISO couldn't react fast enough. Perhaps an industry consortium could, but would (say) Nikon be happy to discuss a proposed new sensor technology, that needed a specification change, with an organisation that included Canon, etc? Perhaps the spcification needs to be owned by other than camera manufacturers.

FWIW I've decided to go with DNG.

Here's a little thought experiment: if, in my case, Canon provided the spec for .CR2 for download tomorrow, would that provide a solution to my RAW issue? The short answer is no. There's a number of problems that would still need solving. Example: Given a proliferation of published RAW standards, where does the critical mass required to make any one implementation worthwhile come from? How do I know that the published RAW format used by a camera I buy today will remain worthy of support in software implementations of the future?

I think the nobel efforts being made by openRAW.org would be better placed campaigning for adoption of DNG. I'd rather my camera manufacturer supported DNG than published their own RAW format.

I've posted some more on the reasons why I think that at http://www.aiddy.com/manningtree/2005/06/dng-roars.html. Feel free to expose flaws in my logic --- if I've made the wrong call I'd like to find out sooner than later ;-)

/aiddy

There is no flaw in your logic. I've recorded that URL, and will use it if the need arises.

I'll provide some minor qualifications:

- If - WHEN - we ask camera manufacturers to use DNG in future, we should perhaps ask them not to use private data, but just to use the standard parts.

- It is still worth getting specifications for existing formats, where this would help with better conversions to DNG. Some current conversions leave a bit to be desired, such as the D2X WB.

- It is worth noting that DCRaw, from Dave Coffin, is able to convert DNG files. So even now we are not completely dependent on Adobe to extract images from our DNG files.

Let us suppose that DNG does get approved as a standard. How long will it be before the manufactureres feel limited by the standard and begin to expand to fit their need to create new features. Will the result then be NikonDNG and CanonDNG etc.? The very browser you are viewing this page on is a perfect example of what can happen when a standard get's modified to fit the vendors needs. There are ZERO browsers available that are fully W3C standards complient. They all have chosen to ignore differing parts of the standards to suit thier individual needs. Some go so far as to advertise support, but the exicution does not happen as intended. Then they add their own sets of "features" that work only on THEIR browsers. What you end up with are pages that do not perform properly on all browsers and all platforms. Sometimes a page will even "break" of fail to work at all.

Back to the subject at hand. Once the authors decide to add their own flavor to DNG, we are no longer 100 percent complient. Taken far enough from that complience and the end result is just short of non-complience. Suddenly we nearly back to where we started.

The software authors who write the decoding utilities will once again struggle to keep up with all the differing flavors, though albeit with less hassle due to "open" documentation.

A standard is not always the answer to a need, but it could be a good place to start. I can't help but wonder: How long before the standard is no longer valid?
John

DNG is designed to evolve. It does not constrain innovation. In fact, it is already in its second version, although the upgrade was relatively minor.

It has a version control scheme that allows the specification, the DNG writers, and the DNG readers, each to evolve at their own rate. I know what you mean about web standards, (I have a number of websites), and DNG avoids the complexity that the web is in.

Each DNG file identifies the range of specification versions that it is compatible with. This range is something that DNG writers are aware of. DNG readers know a range of versions that they can handle, and act accordingly. The range handled by DNG readers will normally start at the first version of all.

Camera makers needing extra features will talk to Adobe, and those features will be added to a new version of the specification, which will be published. Makers of DNG readers will typically then evolve their products. A user of a DNG reader will be warned when the version of that reader cannot handle a DNG file, and will then have the option of upgrading, (or switching to a different DNG reader!)

The important point is: DNG versions never become invalid. So old DNG files, such as this year's photographs, or DNG files written by old writers, such as this year's cameras, continue to be valid.

Obviously, if a particular DNG reader ceases to evolve, it will no longer be able to handle cameras that need the latest versions of the specification. But it will still be able to handle all the earlier versions.

It's like the "old days" with film..... What say for instance, that KODAK was the only maker of chemicals to process film or paper and held ownership and patents on particular and unique processes?
Take Kodachrome as an example.... great stuff, but it held it's own processing scheme, not an E-6 process. If RAW files become proprietary to each company's scheme of how they perceive things should be, I see a slow death to ALL these RAW formats .......until there is a universal RAW format and all manufactorer's are "forced" to use this format, all other formats are headed for obselecents.

OpenRAW and DNG could work well together.

The "ideal" position to get to, from where we are now, is:

1. ALL relevant software FULLY supports DNG.

2. ALL raw files can be FULLY converted to DNG.

If OpenRAW could help us get to that state, it would solve the "Raw Problem".

This strategy is different from what OpenRAW has been trying to do, although largely compatible with it. This strategy has the following advantages:

a). It would be far easier for (say) 100 software products to support DNG than for them to continue to support the ongoing proliferation of raw formats, even if they were all documented. There is no evidence that nearly all software products will ever support a proliferation of raw formats of "minority" cameras, however well documented they are. It is far more likely that nearly all software products can be persuaded to support one format, DNG, as a one-off task.

b). Given that we already have 3 cameras and 4 digital backs that output DNG, this strategy is not diversionary. We already want as many software products as possible to support those cameras. Software developers that can't be persuaded to support a single published format that will catch a large number of cameras, existing ones and those to come, are probably hopeless cases as far as OpenRAW is concerned.

c). OpenRAW has been requesting camera manufacturers to publicly document their raw formats. This strategy offers more options to the manufacturers, and is more likely to succeed. A camera manufacturer can do one of: use DNG as its native raw format or as an optional-extra format; provide its own high quality DNG converter for its raw files; collaberate with someone else to produce such a DNG converter; or publish its raw formats, so that someone else can produce a high-quality DNG converter for its raw files. It can do different things for different cameras.

Up to now, OpenRAW's focus has been on persuading camera manufacturers to document their existing formats and to adopt a common raw format. That remains important, but it is not enough. The above strategy requires OpenRAW ALSO to persuade software developers to support this common raw format. The "RAW Solution" needs software developer support as well as camera manufacturer support. Non-compliant software products should be "named and shamed" too!

OpenRAW left open the decision about whether the manufacturers should adopt DNG or some other "universal raw format". That decision can no longer be left open. With some manufacturers already using DNG, it is the ONLY candidate for a "universal raw format". If some manufacturers agreed another format, we would just have two "non-universal raw formats". It would be ridiculous to waste the progress and experience that has been achieved with DNG over the last year. If Adobe's ownership of DNG is a problem, then it should be tackled directly, and not by accepting a diversionary "also-ran" format.

I presently have Photo Album 6 Deluxe Edition, but feel I should have taken Paint Shop Pro X.
But not been familiar with these programs, I really didn't know.
My problem is presently my RawShooter will not open.
So I am thinking of downloading Paint ShopPro X, as presently I have a good organizational program which does most things Photo alblum 6 Deluxe does.
A complementary copy of Paint Shop Pro X would sure be nice.
Thank you Florian Haley
370-8265-E.Southern Ave.
Mesa, AZ. 85209

DNG - Gaining Steam

With both Hasselblad and Leica now directly supporting DNG in their cameras, it really does seem that the format is gaining some traction.

I've started converting my Nikon files, and I have to say, I'm extremely impressed with the amount of thought that Adobe has put into the design of DNG. I'm sold on the DNG format.

First of all, converting a library of .NEF or other proprietary Raw camera files is simple and relatively quick. I was able to convert 12,000 files in a little less than 24 hours, and the free DNG converter created a mirror folder hierarchy for my many folders and subfolders of NEF files. Nice and painless!

Second, the latest DNG converter offers the ability to embed the original NEF inside the file to be extracted at a later date, if progress has been made in intrepreting the manufacturers' proprietary format. This is the ultimate in flexibility! Sure the files are 38% bigger, but I'll trade this for the insurance.

But to me, one of the most important features of DNG is having the metadata embedded in a documented way. Having my Raw files carry metadata in a transparent way is absolutely crucial, and DNG solves this problem very well. Embedding metadata in proprietary formats is sketchy, and many 3rd party tools won't do it at all (like Adobe and Aperture) for fear of corrupting the files.

Incidentally, the new Camera Raw 3.x shows a great deal of thought with regard to workflow and Raw processing. I love the fact that when changes are made in ACR, those changes are saved inside the DNG file, instead of the stupid XMP sidecar file that's useless to other applications. While I might not love ACR's color 100% of the time, they really have nailed the workflow.

Which leads me to my next point: Camera manufacturers should be making cameras, not software!!! I would much prefer Nikon close down its software operation and bundle in Photoshop Elements, and spend the resources on making their sensors better. While Nikon Capture is a fine product, it still lags far behind products made my true software companies, both in performance and interface. I would give it up in a heartbeat if Nikon would support DNG. (Actually, I pretty much have already - I just don't have the time to wait for it to process my files in Capture)

As noted in other posts, Adobe has shown consistent leadership in the development of standards like postscript, PDF and Tiff. DNG is a fine addition to this legacy. Frankly, I care more that it is a good, well thought-out standard, than that it is completely independent. Think about what the world would be like without PDF, if it had been rejected because we were all waiting for something truly "open." DNG is here, it's well-done, let's get the rest of the camera makers to adopt it.