Converting current RAW images to Open RAW

Please pardon my ignorance if this has been covered somewhere and I just didn't notice it.

I was wondering if there's anything in the works as far as software for converting existing RAW images into an open format? An open format right from the camera would obviously be great, but it does nothing to solve the problem of aging RAW files that may lose support in the future.

I don't know what kind of technical problems exist for this kind of software, but it seems to me that it could be made as a batch processing plugin for UFRaw. Add in the ability to do lossless compression and you'd have something that's quite practical for backup purposes.

While I'm dreaming, let's throw in the ability to create a portable RAW file, something that's reduced in detail to get the file down to under 2mb that could be more easily distributed over the web.

DudeMan – Sat, 2007/02/03 – 6:20am

DNG is a lossless format

DNG is a lossless format

DNG files are capable of holding all the raw image data from cameras with CFA-based sensors, including those with offset sensors such as Fujifilm cameras. They are capable of holding all EXIF data, including Makernotes. They are capable of holding lots more, for example TIFF/EP metadata and XMP metadata.

Whether they actually hold all the data depends on how well the software used, (for example DNG converters, of which there are several), copies it across. The claim is that for NEFs, CR2s, PEFs, and probably some others, it is all copied across. Anyone claiming differently should provide suitable evidence, instead of exploiting "fear, uncertainty, doubt".

Two main types of problem with DNG conversion are known:

One problem is that the developer of the DNG converter doesn't know enough about the original raw file format to be able to copy all the data across. Typically, the raw image data is copied across, so high quality conversions can still be performed. ("Digital Image Preservation" is achieved). But some non-essential data may be missed, for example previews, or some information that isn't actually needed by most mainstream raw converters. (A problem with ORFs was identified with version 3.4 of the DNG Converter).

Another problem is that data that IS copied across into the DNG isn't understood, other than by the company that created the original raw file. As above, high quality conversions can still be performed without knowing this information. Examples include such added-value as "picture styles", "curves", perhaps lens-identification information, etc. In this respect, DNGs are obviously no worse than the original raw files, and since the rest of the DNG file is understand, and also the DNG Converter puts EXTRA data into the DNGs, DNGs are a much better archival format than the originals.

The solutions to both of these problems need better information about the original raw file formats. In the first case, that will result in better DNG converters being written. In the second case, simply documenting the Makernote automatically provides documentation for the copy of it in the DNGs. Since Adobe software and some other mainstream raw converters don't use that data anyway, solving these problems would actually make no difference to the resultant image.

DNGs generated from documented original raw files will be a better archival format than the documented original raw files themselves. This is for two main reasons:

First, the DNG files contain extra camera details so that future raw converters don't have to have camera details built in. In (say) 20 years time, while documentation of an original raw file format may enable a raw converter to READ the file, without being able to test a camera of that model it may be hard to determine how to PROCESS that data. Raw files, other than DNGs, typically don't contain all the necessary information for a high quality raw conversion, so merely documenting every detail of the raw file format is insufficient.

Second, DNG largely solves the problem of the proliferation of raw file formats and variations of these. Therefore, it is easier to achieve critical mass. One good implementation will support DNGs for images captured by many camera models, making it far more likely that a company will bother to implement it. Software companies support cameras according to their business case, not just because their raw file formats are documented.

When OpenRAW was launched, there were about 25 non-Adobe products that supported DNG in some way. Today there are more than 150, including several cameras and digital backs, plus some software products from some camera manufacturers. There is also more known about the contents of Makernotes, hence of copies of those in DNG files.

Barry Pearson – Mon, 2007/02/05 – 11:53pm