22.LDNM - Jul-Dic 2006 Conocimiento libre

RAW format, the captive photo

Javier García Diz

The challenges of digital revolution in photography are driving camera and software manufacturers to make decisions that don't always agree with users interests. Maybe the most critical case is the one around the problems with the RAW image file format, whose lack of standarization could make impossible in time the use of millions of digital pictures.

Ladinamo has talked with two of the people who have more to say about all this, Dave Coffin, software engineer from the States, creator of the ultra widely used dcraw program, and Juergen Specht, German photographer living in Tokyo and leader of the OpenRAW movement.

Talking with Dave Coffin

How did the creation of dcraw come about?
In February 1997, I bought my first digital camera, a 0.5-megapixel Canon PowerShot 600. It could shoot either JPEG or CRW (raw) photos, but the software Canon provided to decode CRW was Windows-only. So I wrote a crude program -crw.c-, which eventually became dcraw.c.

Who are the target people for your software?
(1) Photographers. Dcraw allows them to decode raw photos on any computer running any operating system.
(2) Programmers. Dcraw handles _all_ raw photos in one compact codebase (< 7000 lines). This provides a clean, simple interface compared to linking together a dozen or so buggy, poorly documented, binary-only SDKs.
(3) Historians. Many photos of important current events get archived in various raw formats. If the archive media survive the ravages of time, dcraw.c will be the Rosetta Stone that allows future generations to decipher them.

Some could say: hey, why to present this software when you could become rich with it?
Getting rich off a piece of software means keeping the source code to yourself, away from potential competitors. Yet the source code is what makes dcraw so useful in the first place.

What do you think about the OpenRAW.org initiative?
Their requests are not reasonable. They want camera manufacturers to dedicate scarce engineering resources to writing detailed specifications of their raw formats, and of all algorithms used to process those formats.

What can the photographers of the world do in order to put pressure on manufacturers to build their RAW's open and documented?
Instead of barking at the moon, they should work with me and Phil Harvey (the author of ExifTool) to reverse-engineer whatever they want to know about their raw photos.

What is the reason for the manufacturers to be so closed with its raw specifications, structure and documentation?
As I said above, you don't get rich by sharing all your trade secrets with your competitors.

Is the Adobe DNG format the pragmatic solution for raw problems or maybe the same old song?
Neither -- it's another raw format that Photoshop and dcraw can read. It's good for small camera makers who don't want to design and support their own raw formats. And it provides good color matrices for many cameras, something dcraw didn't have before.

How would you explain in few words to our readers beginning in photography the advantages of the RAW format vs. the more popular JPEG?
JPEG is a display format. To save storage space, it removes all details not perceptible to the human eye. If any mistake is made while converting to JPEG (too dark, wrong color balance, etc.), it's impossible to correct. Raw photos preserve everything: The full color gamut of your camera, not your monitor. Up to fourteen bits of color depth, not eight. And the compression, if any, is lossless, at least in the shadows.

Which language do you feel more at ease with? English, C or Esperanto
Well, it would be pretty hard to have a conversation in C or write dcraw in English. With my wife, it's been all Esperanto since we met. I can understand Spanish-language TV and radio, but I can't converse in Spanish.


Talking with Juergen Specht

How did the foundation of OpenRAW.org come about?
Since 1999 I maintain a mailing list for Nikon digital SLR cameras mostly populated by professional photographers, RAW converter software developers and even Nikon stuff called D1scussion. Quite a while ago we noticed the problem with proprietary andundocumented RAW files, but it totally escalated into a public outcry as Nikon released the D2x and introduced the encryption into their white balance setting. To keep the discussion more focused and as we realized that Nikon is not alone with concealing information in RAW files, we started a new mailing list called OpenRAW and short after the public web site http://openraw.org/explaining the problem to the public.

Which is the main target of the OpenRAW movement?
Our main goal is to educate the public and camera makers alike about the problem with undocumented RAW formats and the danger about photo collections not surviving into the future.
At last count, more than 211 (!) different RAW formats exist today and since OpenRAW was started, Contax went out of business, Kodak stopped producing professional cameras, Konica-Minolta went out of the camera business (to be fair, it will probably live on as Sony) and Mamiya was sold with an uncertain future. Rumor is that in the next few months another camera maker leaves the market, so the RAW problem is not a problem we will have in some years, but right here and now.
But in some years it will be much much more problematic, because without a RAW documentation, programmers have little motivation to reverse engineer the RAW files from old, obsolete or unpopular cameras.
A documentation of past, present, and future RAW formats enables it for every programmer with some imaging background to write converter software and bring old and obsolete image back to live.
Here is a fantastic example of somebody who used old Russian RAW data of the widely unknown Russian Venus exploration and converted them with todays technology: http://www.mentallandscape.com/C_CatalogVenus.htm

How do you think the OpenRAW.org site could help in this matter?
Yes, before OpenRAW most photographers were not even able to understand that there might be a problem with their images in some years from now when camera makers go out of business, merge or decide to invent a newer, better RAW format and stop supporting older formats, leaving whole photo collections orphaned. Before OpenRAW most photographers expected that their photos last forever, just as analog photos (properly stored) do.
We also reached the attention of camera makers who thought its ok to abandon support of older cameras and older RAW formats in their newer software and added this support later back, however sofar no camera maker uses a truly open documented RAW format.

Why did the OpenRAW.org site conduct the 2006 survey? Which were the motivations and which are the conclusions after done?
Before the RAW survey there were not more than rumors and anecdotes about the real problems people already have or expect with RAW images out there, so it was hard to argue with camera makers, they simply ignored our claims because they thought we are just a handful of loud people.
The survey was designed to find out the experiences, requirements, preferences, and concerns of digital photographers and other interested parties regarding RAW imaging technology and create a public available report showing what 19,207 respondents with in average:
* 19.4 years experience with film photography
* 4.5 years experience with digital photography
* 3.7 years experience using a professional-level digital camera, and
* 2.3 years experience shooting primarily RAW images
told us. So far this was the first time somebody conducted such a survey about this topic and on this scale, so we hope that this report starts a thinking process for camera makers to rethink their RAW formats.
The report results surprised ourselves, first we never thought that more than a optimistic maximum of 10,000 people will respond in the 2 month time frame we wanted to conduct the survey, but in the end we stopped it after 6 weeks and had already double this number. This just shows how many people are actually concerned about their photo collection.
The second surprise was that actually more people use a third party RAW converter (based on reverse engineering of non-disclosed RAW information) than the software the camera makers provide themselves, plus that most people value workflow (the speed and usability they can work with their RAW images) over the sometimes better, but more clumsy to use software provided by camera makers.
The report is actually full of surprises, you might want to download and read it yourself: http://openraw.org/files/2006rawsurveyreport.pdf
The conclusions are clear: photographers don't want undocumented RAW formats, because they believe the images they shoot belong 100% to them.

What is the reason for the manufacturers to be so closed with its raw specifications, structure and documentation?
We can only speculate at this stage, but there might be a couple of reasons for their secrecy:
1) To create a vendor lock-in, so that you are required to buy the camera makers software for the camera makers hardware, adding more revenue from software.
2) Because if they are the only ones who see the specifications, they can change them whenever they want or invent something new.
3) To conceal the extensive post processing (especially in terms of bad pixel count of their sensors) they do before they actually save the raw data as a RAW file.


Is the Adobe DNG format the pragmatic solution or maybe the same old song?
I wish it would be a solution, but Adobe decided for marketing reasons that they allow a backdoor to camera makers to store information in so called private tags, which remain undocumented. So even if some cameras can save DNG formats natively, some of the information a camera decides to conceal can be saved inside the DNG format and it becomes another undocumented RAW format after all. Plus Adobe stopped documenting their PDF and PSD format after it reached a certain market share, so they can at any time release a DNG V2.0 format and decide not to document it. No, DNG is unfortunately not a solution.

How would you explain in few words to our readers beginning in photography the advantages of the  RAW format vs. the more popular JPEG?
The RAW format is actually one of the best things about digital photography, it contains the (almost) unprocessed data as seen by the sensor at the moment of exposure plus a memo of all settings used.
At this stage it practically can be compared to a undeveloped negative, which can then over and over again developed with RAW converter software.
Now imagine you kept a 5 year old RAW file from your camera and convert it into a TIFF or JPG with todays much improved computers and RAW converter software, you can get much more quality out of this image then at the time the image was shot originally. Software, processing power and algorithms improve a lot over time, while hardware (like a digital camera) become obsolete fast.
Or again, look at these examples and just imagine how software in 20 years can develop your RAW images, if they were documented: http://www.mentallandscape.com/C_CatalogVenus.htm
JPG files practically origin from camera internal RAW data, but get -processed- and converted with fixed algorithms and camera included processing power into a lossy format, JPG. Even the quality of JPGs as generated by todays cameras is quite good, RAW files give you a higher advantage to change settings (for example the whitebalance) after you shot the image, while its already applied to JPG files.
So in short: Through the conversion from the currently usual native 12bit sensor data to 8bit plus the applied JPG compression, JPG already lost a lot of data before it gets saved. RAW files can always be converted to JPG, but contain the raw data as the sensor saw it as the image was shot.

What can the photographers of the world do in order to put pressure on manufacturers to build their RAW's open and documented?
Photographers should go ahead and tell camera makers at every occasion that they want that their RAW formats should be documented, because this would create a win-win-win situation for
* photographers - they can be certain that their RAW images survive into the future
* software developers - they can spend their time writing better innovative software than spending it reverse engineering new RAW files while being unsure if reverse engineering is lawful or not
* camera makers - open documentation of RAW files would create trust among photographers and allow new imaging technology and usages created, camera makers would not even think of, thus increasing their market share.


en.wikipedia.org/wiki/RAW_image_format
Definition of RAW

openraw.org
Official site of OpenRAW initiative

www.juergenspecht.com
Personal homepage of Juergen Specht

www.cybercom.net/~dcoffin/dcraw
Personal page of Dave Coffin

Dicho por Maquiko el Sun 24-12-2006 20:29 (UTC)

Dicho por walter el Sun 21-01-2007 19:13 (UTC)

Dicho por Barry Pearson el Sun 04-02-2007 23:08 (UTC)

Dicho por Natxo el Mon 05-02-2007 12:39 (UTC)

Dicho por alberto el Mon 12-02-2007 14:54 (UTC)

Dicho por shard el Fri 23-03-2007 21:59 (UTC)

Dicho por aik el Mon 10-09-2007 12:22 (UTC)

Dicho por Lukasz Marszalenko el Mon 10-09-2007 12:56 (UTC)

Dicho por março antonio el Sun 09-12-2007 19:18 (UTC)

Dicho por Christian el Tue 22-04-2008 19:56 (UTC)

Dicho por erny el Tue 30-09-2008 20:49 (UTC)

Dicho por erny el Tue 30-09-2008 20:49 (UTC)

Dicho por cientifico el Fri 31-10-2008 08:05 (UTC)

Dicho por Wilson el Wed 06-05-2009 02:15 (UTC)

Dicho por vivi el Sat 10-10-2009 22:17 (UTC)

Dicho por Andrés Humberto Dávila Torres el Mon 25-01-2010 00:23 (UTC)

Dicho por film izle el Sat 03-04-2010 15:08 (UTC)

Dicho por replicaWatches el Wed 28-04-2010 01:55 (UTC)

Dicho por Louis Vuitton Speedy 30 el Wed 05-05-2010 11:39 (UTC)

Dicho por 123 el Thu 06-05-2010 06:26 (UTC)

Dicho por 123 el Thu 06-05-2010 06:26 (UTC)

Dicho por eq2 gold el Tue 25-05-2010 04:25 (UTC)

Dicho por pp122 el Tue 08-06-2010 07:55 (UTC)

Dicho por pp122 el Tue 08-06-2010 07:55 (UTC)

Dicho por pp122 el Tue 08-06-2010 07:58 (UTC)

Dicho por pp122 el Tue 08-06-2010 10:10 (UTC)

Dicho por alice el Wed 09-06-2010 15:46 (UTC)

Dicho por wholesale chape microsoft windows 7 office 2010 pr el Thu 10-06-2010 03:52 (UTC)

Dicho por el Mon 21-06-2010 03:51 (UTC)

Dicho por Moncler el Thu 02-09-2010 04:57 (UTC)

Dicho por SDFSDF el Thu 02-09-2010 05:19 (UTC)

ÚLTIMOS COMENTARIOS

«1092012045PMwow gold | World of [...]»

Dicho por SDFSDF en 22.LDNM RAW format, the captive photo

«thank you for your sharing [...]»

Dicho por Moncler en 22.LDNM RAW format, the captive photo

«Sólo los que vivimos en [...]»

Dicho por nikita en 20.LDNM El Cabanyal. El arte de resistir

«Hola, soy una chica que [...]»

Dicho por Andrea en 18.LDNM Boda gitana

«la organizacion master of street [...]»

Dicho por principe m.o.s en 16.LDNM The Wu Tang Clan Manual (libro)

«la organizacion master of street [...]»

Dicho por principe m.o.s en 16.LDNM The Wu Tang Clan Manual (libro)

«FELIZ DIA HERMANOS [...]»

Dicho por el niche en 16.LDNM The Wu Tang Clan Manual (libro)

«FELIZ DIA A TODOS MIS [...]»

Dicho por MAESTROGUILLO en 16.LDNM The Wu Tang Clan Manual (libro)

«excelente entrevista, excelente artista, pa [...]»

Dicho por atoq ayahuasca en 31.LDNM Velandia y La Tigra:

«EXCELENTE ENTREVISTA, EXCELENTE ARTISTA, [...]»

Dicho por atoq ayahuasca en 31.LDNM Velandia y La Tigra:

Otros artículos de Javier García Diz (3)