noel_carboni's profile

116 Messages

 • 

1.9K Points

Wed, May 18, 2011 11:39 PM

Closed

Lightroom/Camera Raw: Add a Camera Raw Option to Prevent Writing Back into Input Files

Given:

Under some conditions Camera Raw writes data back into (overwrites) its input files. Example: A JPEG file.

Under other conditions Camera Raw maintains this same kind of information in a separate place (e.g., Sidecar XMP or central database). Example: A CR2 or NEF file.

Assuming one uses Camera Raw to open out-of-camera original files as many photographers do, depending on what mode one has captured the images in, Adobe is inconsistent about whether to keep its hands off them or overwrite them... This seems to be because some formats are proprietary and some are well documented. From a programmer's perspective, this makes perfect sense.

Trouble is, from a user's perspective, this behavior cannot be described as anything but inconsistent.

Personally, I do not want my original out-of-camera JPEGs updated/rewritten under any circumstances.

Camera Raw will not touch a proprietary raw file, such as a Canon .CR2 or Nikon .NEF. There's a whole process for remembering settings in a separate database or sidecar XMP files. So far so good.

However, if you open a JPEG, TIFF, or DNG through Camera Raw, data WILL automatically be written back into it to tell another run of Camera Raw in the future what settings you used - without the software ever having warned you it will do so.

It is true that some functions EXPLICITLY rewrite input files. You can ask the software to write new thumbnails back into DNG files, for example. This seems fine - the user has instructed the software to overwrite the file, and the user is in charge, after all.

Overwriting/rewriting an input file without being instructed to do so is NON-INTUITIVE BEHAVIOR for any application. Simply put, I would not expect an input file to be overwritten by Camera Raw.

And we do see that it causes people confusion and surprise from time to time. You may right now be reading this in disbelief. I recommend you go test it for yourself (on a copy of one of your original JPEG files).

The original file being overwritten is a chief reason why I don't configure Photoshop to open my JPEGs through Camera Raw.

Adobe:

Please give those of us who don't want our input files overwritten an option for using the database/XMP sidecar instead in EVERY case.


Thank you.

-Noel

116 Messages

 • 

1.9K Points

10 y ago

Zalman,

Clearly you understand the concept of never wanting an original file overwritten...

The software is MALFUNCTIONING as it is now.

But if you're worried over consistency with past (incorrect) operation confusing existing users, just add the new "Never Overwrite Original Files" option I mention here and default it to off. You can still use embedded info if you find it, but this would prevent you from writing embedded info back into ANY FILE TYPE.

This is not hard to understand!

Adobe presents Photoshop is a professional product - and I'm sorry to be blunt - but honestly you folks treat operations on your users' files in a very amateur way indeed!

-Noel

116 Messages

 • 

1.9K Points

10 y ago

Sigh. Now we have Camera Raw 7 and no fix for this.

I think we can see how well Adobe listens to its customers.

-Noel

4.5K Messages

 • 

76.3K Points

Hi Noel, I can relate. Consider casting a vote on this one too:

http://feedback.photoshop.com/photosh...

I think this issue qualifies as a longstanding issue of the "low hanging fruit" variety.

116 Messages

 • 

1.9K Points

Yes, or in Adobe terms, JDI. They might argue that it's a design change, but it's a design error so that's what's needed!

-Noel

6 Messages

 • 

210 Points

10 y ago

I want an option to force all filetypes to use xmp sidecar files only, and not to modify original file.

While xmp data is written inside of tiff and jpg files, this makes backup to add unnecessary media consumption as those files are changes (e.gg size and CRC do not match to previous copy). This is totally unnecessary use of media space, and very annoying especially with off-site backups.

Further, this unnecessary change of original file increases possibility of file corruption, should something go wrong.

I agree, that in some cases where e.g. raw+jpg are used, this will require special renaming measures during import, but should not be an issue.

For me, changes to original tiff-files are risking hight quality scans, and are unnecessaryli increating size of backups (incremental, and differential, running always full backup is not an option).

This reply was created from a merged topic originally titled
Lightroom: Option to force all filetypes to use xpm sidecar only (ptotect original file).

13 Messages

 • 

370 Points

10 y ago

Maybe this is possible already, but I didn't see it. Can't Lightroom give us the option to write the metadata to a XMP sidecar file when using DNG? I know one of the advantages of DNG is the lack of sidecar files, but for those of us backing up our photos (especially when it is offsite, like Backblaze or Carbonite) it is a major pain in the... Every time I slightly modify a photo or add a tag it will have to reupload the whole file. Staying with the original RAW file is not an option, as they are completely uncompressed (twice the size).

This reply was created from a merged topic originally titled
Lightroom: Sidecar files when using DNG (for offsite backups etc.).

4.5K Messages

 • 

76.3K Points

Bridge(/ACR?) will write xmp as sidecar if DNG is read-only, but Lightroom will not, ever. Lightroom *will* read xmp from DNG sidecar, *if* xmp in sidecar is newer than embedded xmp.

116 Messages

 • 

1.9K Points

That's interesting... So there already is logic at least partly there to handle settings input from possibly different sources. Thanks for that tidbit, Rob.

-Noel

2 Messages

 • 

112 Points

10 y ago

Develop settings and metadata is saved to the database, and if the user chooses again "closer" to the actual file. For raw files to a xmp sidecar file, and to jpg files (and others) to the header of the file itsself.

This inconsistency ruins one huge advantage to saving the settings/metadata to a place other than the fast database: whereas backing up the work done on raw files, is a dream come true (as during an incremental backup or synch to a network storage only the small xmp files are copied/backed-up), this does not hold true for the jpgs, they must be backed up completely every time anew.

Please add support for xmp sidecar files for jpgs and other non-raw files, and thus offer the option of leaving the jpgs untouched, just like the raw files. Thanks!

This reply was created from a merged topic originally titled
Lightroom: XMP files for JPGs please.

116 Messages

 • 

1.9K Points

10 y ago

Thanks for adding your thoughts, guys. It's good to see this idea getting some traction.

-Noel

4.5K Messages

 • 

76.3K Points

Hi Noel,

Technically, it's cake to read/write from sidecar vs. embedded, *but* there is no spec for how to name files, in the case of RGB sidecars. In other words, the xmp spec says: "same basename, xmp extension", but then if one shot raw + jpeg, and imported separately, the xmp filename maps to same, and saving xmp for one would overwrite the other. One idea:

* include the extension for rgb files, thus asdf.nef would have asdf.xmp, but asdf.jpg would have asdf.jpg.xmp. But that has the problem that asdf.jpg.nef is a legal filename for a raw nef file (stupid, but legal), which would then have the name conflict problem again.

So, for 100% robustness, it almost demands a name change for the raw's too. In other words, for zero possibility of name conflict, one would need to change to asdf.nef.xmp for raw xmp files. But, then it would not be backward compatible.

Another idea, use different extension for rgb xmp files, e.g. keep asdf.xmp for asdf.nef sidecar, but have asdf.jpg use asdf.jxmp for it's sidecar name. But then asdf.tif would need a different one too, e.g. asdf.txmp. Then if there was a png, it would need a different one too, etc.

So, there are some tricky issues to be worked out. Not insurmountable, but tricky.

My sense is that we will *never* see this change. Hope I'm wrong, but I do not expect it. Why? Because I think these issues were already thought about, and decision made - not to look back... Adobe thinks of sidecars as a necessary evil when file format does not support embedded xmp, as opposed to a plus for people who prefer sidecars for various reasons... DNG supports embedded xmp, and that's the file-format Adobe is pushing...

Cheers,
Rob

4.5K Messages

 • 

76.3K Points

10 y ago

Technically, it's cake to read/write from sidecar vs. embedded, *but* there is no spec for how to name files, in the case of RGB sidecars. In other words, the xmp spec says: "same basename, xmp extension", but then if one shot raw + jpeg, and imported separately, the xmp filename maps to same, and saving xmp for one would overwrite the other. One idea:

* include the extension for rgb files, thus asdf.nef would have asdf.xmp, but asdf.jpg would have asdf.jpg.xmp. But that has the problem that asdf.jpg.nef is a legal filename for a raw nef file (stupid, but legal), which would then have the name conflict problem again.

So, for 100% robustness, it almost demands a name change for the raw's too. In other words, for zero possibility of name conflict, one would need to change to asdf.nef.xmp for raw xmp files. But, then it would not be backward compatible.

Another idea, use different extension for rgb xmp files, e.g. keep asdf.xmp for asdf.nef sidecar, but have asdf.jpg use asdf.jxmp for it's sidecar name. But then asdf.tif would need a different one too, e.g. asdf.txmp. Then if there was a png, it would need a different one too, etc.

So, there are some tricky issues to be worked out. Not insurmountable, but tricky.

My sense is that we will *never* see this change. Hope I'm wrong, but I do not expect it. Why? Because I think these issues were already thought about, and decision made - not to look back... Adobe thinks of sidecars as a necessary evil when file format does not support embedded xmp, as opposed to a plus for people who prefer sidecars for various reasons... DNG supports embedded xmp, and that's the file-format Adobe is pushing...

Rob

116 Messages

 • 

1.9K Points

10 y ago

Cake to you and I seems to be an obstacle to Adobe, referencing how long we've been discussing this.

Regarding the name ambiguity, couldn't the very same XMP file carry multiple sections, one for each similarly named input file?

-Noel

4.5K Messages

 • 

76.3K Points

|> "couldn't the very same XMP file carry multiple sections, one for each similarly named input file?"

I suppose it could - but I haven't thought through the implications. - definitely not insurmountable...

~Rob

267 Messages

 • 

4.2K Points

10 y ago

Noel you wrote

"Personally, I do not want my original out-of-camera JPEGs updated/rewritten under any circumstances."

Then you should not use LR or the Bridge for they are metadata editors. I personally think that that good. Raw camera image data is not altered. Which is what I care most about. However when it come to Jpeg Image data other application will alter this data. Only Camera RAW data seems safe for now. I also feel the some metadata should not be changeable like some EXIF fields not all though. I also feel the best place to record RAW conversion setting is better placed there then in external locations like databases and sidecar files. I like the ability of keeping information about an image in the image files themselves. In standard formats like EXIF IPTC etc. My vote would be to do away with the ACR database and sidecar files in favor of keeping RAW setting in metadata and perhaps even supporting versions of raw conversions. I also feel more application should utilize this data like web galleries generators should get copyright, title, description, EXIF and other metadata information. I also have no objection of LR building logical database over this information quick indexed quires are useful for many applications.

4.5K Messages

 • 

76.3K Points

Nobody is suggesting the removal of the option for embedded metadata. But for those of us who understand all the ins and outs and would still prefer sidecars or database only, we would like the option. I can even imagine multiple checkboxes:

1. Embedded.
2. Sidecar
3. Database (if non-Lightroom environment).

That way, user can have his/her cake with frosting, or without, and eat it too...

Rob

116 Messages

 • 

1.9K Points

10 y ago

John, you say that it's good that image data is not altered, that it all should be done via metadata, yet you advocate the Adobe software writing the metadata into the original file.

It's not a big leap to understand that some folks think ALL the data in the original file is something we don't want altered, is it?

Like Rob says, I'm not looking to change the product to avoid writing in any metadata-capable file for everyone. I'd just like the OPTION to do that. I'd even expect the default would be to work as it does today.

I don't understand your comment "However when it come to Jpeg Image data other application will alter this data". From my perspective, properly written computer applications in general have INPUT files (which they open) and OUTPUT files (which they save), and if one doesn't save over an input file one really doesn't expect it to be altered!

-Noel

267 Messages

 • 

4.2K Points

10 y ago

If a Program like Photoshop opens an as you call it an input file and you make updates to it and you use menu File>save or with CS6 have left the default autosave preference setting un changes does not the input file get over written with the exception of RAW files. For Photoshop can not write Camera RAW files. You have to got to be careful or defaults will kill you. No!

116 Messages

 • 

1.9K Points

10 y ago

First off, you clearly don't understand AutoSave.

But secondly, if *I* choose File - Save, then I'm commanding the software to save over the file. That's not the same as overwriting my file when I just choose File - Open - not by a longshot!

I sense that you're trying to get me to "think differently" about files and applications, or maybe to try to teach me basic computer usage. Thanks for the effort, but with 36 years software engineering experience under my belt neither of those things is likely.

-Noel

267 Messages

 • 

4.2K Points

10 y ago

As you put it its good to have options. Users have different needs and preferences. There is no clear one right way to do something as complex as asset management. For some making use of a database is the right way to go. Even then there having redundant information in image file may give them more options for an application may come about that the may feel a need to may know how to use metadata and know nothing about how to use the users fantastic database. Databases can facilitate many thing some user want and need to do and they can and do co exist with metadata. Many do not want to become or have to become software engineers they want a life. Some even want to put that behind them. Adobe is making that harder for me to do for their new releases are not up to snuff.

Yes I know autosave saves versions of a document your editing not over the original was just testing you there...

However if you give a novas user a loaded gun what prevent them form shooting themselves in the foot when they use File>Save. I'm willing to bet that more the one new user has resized an image to email to someone using File>Save only to find out later on they no longer had an image file worth printing.

1 Message

 • 

68 Points

8 y ago

I like a clear separation between Digital-Asset-Management and editing of my video, audio, image and office files. With this I use for DAM other tools and for editing Lightroom, CaptureOne and others. I dislike updates or modifications of any big video or raw image files just because of minor meta data updates. That is why I set read-only for video, dng, jpg, ... and I really would appreciate consistent settings for xmp sidecar handling in Lightroom. GeoSetter, IMatch and JPhotoTagger are good examples of tools which are respecting consistent xmp sidecar settings.

In addition to the original posting idea/proposal I like to protect not only original out-of-camera JPEGs under any circumstances, but also ALL other original files like DNG, PNG, ... If possible this could be realized in addition through detection of read-only attributes of files. E.g.: Instead of error message when exporting XMP data ot DNGs wiht LR, there could be an automatic or user confirmed generation of XMP sidecar.

Roman

1 Message

 • 

64 Points

8 y ago

I was constantly annoyed that .jpg files downloaded from phones got timestamps changed, making it impossible to find out even which month they ware taken, until I found that Lightroom is destroying the (file) timestamps.
As Adobe seems not to know (or does not care) - not all phones embed EXIF data in photos.
Thus I got most phone photos with timestamp destroyed, which made me extremely angry! Why the hell you do this? DAM software destroys metadata!!!

It should be not so complex to put option about whehter to modify any existing file or just create sidecar file for it.

It makes much more sence that file's timestamp equals exif's "create time/date" (if any).