sidecar files
With the 2019 update all my sidecar files are reset. I have the strong idea that the files from 2019 are okay, but the files from 2018 are overwritten with some kind of default.
Anyone else with this problem?
-
I've not well understood the need for sidecar file.
It seems that, when activated, everytime you enter in the edit mode, make a modification and then go to the Browse, the system will generate or update the existing .on1 file.
I tried to delete the sidecar file, switch off the computer and then run again Photo Raw.
I would have expected that without the file the system would not be able to remember the changes made. On the contrary, once I opened the file in the editing mode I found all the filters and modifications made previously. When I came out of the editing mode, the system created a new file.
I wonder then what is the sidecar file used for? And what happens if I delete it or avoid its generation every time...
0 -
Andrea, sidecar files are an option you can turn on or off in preferences. ON1 maintains a database of your edits. A couple benefits of sidecars are:
1: There is no way to back up the On1 database of edits. In the event you do not have side cars turned on and the database becomes corrupted or lost for any reason all you edits will be lost. ( Think of sidecar files as a backup of your edits )
2: If your photos are on a networked drive and you have sidecar files enabled you can edit the photos from multiple computers and all computers accessing the networked drive will see the same edits. Without sidecar files turned on you can edit from multiple computers BUT each computer will only see the edits made on that computer.
3: If, for some reason, you wanted to share your photo and edits with another user of On1 you can simply share the original photo file and the associated .on1 sidcar file and XMP file with that use and they will have all you edits etc..
0 -
David,
Thank you for the prompt and very clear explanation. Please let me have few additional clarifications:
- in which case PhotoRaw generate the .onphoto file? I noted it is generated when I add multiple layers but I do not know if it is the only case
- If I keep the .onphoto file and remove the sidecar, can I use the onphoto as "backup of my edits" or to use it to edit the same picture from multiple computer or to share picture and edits with with another user of ON1 ?
Basically the issue is: do we really need to store 4 different files for each edited file (i.e. .NEF, .on1, onphoto, xmp)?
The optimum would be to have a single file which incorporates all you need for backup/share edits/metadata iclusion, etc...
Last but no least, which is the suggested workflow in term of ON1 Photo Raw file storage?
Sorry for the multiple questions, I really like PhotoRaw but there are some aspects that I still have to understand
0 -
andrea,
- onphoto is On1s own file format used for layers. It is created any time you have 2 or more layers open in the editor. It replaces PSD in older versions.
- onphoto is not the same as a sidecar. All edits including the onphoto edits are kept in Photo Raws database and a copy of those edits are kept in the .on1 sidecar if sidecars are enabled. You don't have use the .on1 sidecar and can turn them off in options.
- xmp is used to maintain metadata compatibility with other programs. There is no option to turn xmp files off.
- NEF is your file, since On1 is non destructive, it won't touch that file at all. The .on1 sidecar would be the single file to keep edits and metadata if other programs knew how to read it. It also provides a way to back up your work. Without it you risk a chance of losing years of work if the database is lost for any reason.
- You can keep your photos on your hard drive organized in any way you want. If you back those photos up, you'll want to back up the .on1 sidecars with them.
0 -
Rick,
thank you very much for the celar explanation.
Does Photo Raw manage DNG file in the same way of Lightroom or Camera Raw?
To better explain my question, considering that DNG format has the capability to incorporate in a single file the RAW data, the Metadata and store all the edits performed (at least with Lightroom or CameraRaw), usually when I work with Lightroom or Camera Raw I convert NEF file into DNG, then I edit the DNG. In this way I store only the edited DNG which (if used with Lightroom or CR) preserve the edits, the metadata and the original RAW data.
Do you think it would work also with Photo Raw?
0 -
Andrea,
It does not matter what type of file you are editing DNG or raw.
As soon as you go into edit on the file a .on1 sidecar will be generated (if turned on) as well as a .xmp
If you add a layer an onphoto file will be generated.
0 -
All is clear now thank you!!
Even editing a DNG, you loose all the edits performed when you move to another PC without moving also the sidecar.
Basically the capability of the DNG to save the edits works only with Adobe producs.
0 -
Sorry to bump into the thread but I found these infos very interstings.
Though why are there 2 versions of edit datas: one in the database (in appdata... roaming... On1)
and one on the sidecar files (.on1) in our photo folders ? (if I did understand your explanations...)Is that just for a security matter?
0 -
Nicolas, David explained all of that in his first post above. What are you not understanding?
0 -
Damn Rick you're def. right.
Sorry, I must have jumped over his post as I'm reading the forum while at work, so things like this happen to me from time to time.. Really sorry It's now very clear indeed.
Wishing you a very best new year's eve !
0 -
This thread is three years old and I am understanding everythig written so far. But with PR2022 things have changed. Can somebody tell me "who wins" after a restore of the database? In case I modify a picture after a backup of the database, an .on1 file will be written. What happens after a restore, which settings will be applied - those from the database or those from the file? ... and will the database be updated with the settings read from the file or vice versa if the other case applies?
0 -
The User Guide (page 240) says it backs up "All of the edits made on any image in the Edit module." I'd assume that a restore would overwrite any edits made after the backup. You'll have to ask support for a definitive answer.
0 -
Thanks Brian! BTW: in cluster terminology this is called a split-brain condition. I have been curious and made a test. I backed up the catalog, edited one picture (just brightened it up by pulling up the exposure), moved to a different directory and closed the program. After that, I restarted PR, restored the backup and moved back to the directory with the modified image.It first appeared with the correct exposure and was brighted up after an instant. So the .on1 file "won".
One could argue whether this is a desired behaviour (in my eyes it's not - always) but in any case it's good to know what happens.
In my opinion it would be nice to extend the restore with a choice (a checkbox or whatsoever) whether .on1 files which are younger than the backup should be replaced by the contents of the (restored) database. I am aware that that's riksky if used with different computers whose clocks which might not be in sync, but that could be pointed out with a big "ATTENTION" added.
0 -
Tricky. Does that mean that if I use sidecars and synced say a whole folder by mistake and realized too late for undo I couldn't restore the edits to before the sync? I suppose you could always delete the sidecars before the restore, but would the sidecars then be created by the restore?
0 -
Hi Gerry, you asked "... but would the sidecars then be created by the restore? " Nope. I just deleted the sidecar file and restored data. The image looks fine again, but the .on1 file is "missing".
0 -
Hi Ralf, please bear with me a moment. You first made a backup, then brightened an image. After deleting the .ON1 file you restored and the image stayed brightened with no .ON1? So in effect what was in the database (the edits) was not overwritten. As I see it, you cannot then return to a point before you brightened the image. I agree, you must have the choice to overwrite the edits and/of create new sidecars.
0 -
Hi Gerry! I should clarify this: I made a backup, then brightened an image. After a restore the image was first normally exposed but then, after a small delay, "rebrightened" again. I suppose that happened, as the .on1 file was read and i think these changes were written to the database as well.
In a second step (in order to verify my assumption), I deleted the sidecar file, restored the same backup again and got - as expected - the unbrightened image, but no newly written .on1 file.
You can return to the point in time a backup was made, but you'd get rid of the sidecar files.
0 -
Ok, clear. Thanks for testing this.
0 -
Thanks Ralf, this should be reported to tech support.
0 -
@Gerry: No problem - I was curious anyways.
@Brian: I "opened a ticket" at "The ON1 Photo Raw Project", not the tech support, since I am not sure if it can be called a bug. It depends on how you look at it - you could also call it a feature ;-) There are two competing backup systems for the edits you made, which might be an advantage, but in my eyes the handling is somewhat messy.
At least it should be noted in the docs.
0 -
I think you should report it to tech support too. A restore that does not restore is a bug. A feature to allow us to choose which gets picked for restoring does belong on the Project page. IMO.
I'm planning on doing some testing of my own to write up for the engineers.
0 -
Hi Brian, in the meantime I reported the issue to them and also referred to this page.
Please let us know if your testing yields interesting results.
0 -
Thanks Ralf, will do.
0 -
I think that the current behaviour makes sense if one looks at backup/restore being part of a process to get system back as near as possible to before disaster struck with minimal loss of edits.
If one has done some editing since the last backup and have sidecar files enabled then the fact that the .on1 files are used to update the database from the restore makes sense, you don't lose the edits that are in the sidecar files but not the database.
Also using sidecars to repopulate the database is how things have worked for as long as I have used On1. I have just moved to a new M1 iMac and decided to build it completely from scratch rather than any migration, just to ensure I got rid of any residual stuff built up over the years. So I had a clean On1 database and it got rebuilt from the sidecar files.
What seems to be being asked for here is for the backup/restore to be a snapshot mechanism that can be used to wind back everything to the time of the backup. I think something like Time Machine on Mac might be better for this and use this to restore photo files and sidecar files as well as On1 database etc. That way everything is in sync.
0 -
The documentation says the edits are being backed up, and they are when the internal database containing those edits is backed up. The .on1 sidecar files are not backed up at all. They are not in the .onbackup file which is actually a .zip file without that extension. Put it on and you can open the file in any zip manager.
Now the problem from my testing is that the internal database is not getting restored correctly. If I backup the database then make edits to an image those edits are saved in both the sidecar file and the internal database. Quit the program, trash the .on1 sidecar with all its edits, relaunch the program and run a restore. The edits from the .on1 file are gone, the database should have been returned to the state prior to the edits so they should be gone from there as well but they still show up when you open the file in the editor again. Where did they come from? I can understand the Browser previews still showing the old versions rendered with the edits but why does the editor retain them?
In my tests I also ran into a situation where I was being told that there were hundreds of missing folders from locations that have nothing to do with Photo RAW. Why would it care about anything to do with Capture One which I tested for a couple of days last month and is no longer on my system. I had to go through each and every one, one by one, and either tell the program to ignore it or recreate the missing folder. That was very time consuming as my only alternative would have been to abort the program in the middle of the restore! I don't think so.
There's a lot here I don't understand.
0 -
I hit the issue with missing folders while doing pre-release testing and put in a bug report then.
I think that it is not so much an issue with the backup/restore process but more with the fact that On1 insists on adding things to the database whenever folders are browsed. This also leads to the issue with lots of unknown cameras ending up in Preferences and extra keywords appearing in keyword list that come from example photos etc.
My list of missing folders was so long that I never managed to clear it without the restore crashing.
However, I think it comes up as part of tidying up after the restoration has completed. As a test I did a restore onto my clean system and when I looked at On1 after the crash it did seem that things like presets, catalogue folders etc had been restored, though I did not do a completed check before I cleaned everything off again.
Having set up my new build I did some more backup/restore tests. This time the list of missing folders was only a few and these were ones I had deleted from one of my external disks that also holds some of my image files. I successfully deleted these and was able to click Done to complete the restore.
Without deleting any more folders I then did another backup/restore and this time there were no missing folders. This confirms my suspicion that it is rubbish left behind in the database, and this then gets cleared if one skips the relocation of missing folders during restore.
I am planning on doing regular backup/restores to keep the database tidy so that I will be able to do a restore for real if I ever need it. If one of these fails then I can always go back to using the Time Machine backup.
I have updated On1 with my findings. It would appear that a number of issues are being investigated following the release of the backup/restore facility.
0 -
@David: You wrote "What seems to be being asked for here is for the backup/restore to be a snapshot mechanism that can be used to wind back everything to the time of the backup."
It depends on what you expect from a backup. I am dealing a lot with databases of different purposes and would normally expect a backup to be a snapshot of the current state ... and that's exactly what the manual states: "All of the edits made on any image in the Edit module". and "To restore your system to the latest backup" That's what I call a "full backup". Any changes made since then are stored in the sidecar files and replaying them acts like an "incremental backup" . Both kinds of backup serve their own purposes and I understand well that an incremental backup serves your needs and comes in quite handy - for you, in your particular situation.
Other people (like me) might rather (or also) work with full backups - as a snaphot - in order to save (the state of) their work at a particular point in time just like you might want to save a text document while writing. I think there's nothing wrong with that. It's just expecting being able to do what the manual says. Furthermore, don't forget that PR is not a mac-only program. As far as I know windows does not have anything like time machine .
I think both ways of handling a restore are usefull, that's why I wrote it would be great to have the choice.
0 -
@Brian: having read your last post, I did test it again. I made some edits to an image (just brightened it up), then backed up the database, after that I darkened it. After a restore the image was still darkened. Then I deleted the .on1 file, restored the backup again and got the brightened image - in grid view as well as in the editor. Did not experience the strange behaviour of PR as you did. Hope this helps more than it confuses :-)
0 -
I'm with David about using other backup software. I use Time Machine to do hourly incremental backups and I do a full clone backup of both my boot drive and the drive with my photos each night. If I only need to recover the previous state of an image I can simply delete it from Photo RAW then use Time Machine to recover it at whatever earlier state I need. This gets rid of any confusion about whether the internal database or the sidecar edits are in charge; the sidecars alway win in this scenario.
I started writing up something to send to the company yesterday but I need to think it through some more so I get it all right in my report. There definitely needs to be a better way to handle "missing" items than spending 40 minutes dealing with each of them individually. I also still do not understand why it wants to know about folders for 3rd party software I deleted weeks ago and why I don't see any references to those items in the actual backup files.
0 -
I can not fully agree. This might be fine for Mac users, but what about all us "Time Machine"less Windows users?
0
Please sign in to leave a comment.
Comments
35 comments