Skip to main content

Lost edits

Comments

24 comments

  • Rick Sammartino Community moderator

    If you use the Browser in On1 to move folders, all of your Edits and metadata should stay intact and in sync with the database. If you use your PCs Browser to do that, moving a folder will break the connection with the database, but the sidecars should still contain everything.

    If you restore a sidecar from Backup, try refreshing the Preview (in the Photo menu). Just because you dropped a sidecar there doesn't mean On1 has detected it. A refresh should fix that.

    0
  • Jos Deschagt

    Hi Rick, restoring and refreshing doesn't always work. Sometimes it does, sometimes not. Very confusing.

    Moving folders and preserving edit must really really be made more robust by ON1. If I cannot trust my edits to stay, or that I can restore them from backup, then I need to find another software, it's that simple.

    Is this being looked into by support/development? 

     

    0
  • Rick Sammartino Community moderator

    We have no idea what's being looked at. You can raise your issue with tech support if you like.

    https://on1help.zendesk.com/hc/en-us/requests/new

    0
  • Holger Danielsson

    Just for information: I also lost hundred of my edits today, when  moving a folder inside the Browse-Module on my Windows system (PR 15.1.0.10100).

    Great parts of my edits have gone away, especially local adjustments, where the size of some ON1-files with masks was reduced from 1,2 MB to 46 or 47 KB .

    I already submitted a report to tech support, containing some examples and other info. I will publish any important replies here.

     

      

    0
  • Brian Lawson Community moderator

    I had a similar situation yesterday. Sometime the day before the program hung during the shutting down, syncing with cloud sync process. I had to force quit the program. What I did not notice until late yesterday is that all of my Albums were empty. Fortunately I use keywords so I was able to rebuild them except that every single image that had been in an album appeared to have lost its edits.

    I found that moving the photos into a sub-folder then moving them back to their original location would restore the edits in most cases. For some of them I had to resort to restoring the .on1 sidecar from a Time Machine backup. In a few cases the losses were permanent.

    The point is to try the different recovery techniques that others have posted on these forums before assuming any loses are permanently gone.

    0
  • Holger Danielsson

    Brian: I think that this is an other problem than mine. PhotoRaw never hung during shutdown and I never realized any other problems when running it. It really runs stable now for me (until yesterday).

    But some sidecar files are reduced dramatically in file size, so that most of the edits are lost. I think that only the starting settings from the 'AI Auto' Tool in 'Tone and Color' are still available then. This also happens, when I restore them from my external backup drive. As soon as PhotoRaw writes to the sidecar files, they are wrong.

    As I don't have any idea to solve this, I decided to stop editing images, until I will get an answer from ON1 support.

     

    0
  • Brian Lawson Community moderator

    I understand that the cause of the problem is different but the symptoms are the same — edits are not showing up any more. My point was that they may still be there and one should check before giving up on recovering them.

    Also, if you have an incremental backup like Time Machine provides, you can restore the .on1 sidecar from before the loss and you should be able to recover the edits. You may have to do the move to a new folder then move back trick before the program will recognize the restored sidecar.

    0
  • Holger Danielsson

    Thanks for your ideas, Brian, but the move back trick, you mentioned, also doesn't help. 

    As soon as I open the folder in the Browse Module to see the images and then immediately close PhotoRaw, without using the Edit Module at all, the sidecar files are reduced in file size like I described already above. Really crazy.

    And I use a really modern PC with a lot of Ram, a good NVIDIA graphics card and system settings optimized for PhotoRaw. And I use this PC only for photography, only six programs installed, not even a browser. So this shouldn't be a cause either.

    So I think, we better stop here until I have more information. 

    0
  • Brian Lawson Community moderator

    Let us know what support has to say. I hope you've got a good backup of your photos folder(s).

    0
  • Rick Sammartino Community moderator

    My photo raw froze during shutdown 2 days ago as well. It was sitting for almost 2 hours until I came back and killed it.

    I just checked and don't see any edits missing and the albums seem to be intact too.

    0
  • Holger Danielsson

    But I think that PhotoRaw urgently needs a backup option for their internal databases. Together with the backup of the sidecar files this should prevent the loss of any work. 

     

     

    0
  • Brian Lawson Community moderator

    All of the databases are in your user's Application Support or Roaming ProgramData folder and the sidecars are alongside their photos so it's pretty easy to set up a nightly backup to cover your bases. 

    0
  • Holger Danielsson

    First of all: my lost edits have nothing to do with moving folders!

    Brian wrote: Let us know what support has to say 

    Only a standard response, nothing really important, although I have sent them all results, I found.

    Brian wrote: I hope you've got a good backup of your photos folder(s)

    Yes, I have a backup of all sidecar files. But this doesn't help at all for 21 of about 600 files, because PhotoRaw will not show the edits and even destroys these files additionally. Affected are only the 21 images in one single subfolder.

    Therefore I have analyzed the structure of these files and found the reason, why this happens reproduceable: PhotoRaw has created some sidecar files with a wrong internal structure. All edits are still contained in the files, but PhotoRaw cannot apply them. Unfortunately I cannot say, why and when this happened.

    With this knowledge it was rather easy for me to repair these sidecar files, so that PhotoRaw was able to read all my edits (tone and color, effects and especially local adjustments with all its masks) of the 21 buggy sidecar files again and shows them as they originally have been.

    Details are very technical, although easy to perform. If you want to know them, you can send me a private message, if this is possible in this forum. (I don't know).

     

    0
  • Brian Lawson Community moderator

    I know that moving folders was not the cause of the problem, it was offered as a way to recover the lost edits. You don’t have to tell me about how difficult it is to move 600 images around, I just went through it for 14,000 of them.

    No, there isn’t any way for us to email one another through the forums. Go ahead and post the details here please. I would also ask that you pass that information along to support so it can be given to the engineers.

    0
  • Holger Danielsson

    Brian wrote: You don't have to tell me about how difficult it is to move 600 images around, I just went through it for 14,000 of them.

    I wrote a tiny script, which recognized the damaged sidecar files in some seconds.

    Brian wrote: Go ahead and post the details here please.

    Too much technical details and not interesting for most users. If there will be another user with similar problems, I will find a way to contact him and send some information.

    BTW: the sidecar files are JSON files, which is an open standard data interchange format, that uses human-readable text. You can prettyprint the contents of these files for example with the tool 'jq', which is available for Windows, OSX and Linux.

    Link: https://stedolan.github.io/jq/download/ 

    Call it from a terminal 

    jq . sidecar-file.on1

    and redirect the output to a file. The exact call may depend from your system.

    0
  • Brian Lawson Community moderator

    Please post the details. It doesn’t do much good to come here and tell us you have a solution but then you won’t tell us what it is. I’m a software engineer, I want the technical stuff. 😀

    0
  • Holger Danielsson

    Ok, if you want...

    But please tell me before, why the contents of the Code block is nearly black, so that it's impossible to read it. Only when you select the text with the mouse.

    0
  • Brian Lawson Community moderator

    I don't know the answer to that, sorry. I'm not even sure what you mean. I've looked at the .on1 sidecars in a text editor (I use a Mac program called BBEdit) and I've always been able to read their contents.

    0
  • Holger Danielsson

    I'm not even sure what you mean. I've looked at the .on1 sidecars in a text editor and I've always been able to read their contents.

    Good for you on OSX. But this doesn't works for me with my text editors neither on Windows nor on Linux.

    Note: I have replaced not so interesting parts of the sidecar files with dots.

    In the first Code block you can see a good sidecar file. In section 'photos' you will see the UUID for the corresponding RAW file. You will recognize for example the capture date of the image, the name of it and another UUID, which identities the corresponding settings entry.

    This 'photos' entry is accompanied by an entry in the 'settings' section. The UUID is the same as that one, which is already mentioned in the 'photos' section. Very important is the 'data' entry, where all edits are stored as a base64 encoded readable xml file.

    {
    "photos": {
    "0ff2c369-606a-4965-a333-9a4a1d421002": {
    ...
    "metadata": {
    ...
    "CaptureDate": "Tue Jul 24 11:29:37 2012",
    ...
    },
    "name": "IMG_2012_07_24_4987.CR2",
    "settings": {
    "com.ononesoftware.vcb.processplugin4": "5e70c4e7-389a-4fc9-8a9a-732865032b55"
    },
    ...
    }
    },
    "settings": {
    "5e70c4e7-389a-4fc9-8a9a-732865032b55": {
    "data": "...",
    ...
    }
    },
    ...
    }

    This is true for images without versions. With each version there one additional entry in section 'photos' as well as in section 'settings' and the image name would also change to 'IMG_2012_07_24_4987v1.CR2' for the version entry.

    Now the buggy sidecar file.

    {
    "photos": {
    "0ff2c369-606a-4965-a333-9a4a1d421002": {
    ...
    "metadata": {
    ...
    "CameraModel": "Canon EOS 5D Mark II",
    ...
    "CaptureDate": "Tue Jul 24 11:29:37 2012",
    ...
    },
    "name": "IMG_2012_07_24_4987.CR2",
    "settings": {
    "com.ononesoftware.vcb.processplugin4": "7232ae88-da52-4627-b4fc-b25681244742"
    },
    ...
    },
    "16c37e7a-3c9e-43a5-b4f7-fab2a6b6e101": {
    ...
    "metadata": {
    ...
    "CameraModel": "Canon EOS 5D Mark II",
    ...
    "CaptureDate": "Tue Jul 24 11:29:37 2012",
    ...
    },
    "name": "IMG_2012_07_24_4987.CR2",
    "settings": {
    "com.ononesoftware.vcb.processplugin4": "dbaadd32-8411-4430-b14e-13b8afc8df49"
    },
    ...
    }
    },
    "settings": {
    "7232ae88-da52-4627-b4fc-b25681244742": {
    "data": "...",
    ...
    },
    "dbaadd32-8411-4430-b14e-13b8afc8df49": {
    "data": "...",
    ...
    }
    },
    ...
    }

    You can see two entries for the same RAW file in both sections 'photos' and 'settings'. And because there is no version for this image, this is definitely wrong and PhotoRaw struggles and destroys the sidecar file.

    So I searched for the settings entry with the longest data value (because there are a lot of local adjustments with masks) and removed the second one. Then I looked for the right UUID in the 'photos' section and also removed the other one. This can of course be done by a script.

    That's all. So I restored 21 buggy sidecar files and PhotoRaw shows these images as they originally have been edited.

     

     

     

     

     

     

     

     

     

     

    0
  • Brian Lawson Community moderator

    Thanks for that Holger, I appreciate it. The problem I have with BBEdit is that it does not format the output so it is harder to interpret. It's weird that your text editor is drawing the text like that.

    Did you ever create a version for any of those files? I use them as temporary backups then delete them when they are no longer needed. I wonder if that could lead to the kind of corruption you're describing.

    Have any of them ever had another copy in a different format, i.e., a .CR2 and a .tif with the same file name? I've often wondered about how the program handles multiple files in the same sidecar and what happens when one of those files gets renamed. I'm wondering if there could be a bug in the code that handles all of this that leads to the sidecars becoming corrupt. I haven't taken the time yet to come up with a good test scenario so I can examine the changes to the sidecars. Of course, if I could get it formatted for easier reading it might improve my motivation. 😀

    0
  • Holger Danielsson

    Thanks for that Holger, I appreciate it. The problem I have with BBEdit is that it does not format the output so it is harder to interpret. It's weird that your text editor is drawing the text like that.

    It's only the output of 'jq'.

    Did you ever create a version for any of those files? I use them as temporary backups then delete them when they are no longer needed. I wonder if that could lead to the kind of corruption you're describing.

    I have some files with versions, which I never deleted, and never saw this corruption there. And my script also not.

    Have any of them ever had another copy in a different format, i.e., a .CR2 and a .tif with the same file name? I've often wondered about how the program handles multiple files in the same sidecar and what happens when one of those files gets renamed.

    I never thought of such a situation, but did just a try: no problems at all.

    I edited a file called testimage.CR2 and another called testimage.tif. Both different edits are written to the same sidecar files and I found two entries in section 'photos' and two in section 'settings'. One for the CR2 and one for the tif file. Like it should be.

    0
  • Brian Lawson Community moderator

    So you are running the .on1 file through the jq program then examining its output in your text editor?

    I took a brief look at jq but haven't dug into it yet.

    0
  • Holger Danielsson

    Oh, I forgot the second part of your question:

    I've often wondered about how the program handles multiple files in the same sidecar and what happens when one of those files gets renamed.

    I just tested this also and renamed the tif file testimage.tif to testimage2.tif. PhotoRaw does, what I expected. Both entries for the tif file are removed from testimage.on1 and only the entries for CR2 remain and a new sidecar file testimage2.on1 was created with entries for the tif file.

    So you are running the .on1 file through the jq program then examining its output in your text editor?

    Yes. As OSX has a terminal similar to Linux, the following command should do the work:

    jq . sidecar-file.on1 > sidecar-file.txt

    But really untested, as I never really worked wit a Mac.

     

    0
  • Brian Lawson Community moderator

    Mac supports Linux terminal commands. That will work quite well as would 

    jq . sidecar-file.on1 | bbedit

    Thanks again Holger.

    0

Please sign in to leave a comment.