side cars added to non catalogue images
Answered
I maintain a library of images for my camera club. This means that I have regular additions to add from flash drives, which I do using Windows Explorer. However since installing ON1 2023 I find that the images I have downloaded in this way suddenly have ON1 side car files and even if I delete the sidecars, they later reappear.
I don't know what information these sidecars contain and wonder whether it is somehow related to the catalogue function of ON1. I have to use the images to create some reports (eg, listmyfolder command) and had to delete the ON1 side car files to do so. However, they reappear later as already mentioned.
What is causing this and can I stop this function with a change to settings of some kind?
-
The unwanted sidecars would have been created after installing 2023, so you might be able to find them by date (after Oct 10). You'd just need to leave out the ones you actually edited with 2023.
1 -
I have the same. Even though I've never edited any of these, they all have sidecars.
You can turn sidecars off in settings, but I wouldn't advise it, those are your only backup to the database.
If you're using original photos in your project, you could just copy them somewhere outside of On1's catalog structure and delete them when you're done.
0 -
Thanks Rick.
I must have added the top folder in this section of my image files when I did some editing some time ago of a few images not normally part of my workflow. I have removed this folder from my catalogue so will see whether this resolves the issue.
I had not noticed side car files in the sub folders before, but perhaps the few that were affected went unnoticed. Interesting that this function has nowe been activated though to all new files added to a sub folder. xmp info? A feature of the AI keyword function is my guess.
Cheers, Vic.
0 -
Victor Boase - It seems like the AI Keywords are indeed stored in the sidecar file. So I would guess, removing the image from a catalogued folder and then deleting the sidecar file should leave the image without a sidecar file.
0 -
It is not the AI Keywords. They do not get added to the image's metadata unless you click on one of them.
I exported a jpeg from an image I worked on last night. I have the option for exporting keywords turned off. At first there was no .on1 file with the new jpeg. I left my Exports folder open in the browser and about 5 minutes later the sidecar showed up in the Finder. I opened that in a text editor and there are no keywords in it. It contains just some basic EXIF info.
{"photos":{"3f0f07d4-7509-42c3-b17f-ad927ae3c0b2":{"blurhash":"of8;+cJYtTV?bxr;O_tTWXj=j[WVXVoMacjXaxaxbdjDacf5fRaybJjDe-flbIbHfmf4fkf,flj[","face_scan_timestamp":"2022-10-17T13:47:56Z","guid_locked":false,"metadata":{"Artists":[""],"CameraOrientation":8,"CameraScaleFactor":1,"CaptureDate":"Mon Oct 17 00:29:31 2022","CaptureDateOffset":-21600,"ColorProfile":"sRGB","CopyrightStatus":"http://ns.useplus.org/ldf/vocab/CS-PRO","Depth":16,"Dimensions":{"Height":2048,"Width":1366},"FocalLengthIn35mm":53,"Label":"","MetadataDate":"Mon Oct 17 13:45:26 2022","MetadataTemplateChecksum":0,"Orientation":1,"Rating":0,"Resolution":{"h":96,"v":96},"RightsMarked":true,"UserFlag":0,"WhiteBalance":0},"ml_classes":{"classifications":[{"appMajorVersion":0,"build":12912,"entries":[{"id":"histo20","v":true,"value":"Low-key"},{"id":"histo3","v":true,"value":"Vivid"}],"mversion":0,"timestamp":"2022-10-17T13:47:54Z","type":"histo_data1"},{"appMajorVersion":0,"build":12912,"entries":[{"id":"/m/01sdr","p":0.9822151064872742,"v":true},{"id":"/m/0jg24","p":0.9664974212646484,"v":true}],"mversion":0,"timestamp":"2022-10-17T13:47:56Z","type":"PartialLabelingCSL"}]},"name":"The Wynn Hotel Fb.jpg","type":2}},"type":1,"version":2023}
0 -
Auto-AI-keywording is not working for me. But if open a folder of images in Browse, hit Ctrl-A to select all, and click on the AI Keywords "Scan" button. PR proceeds to do AI Keywording for all images in the folder. After it finishes, all images in the folder have a new or updated . ON1 file - even those that had not been edited.
Perhaps even worse, If I do this on a folder of images that have already been AI-Keyworded, EVERY one of the .ON1 files gets updated! Even if I have not done any changes to any of the images.
I think using .ON1 files to store AI Keywords is terribly inefficient. It litters my disk with a lot of otherwise totally useless .ON1 files (and makes my backup program do much more than otherwise necessary).
I think it would be better to store AI Keywords in the database. Storage would be much more efficient, it would not cause creation of .ON1 files for unedited photos, and if you move images to another computer, AI Keywords could just update automatically on that new computer.
0 -
I wish to modify my previous statement after further thought about it.
The AI generated keywords are NOT being stored in the sidecar file as I showed above. The AI found 4 keywords for my test photo — Color, Image, Low-key, Vivid. As you can see from the metadata I posted none of those words appear in the file.
However, the program has to associate the keywords it thinks are appropriate with the image somehow and that is with the UUID, the long string of letters and numbers you see as the 2nd entry in the metadata. "3f0f07d4-7509-42c3-b17f-ad927ae3c0b2" in this case. That is most likely when the sidecar gets generated, when the internal database is updated with that info so AI Keyword searches can find the image again.
I quit Photo RAW and deleted the sidecar file then relaunched the program with it browsing that image's folder. It's been over 10 minutes now and the sidecar has not reappeared.
0 -
Chris, the AI Keywording is done as a very low priority background process that is stopped when you are actively using the program. It does take a while for the AI to do its job and update all your images and it cannot do that while you are using the program. Just like when building Catalogs, I recommend leaving the program running over night or while you are away from home so the AI process can be given CPU time to do its job.
0 -
Brian Lawson Thanks Brian. I have left my computer on and idle overnight, with PR running, for days and PR has not done any automatic AI Keywording. Overnight, my CPU utilization is typically less than 5%. I bugged this with ON1 and they say they are investigating as others have reported the same.
Interestingly, every now and then PR will all of a sudden use a burst of CPU activity for a few seconds, sending the overall CPU utilization to over 80%. But I search for new AI Keywords and can't find any. I would guess that, over the last week, I have had my computer idle for well over 70 hours and I see no evidence that any AI Keywordkign has happened.
As for AI generated keywords not being stored in the sidecar files - that blew me away!
- I tried with an unedited image
- I selected all images in that folder and forced AI Keywording by clicking Scan.
- I had a new .ON1 file for the unedited image
- I verified that PR listed some AI Keywords for the unedited image
- Closed PR and deleted the .ON1 file
- Opened PR and I see the AI Keywords still there! I was not expecting that!!
So why create a .ON1 file for an unedited image when all I did was click Scan in "AI Keywords"? It makes no sense to me!
0 -
This is just a guess but I suspect that whenever a database entry is created for an image file, a new sidecar file is created as well.
0 -
I recognized that with PR 2023 when exporting jpg files from original NEFs, always .on1 sidecars were created. I asked support if this could be switched off. Dan stated that the creation of these sidecar files is caused by using the AI-keywords automatically.
Since AI-keywords don't make sense for me anyhow, I disabled this in Preferences. Since then, these unwanted .on1 sidecars are not created anymore.
0 -
Volker - keep in mind If you disable sidecars if something happens to the install of Photo Raw and it's internal database gets corrupted/lost you could lose all of your edits. The sidecars are there to help insure you don't lose edits.
0 -
David,
I did not disable sidecars in general, and I'm aware about the implication to do that. I use sidecars for all my edits since day 1 of using ON1 PR in 2017. So sidecras are "on".
I only disabled the creation of keywords for my cataloged photos
This stops this creation of .on1 sidecars for photos that have never beed edited and espacially for exported jpgs
0 -
Ahh I understand now,
0 -
Thanks everybody for the tour of AI keywords and sidecar land.
The need for sidecar files I understand (in relation to recording edits) and like Volker I use the same settings as him. Thanks Chris.
The lesson for me is in relation to the folders that you add to the PR catalogue. I have other images obtained from the web, friends and relatives, camera club library, email attachments downloaded, etc that I rarely use in PR. I will be more discerning about which if any of these are added to my PR catalogue and at what level within any folder hierachy. That will avoid unneeded sidecar files (exif) cluttering these "other" folders.
However Volker has alerted me to the fact that exported jpg files that are in a catalogued folder have sidecar files added to them. Horreur! I checked and yes, my exported jpg folders are littered with sidecar files. This is because my exported jpg images go into a sub folder of the folder for my RAW files (chronological structure). This is horrible and utterly superfluous!
Does this need/ deserve a ticket for PR support?
0 -
Just delete the sidecars. They shouldn't come back. In my testing they did not.
0 -
You can submit a ticket if you like but I suspect nothing will be done. This is deeply embedded into the basic design of the program and unlikely to be changed IMO.
0 -
Brian Lawson - I would delete the sidecar cars that were created JUST because AI Keywording did it if I could. The problem is I can't tell just by looking at the sidecar file if it also contains edits.
I don't want to delete sidecar files that include edits.
0 -
If the file was Exported it does not contain any edit data. I posted the contents of one above. If there was any editing data in it it would look like gibberish and not clear ASCII text because that all gets stored as binary data which cannot be interpreted.
They are just a customized JSON file and you can view their contents with any text editor. Not a word processor.
0 -
Yes, but I have 22,714 .ON1 files in my catalogued folders, most of which would be for images that have not been edited. I don't relish the idea of trying to figure out which of them have edits and which were created by PR just because I forced a scan in AI Keywords.
I guess I am glad I have not yet forced a scan on all catalogued folders yet.
0 -
Go to Window Explorer and sort them by date. The jpegs and their .on1 sidecars will be paired making identification simpler.
I'm guessing your exported images are sitting alongside the RAW's from which they were created? That would complicate things. Do you edit your exported jpgs? That also would complicate things but file size would be a good indicator here. If you do not edit them then all .on1 files associated with jpegs can go.
I realize it's a PITA, I'm just trying to help. I export all my jpegs to a single folder for the most part. I'm thinking about moving it outside of my Cataloged folders.
0 -
Unfortunately, that won't work either. If you select all images in a catalogued folder and click the "Scan" button in AI Keywording EVERY image in the folder get either a new (in the case of an unedited image) .ON1 file or PR updates the existing .ON1 file (in the case of an edited image).
So, all images in the folder end up with .ON1 files date stamped when the scan for AI Keywords ran.
0 -
The only option left would ba to Restore from a backup made before using the Scan button
0 -
Brian Lawson - actually, for me, it has nothing to do with exported images. For me it was because I clicked "Scan" in the AI Keywords section.
When I export images (which will also be an issue from what I see above), they go all over the place depending on the export preset I use.
0 -
That "only option would be a restore from a backup made before using the Scan button" is my assumption as well.
Unless ON1 (hint, hint, ON1) would realize the mess they have made and come out with a way of automating the remove of .ON1 files that do not contain edit information. Maybe I will submit that to the Photo RAW Project.
Although that web page seems a little broken now in that I can't upvote any suggestions there. If they prioritize ideas by upvotes, nothing will get implemented amongst the suggestions!
0 -
With so much speculation about how and when is there a way of just bring this discussion to the attention of the PR technical support team? If only to gives users a guideline on how sidecars are created other than those created by catalogued file edits. And what is the point of side cars for unedited jpg images?
0 -
"And what is the point of side cars for unedited jpg images?" I talked about this above. When the AI generated the keywords, they were entered into a database along with a UUID identifier for the images. Otherwise a search for those keywords would not find those files. When the database is updated the .on1 sidecar files are also updated, or, in this case, created.
0 -
Brian - this doesn't appear quite correct to me.
I put a few images in a folder within my Catalogued folders. I then went in Browse to the folder. There was no AI Keywording done. I selected all and hit the Scan button. Sidecar .ON1 files were created for each image. I exited PR and deleted all the .ON1 files.
I went back into PR and looked in Browse at each image in the folder and the AI Keywords were still there.
I did a search, for "Look in" specified "Search Cataloged Folders" and for the search criteria, I specified only "Keyword AI", "Contains" and specified several AI Keywords that had been applied to those images.
The images were found.
So it appears that the .ON1 sidecar files are not required for searching on AI Keywords.
0 -
Apologies Brian. The significance of your words at that stage was lost on me. Once I got my head around the issues I now understand.
I guess from my point of view I only bother searching on RAW images because the jpg derivatives are in sub folders for the RAW images. I have never been much of a keyword generator or user. Your point about how AI keywords are generated through an EXIF function which makes sense but in doing so it creates a lot of clutter around exported jpg files, but I guess they don't use much space on the disk.
My file structute is chronological hierarchy with place or event names included in the folder name that I create durring the import process. PR has an excellent dialogue for this now. This means that searching is informed first from memory of an approximate date (not infallable) and then scanning folder names.
I am inclined to turn AI keywords off now I understand how its done and the implications.
0 -
I did not say the sidecars are needed to search for keywords, AI or otherwise. How would that work for people who choose to not use them? Also, it would be extremely slow and time consuming to search all those files.
It is the UUID that is necessary for searching. It ties the keywords to the images. It gets added to the database where the searching actually happens. Because new data was added to the database for an image, its .on1 file gets updated. Where one does not exist it is created.
0
Please sign in to leave a comment.
Comments
34 comments