Database Confusion
This may be a dumb question, but I am suddenly confused about the differences between the ON1 "database" files versus the sidecar files. My understanding was that by turning on the sidecar generation, the sidecar files will contain ALL information necessary to store all of my edits standalone. I also thought I understood that the internal edit "database" files were somewhat duplicative with the sidecars, and that if anything ever happened to the edit database it could be regenerated from the sidecar files.
When I look at the sidecar files, they are small (10's of kB) in size. However, when I go to Application Support/ON1 Photo RAW 2023/PhotoSettingsCache, I find 50GB of files, consisting entirely of large .tiff files of masks, up to 90 MB each. So I don't understand how the small multi-kB-sized sidecar files can contain all the edit information if it takes a 90MB tiff to represent a single mask. What am I missing? Normally when files are named "cache" in Application Support I would think they represent temporary files that can be deleted, and I would love to do so to free up 50GB, but don't dare until I understand it.
-
What you're missing is catalogs. Also, a few things are backwards.
The internal DATABASE contains your edits. The sidecars are a BACKUP to the Database and the Database can be reconstructed from the Sidecars.
Then there are Catalogs which are different than the database. Catalogs contain previews for your images which is what you're seeing in the cache.
You should review this article for more info...
https://on1help.zendesk.com/hc/en-us/articles/360035750991
0 -
You kind of have things a bit backwards. Because the .on1 sidecar files are optional, they are the backup to the internal edits database. They also allow you to move the photo with its edits to a different computer to continue editing. Yes, they can be used to rebuild the internal database which is why I recommend using them. Some users complain about the "clutter" and the space required for them.
The PhotoSettingsCache is used to hold what is needed to quickly rebuild the full sized preview onscreen. Its size is controlled with a slider at Preferences > System > Cache Size. Not all caches are "temporary" depending up how you define that word. A cache is a place to store things until you need them later. They may go away when the program is quit like Photo RAW's Scratch space, or they may be more long lasting. The PerfectBrowseCache is permanent and will grow to occupy as much space as is needed. There is an Empty button in the Preferences to clear the cache if you want. It only empties the PhotoSettingsCache and not the PBC.
The PBC caches previews for images inside Cataloged folders. The PhotoSettingsCache holds them for non-cataloged folders and they are in fact temporary. When the cache has reached its max size, older entries are removed to make room for newer entries. This is why it can take some time occasionally for the Browser to fill the screen with previews. If there are a lot of images not in the cache, room has to be made for them before they can be rendered. This is also why it retains things for as long as it can, to keep the program's speed up.
The article Rick linked to above explains all this in more detail. It may take a couple of readings for things to sink in. It did for me.
0 -
Oh yeah, about the size of the .on1 files vs the cached previews - the .on1 files are basically text files. You can open them in a text editor to examine their contents. A lot of the data is in binary form and those would be things like the instructions or brush strokes needed to recreate the masks. The tiff files are image files and necessarily larger. The rest of the data is in ASCII and can be read easily. Things like Keywords and other Metadata are stored in this form.
0 -
Hi Rick, I've read that article in the past, and understand that the edit database is primary and the sidecars are optional/secondary. And I understand that creating Catalogs necessitate yet another database.
Catalog previews are not what I am seeing in the PhotoSettingsCache folder. It contains 50GB of grayscale TIFF images. Open any of them, and each is obviously a single mask from one of the images I had edited. It is easily recognizable. Not one single preview in that folder, only masks. I suspect this folder is related to the settings database, not catalogs, as its name implies. A little concerned that it is already 50GB after only 6 months of light editing. Wondering if it is a true cache, and would like to know if it will gow indefinitely and if it could be deleted from time to time to reclaim space.
When I look in the separate PerfectBrowseCache folder, on the other hand , it does contain full jpg previews as you would expect. Interestingly it is 11GB despite having set the max size to 5GB in preferences. That makes sense, the article you linked indicates that the browse cache also contains both recent browse and catalog previews and that catalog previews are not subject to the limit set in preferences and are not affected by the "empty" button. So this is where the catalog previews reside.
It amazes me that a sidecar can be 10-100kB in size and contain all the edit information including masks, while at the same time I see the settings cache contain a 50MB file for just a single mask. That's some serious data compression. I guess I'll just have to believe in the sidecar without understanding its magic.
0 -
Catalog previews are not what I am seeing in the PhotoSettingsCache folder. … They may not hold the actual previews, just what is needed to recreate them more quickly.
A little concerned that it is already 50GB after only 6 months of light editing. As I explained above and is explained in the article, that size is set with the Cache Size slider. It will not grow any larger than that.
When I look in the separate PerfectBrowseCache folder… This one does grow to become as large as needed. It will grow indefinitely as the number of cataloged images grows. We have no control over this.
It amazes me that a sidecar can be 10-100kB in size… I explained this above. What is in the .on1 sidecar file is not the actual mask. It is the data the program needs to recreate the mask. The images you see in the PhotoSettingsCache are full sized images of those masks and will be whatever size is needed based on the pixel dimensions of your image files. Even though they are black and white they still hold 64bits of R, G, & B data for each pixel in the image. You can see how that requires a lot more space than a description (or whatever the actual data is) needed to redraw the mask.
You said you've read the article in the past. I highly recommend reading it again. As I mentioned above, it took me several readings to get my head wrapped completely around all of it. For a while I had a complete misunderstanding of what I had read which lead to my own confusion about this topic. Read it again. Twice! 😁
0 -
Thanks Brian, I think that is what I was missing - that the sidecars describe the instructions/strokes used to create the mask and that is where the huge compression comes versus the actual mask itself.
I'm a little perplexed by the PhotoSettingsCache you described though. I don't see any previews there (maybe because I really only browse cataloged folders), but it is filled with tiff masks. 50GB of tiff masks. And it isn't affected by the preference for cache size (which is labeled "Browse Cache" by the way) that I have set to 5 GB. The empty cache button has no effect on the PhotoSettingsCache folder either.
0 -
Well, there you go. Apparently I still have some misunderstands about all of this too. At this point my suggestion is to ask tech support for a better explanation of exactly what the PhotoSettingsCache is used for and how.
This is the section of that article that I was drawing my explanation from:
The Browse Cache stores small .jpg previews of photos you have previously viewed or cataloged. These make it faster to browse and preview the same photos again. You can adjust the amount of disk space used for previews of non-cataloged folders by adjusting the Cache Size. The default 5000mb (5gb) size is enough to store previews for about 5,000 photos (depending on screen size). When you reach the cap you have set, the oldest previews are removed. .Jpg previews of photos in Cataloged Folders are not removed and are not governed by the Cache Size slider. You can also Move the cache location. This is handy if you have limited space on your boot volume. If you are working on a laptop with a small internal hard drive, setting the cache location to an external disk, such as the one you store your photos on, allows you to catalog more photos. Emptying the Browse cache removes all the previews for stored for frequently browsed photos; Previews from Cataloged Folders are maintained.
0 -
Lol. I keep falling behind your responses while typing myself. I was about to cite that exact same paragraph about the system preference applying to the browse cache, not the PhotoSettingsCache. Just curious, since you've been using ON1 longer than me, did you look to see how big is your PhotoSettingsCache folder?
0 -
My PhotoSettingsCache occupies 25.17GB. I have ~19K images in my Pictures folder which is cataloged in its entirety.
That paragraph states, "You can adjust the amount of disk space used for previews of non-cataloged folders by adjusting the Cache Size." I had thought that the Cache Size slider controlled the size of the PhotoSettingsCache. Apparently not. Now I'm unsure of exactly what it is controlling. My question now is where is that disk space in the file system? How do I find it?
Part of the problem with that article is that it talks about multiple caches without specifying exactly which one it is talking about at any given time. It switches which one it is talking about sometimes in the same paragraph. "You can also Move the cache location." Only the PerfectBrowseCache which the Empty button does not touch.
0 -
My interpretation of the article is that the Browse Cache contains both previews generated by browsing folders and previews generated by cataloging, and the cache size preference and Empty button both apply to the Browse Cache, with the caveat that they will not touch previews loaded generated by Catalogs. In my case also, all of my folders are catalogued. My Browse Cache size is 11GB even with the limit set to 5GB, because the limit doesn't apply to catalog previews. The Empty button does nothing as well, since all of the previews in the cache are from catalog. The 11GB size is quite consistent with that table of estimated preview cache sizes in the article, for the number of files I have cataloged at medium/4K display. So it will just grow as catalogs grow.
Just to be sure I did a text search for PhotoSettingsCache in that article, no hits.
I submitted a ticket asking for more insight into the PhotoSettingsCache, and what if anything can be done to manage its size. I'm fine with its size right now, but it is the biggest ticket item on my internal SSD right now and if it were to grow indefinitely it would pose a problem some day.
0 -
"You can set the location of the image caches in the Preferences dialog. Moving the cache to a fast external drive can free up space on your boot volume. The cache folders cannot be moved to another computer at this time." This is where things get confusing. It uses caches plural when telling us we can move them then uses it singularly in the very next sentence.
I have been thinking there are 2 separate caches - the PBC for cataloged images and another, unspecified one for non-cataloged stuff. That could be wrong. What exactly does the Empty button empty? Where would we see the results of that in the Finder or Windows Explorer.
We really need support to clarify some of this for us. I'm going to send a request that they read through this and help set us straight. If I'm going to be telling others about it I need to understand it better so I'm not mis-informing anyone.
0 -
I just sent the request to support asking them to go through this and either chime in here or respond directly to me so I can pass it along.
0 -
Found a 3 year old thread in the forums where a PC user edited a new file, created a mask, watched a new directory appear in the PhotoSettingsCache folder, and in that folder was a 161MB tiff file that looked like the mask he had just created. Deleted the large mask tiff in the PhotoSettingsCache and did not effect re-editing the photo. Another user chimed in saying that he regularly deletes PhotoSettingsCache as long as the sidecars are present and it appears to make no difference, and may even speed things up for awhile. Again, PC user, but probably still applicable.
Will be interesting to see what support has to say.
0 -
From Support:
The PhotoSettings Cache helps rebuild and store non-destructive settings like brushing/masks. If you have done a lot of masking/brushing, then the cache can get quite large. This should be cleaned out by the program itself periodically. Deleting it will not harm any settings, but it will slow down opening an image which has a large amount of brushing/masking.
Unfortunately this cache can't be moved like the Perfect Browe Cache.
We have an open request to allow the user to move these larger caches to external or other drives, which is being considered by our product team.0 -
I got the same response along with the following, "The Watched Content folder is the cataloged folder cache and Fast Browse is non-cataloged cache."
Both of those folders are inside the PerfectBrowseCache folder. That probably added to my confusion about where to find things. It explains why the 2 caches are treated as a single item in the documentation. Stevie also told me they would taking another look at the KB article.
This has been a productive discussion David. Thanks for bringing it up.
0
Please sign in to leave a comment.
Comments
15 comments