Continuously Re-cataloging
One problem I have noted since upgrading to 2019.5 is that it seems to keep restarting the cataloging process. The little pie next to my pictures folder starts out blank, then after a long time (I have many 10's of thousands of pictures) it seems to reach 100%. So far, so good. Then later, after restarting the program, it starts this process all over again, from the start (empty pie). I'm not sure what triggers this, it does not seem to happen all the time.
Suggestions?
-
Rick, I did as you asked - I took a raw photo from 2010 that has been sitting in my cataloged file, edited it and saved a JPG of it; so in essence there were 2 new files for it to catalog. I didn't add any metadata as I haven't really used cataloging so I'm not used to adding it.
It did not catalog the folder when I did the edit and saved the JPG but it did recataloged the whole Pictures folder when I reopened RAW about a minute later. I tried looking for the clock icon next to the subfolders but didn't see it and it only took a minute or two to say 100% complete. It was using about 65% of my CPU when it was cataloging then resources jumped down to nothing being used. This is telling me that at least on my computer it only catalogs files when it first starts up and then just sits there. I did a layers file the other day so it was a PSD saved with a JPG and RAW didn't behave badly either when I restarted it.
I know my findings are not what you're seeing.
0 -
Vinny,
Ok, thanks for checking that for me.
I just got back to my PC and had left On1 running for more than an hour. It was cataloging when I left and the catalog for my photos folder now shows 90% but at the moment it's only using 24% CPU. So either it didn't finish till now or it started again before I got back.
As I've been typing this, it's gone to 99% done with the CPU jumping to 55% at times.
0 -
Unbelievable this hasn´t been addressed yet. I remember starting a support case back in December that didn´t go anywhere useful other than "reinstall" and "collect logs".
Is anyone at On1 paying attention to this at all?
0 -
Allan, in all fairness to ON1, I no longer have the continuous re-cataloging that I had under 2019.2.
So they must have fixed some of the issues. I am now using catalogs again and I do not see the issues that I had under 2019.2.
These issues where pretty much what Rick is still seeing and it was extremely bad on my "powerhouse" computer.
Kevin
0 -
I have to agree with Kevin, for the most part RAW is working pretty well on my computer so things have been done. I do have minor glitches but nothing major now. The recataloging that was present for me in 2019.2 is gone and things that were bogging down my system seems to have gone away; yes I see issues when I have a lot of cloning going on but my system is not as fast as newer systems so at this point I accept the problem as being on my side.
With this said I have been on the other end where RAW was giving me issues and others were OK so I know the frustration.
0 -
OK, made some progress on this, but its taken most of my memorial day holiday. I have some insights, but I'm not sure many of you are going to like them.
Most of my files ON1 is having problems with seem to relate to ones that have danced with Lightroom. This includes .psd files with a LR icon in the browser, additional .psd files, some .dng's and .dng files that were created from HDR mergers in LR.
I found this my methodically moving folders and files in and out of the cataloged folders. I noticed that I had a significant number of problems with photos from 2016. This was a time when I was trying to use ON1 and LR together as much as possible. I used LR mostly for HDR and Pano merges, but also for its "smart publish" capabilities. For many of my better pictures, I would save a .psd version so that LR could see it and use the metadata tags to publish images of different and resolutions and file types to various distribution channels. Unfortunately for me, everything lived in the same file system.
I found that the problems frequently did not show up with the "pie" icon in subfolders (partial pie at the upper directory level, but no activity showing in folders below). But, I found that often I could infer cataloging problems because it would take some time to populate images in the grid view in folders. Interestingly enough, some of the problems went away if I moved files within Browse from one folder to another. Seems like voodoo, but perhaps there is some part of the database that has become corrupted by losing track of the files, and this action resets it.
My process so far has been to move .dng and .psd files out of the folders to a temporary location. Then move them back to the original folder and add the folder to cataloging. This seems to remove a lot of problems. Then I look for pie activity. I there is some, I go back to see if any of the files were .dng files from HDR or Panos. If so, I move the to a temporary parking area until I figure out what to do with them.
All very painful. So far I have processed the last four years of photos and three years since 2000 and seem to have a catalog that doesn't regenerate all the time (still a bit of occasional stuff, but I can live with that). Hopefully the folders between 2002 and 2015 will be easier, because there are a lot of jpgs there.
Hopefully this helps someone else, but it looks like I will need to reprocess a lot of HDR photos and I'm not sure I want to do that until I have more confidence in ON1 toolset for this purpose (i have seen a lot of severe chromatic aberration in HDR images compared to LR). So maybe I'm 30% done with this for now (38K images and over 850MB data)
0 -
I've found that removing the .on1 files can clear some problems. I'm not suggesting that anyone should do this, only reporting that the problem seems to be related to something in there. I passed that along to support. Patrick, thanks for the write up. Hopefully you passed it along to support as well along with some relevant files?
0 -
Brian,
Yeah, I'm keeping Stevie in the loop. Out of curiosity, based on what I saw and what you said about removing .on1 files, have you tried just moving those images in browse to a temporary directory and then back to the original location? Perhaps some database corruption? Seems like a stretch as it doesn't seem like cataloging would need to track individual files. I would think that they would just need to check the directory to see which changed since the last time a directory was cataloged. But who knows.
0 -
Interesting, moving one of the problem .psd files to another folder did give it the proper preview. Moving it back to its original location caused it to loose the preview again.
While the program is running the operating system can send notifications of folder/file changes so constant scanning isn't necessary but the program would have to rescan everything at startup to find changes made while it was not running. I don't know how they are doing it.
0 -
Brian,
Did you try moving the suspect file to another folder, then removing the original folder from the catalog, adding the original folder back to the catalog, then moving the file back to the original folder? That's the scenario that seems to work many times for me. I'm thinking that for the remainder of my "migration" I may try to create a new empty "shadow" directory, add the folders to cataloging, and then start moving files into it from my old directory structure.
0 -
I wasn't able to do that for that particular test as I was working from the root folder of the catalog. I'll keep it in mind for the future however, thanks.
0 -
OK, when last we spoke I took a breather. I had moved all problematic photos I could find to a separate non-cataloged folder. When the 2019.5.1 patch came out, I decided I'd try to see if anything had been fixed in this regard. As a simple test, I tried adding the "problematic files folder" to cataloging, and voila, things SEEMED to work. No re-cataloging. So then I tried moving the individual photos back to their original folders....bad move. Back to the same old behavior of triggering re-cataloging. As previously noted, previews with a LR icon in the lower right seemed to be the problem. I found that I could open those files for editing in their proper folder, close the edit session, and the LR logo was removed and the re-cataloging problem ceased. Ten hours later after scanning for .psd files with LR logos and opening and closing edit sessions, the problem seems mostly resolved. Note that I did come across a few .jpg files that seemed to cause the same problem, files which ON1 seemed to think were damaged once I tried to open them. I tossed most of them.
So, very strange behavior, but hopefully mostly resolved. Hopefully my experience provides some clues as to a fix. If anyone from ON1 is reading this, several things I'd like to see addressed:
1) why doesn't ON1 log files that its having issues and skip them the next time, rather than just trying again and again to catalog them. If in fact this is done, why don't they make the user aware of this? A big X on a file would and a list of them would be a great help to fixing a problem
2) It would be nice to be able to search in the browser for files that ON1 thinks are damaged. In my case, I knew most of them were .psd files that had a LR icon in the preview, but those were a small percentage of the .psd files in my catalog. I ended up searching on .psd file type, but then had to visually scan the images to look for the LR files, then edit and close them to resolve the problem. Being able to search for files with LR or that PR thought were damaged would have been helpful
3) Better yet, if all I had to do was edit and close the files, why doesn't PR have a script or built in "fix photo" feature to resolve this.
This was all really annoying and disturbing, and I don't feel that I really got any effective support from ON1 to resolve the problem. If I'm going to be on my own on this stuff, please provide us with some tools or at least a list of what the Error numbers in the ON1 logs mean so we can have some clue as to how to work around these things ourselves until ON1 gets around to fixing them.
Don't get me wrong, I appreciate that this is a really complicated program with a lot of amazing capability. So I appreciate all the hard work. But if the features aren't reliable, I really can't use it.
0 -
In the last three months on advice from ON1 I have deleted my catalogued folders and emptied my cache three times. Each time I restart it works for about a week before it all gets clogged up again and grinds to a halt. Along with day-to-day stuff I run several other image packages alongside ON1 and the cataloguing problem slows everything down to a crawl. ON1 has serious memory management issues that are impacting on other operations on my Mac and if anyone tells me to delete and rebuild my system again I am going to scream.
0 -
If you've already done a clean re-install, once is all you need. The purpose of that is to remove any older conflicting On1 versions that may be causing a problem. This, of course, assumes you've done it correctly.
Catalogues in On1 have been a huge problem in the past, but with the newest version for me, they seem to be working well enough, not perfectly, but if I let them finish initially, then the intermittent updates will let me run other programs without having to close On1. The CPU still gets maxed out occasionally, but not so much that everything stops.
All of this depends on your PC as well. Is it fast enough? Have enough memory? My 10yo PC has 8gb mem and 266mhz CPU and yet handles On1 (the new version) well enough.
I'm not going to go searching to see if you've posted this to another thread. If you haven't gone through the list of suggestions to speed things up (mostly for windows) I can post the list here.
0 -
Rick, I’m using a three-year old Apple iMac Retina with 32Gb of ram. It’s fast. Well, fast that is until I run ON1. When ON1 is running, it completely hogs the cpu and sucks up the memory. Everything slows to a crawl.
It has been cataloguing constantly for the last six or seven days since the last time I overhauled the directories, I’ve left the discs running but the time circles next to the directories never close and the hard drives chug and chug and chug. The cpu fan is constantly starting up and going into overdrive which it never does when I’m not running ON1.
ON1 is wearing out my computer and is wearing out my patience. On paper, it is the bees knees, in practice it is a brick. Some informative feedback from customer service would be nice, not the condescending claptrap they are dishing out at the moment, some information about what they are doing to fix the problem and when they are going to fix it. Their silence is deafening. We are in the second half of 2019 and the program feels like is still in beta.
When it works, which in all honesty it has done occasionally, it is fantastic, but it is failing to maintain any consistency.
0 -
Raymond, you don't say how many photos you're cataloging but you just stated your own answer. You overhauled the directories. On1 needs to rebuild the catalogue, it does take a long time but when it's done you won't have as many problems.
0 -
Six days? And meanwhile all other programs I run are being severely hampered? Not in any sense or in any scenario is this right. I have a fast computer that is supposed to be multi-tasking and it is being hamstrung by this playground bully. I do not accept that this should be the norm. There is something that is very wrong.
0 -
LOL, I didn't say it was right. In fact, I have clearly said a number of times that there are still problems. All I'm saying is to wait until it finishes.
On the Ideas page, since it doesn't seem possible after 1 1/2 years to completely fix this, I added a suggestion to be able to suspend catalogues while working on other things. I can't find it. Either it hasn't been accepted yet or it was deleted as not being reasonable.
0 -
I think your suggestion is perfectly reasonable. If the ON1 gnomes cannot fix their problem a workaround that gives users a chance of getting some work done sounds pretty good to me.
I just shut everything down and restarted, and had a glorious five minutes when it all worked as it should. Then the evil cataloging function kicked in and annihilated my system. If it’s been with them for a year-and-a-half, I guess they’re happy to let it continue as things are, and without any feedback to the contrary from ON1, maybe it’s time to start looking at alternative DAMs.
0 -
No, you don't understand, it's WAY better now with the latest release than it has been, so there has been progress. Still, during the cataloging process it's a real pain, but once it's done, now with the newest version, it won't overwhelm the PC.
You don't say how many photos you have (I have about 50,000) and you don't say how far the cataloging is, but I suspect that something is slowing it down for you. In any case. assuming it IS progressing, don't change anything and just wait. Once it finally finishes, it will only have to catalogue new files which shouldn't take so long.
By the way, if I'm not mistaken the cataloging either stops or slows down when you're outside of Browse. I haven't tested that, but I've noticed that Browse is when the CPU spikes.
0 -
After following all the suggestions in this stream I found a solution that works for me in win10. If I only catalogue the folders from the past two years (all processed exclusively with
on1) then the program is flawless and speed is as advertised. If I catalogue any files from previous years when I used lightroom then I have problems with continuous cataloguing and high cpu use. These files were alll converted to on1 from Lightroom. The problem seems to disappear and reappear randomly sometimes with days in between. My solution is to catalogue only my last two years worth of folders and access everything else using the folder section. With this approach I am really happy with the program and it meets all expectations. Whatever the problem is, it is unfortunate that this might turn some users away from the program which I feel is a step up from most other software options.
0 -
If you bought a car that they claimed would go anywhere then found that it only made left turns would you complain or plan your journey so it did not have any right turns?
ON1 is good, very good, but it is flawed. I like the idea of being able to turn cataloguing on and off and/or, in preferences, a choice of timeframe so it could br run overnight, for instance.
0 -
I upgraded to the 2020 version and this re-cataloging is still an issue. It starts all over as soon as it reach 100%. Making this software useless!
0 -
There is a file in your catalog that the program is having problems with. The trick is being able to find it.
You can start by looking in the onOneLog.txt file to see if it is reporting anything. The next place to look is in the CorruptFiles.log and LastRunCorruptFiles.log.
If the logs are not helpful you'll have to find it yourself. I did this by walking through the cataloged folder, expanding each one to the bottom of my directory tree to find the folders still showing the clock icon. From there I looked at the files in that folder using the Finder. (I'm a Mac user, Windows users should use Windows Explorer of course.) There could be corrupt files, files not in RGB space (which will show in Photo RAW's browser without any preview), or file types that on1 does not recognize that cause it problems. You may have to pull all the files out of the folder and replace them one at a time until the cataloging process hangs again. That will be the problem file.
I have my entire photos library cataloged. I had problems with it at first but once I found the problematic files those problems went away. In the case of unrecognized files I reported them to support and the next release stopped hanging on them. I just recreated my catalog the other day and it finished in less than 24 hours.
0 -
Torje, Regarding "making the software useless" Please remember you do not have to use cataloging. Catalog simply allows a search of all cataloged folders at once and that is pretty much it. If you don't want/need to search all folders at once don't catalog. If you do need it follow Brian's latest suggestion about finding problem files.
1 -
When I upgraded to 2020.1 today, the program decided to completely catalogue everything for no reason that I know of. I've been trying all afternoon to figure out why On1 was using so much CPU when it DIDN'T show cataloging in progress. I eventually figured out that this new version doesn't show the 'clocks' at all so you can't tell at a glance what's happening anymore. Now you have to deliberately hover over the catalog name to see if it's cataloging.
0 -
I think I started this thread 10 months ago, and I'll have to say that I wish ON1 one would provide some additional (non geeky) flagging of these errors (or better yet, move the offending files to a "safe house), but if you consult the logs for the most part you will find the errors that prevent cataloging. I did an extensive scrub after my initial problem (mostly from logs, but also looking for folders that did not catalog properly) and I have upgraded through multiple ON1 upgrades since then without problems. Its a pain going through it, but you won't regret the time spent once you do. ON1 has given good support suggestions in this regard, but given the complaints from the community I'm surprised they haven't come up with a more comprehensive and automated solution to the problem.
0 -
Rick, saw your comment after I posted and checked it out. I am 100% cataloged, but saw that the cataloging progress only shows at the top level of the catalog (if you hover over it) but not at the folder level. That's pretty bad form on ON1's part IMO. Checking individual folders for problems was key to flushing out all my bad files and fixing my problems.
0 -
I've been running 2020.1 for about a week now and I've seen the clocks. Not nearly as much as in the past, maybe because it isn't doing as much work in maintenance mode, but they were definitely there during the cataloging process. Perhaps there's a Mac/Windows difference?
0 -
Could be. I've noticed in the videos it shows the Quick start having 3 dots on the upper left for Mac to close it, but Windows doesn't have any way to close the window once it's opened from the help menu. Although it does if you open it by clicking the +
0
Please sign in to leave a comment.
Comments
60 comments