Raw files slow to load
I have been having a problem with Sony raw files taking anything between 5 and 35 seconds to load.They are from a A7Riv, around 120mb per file.
This happens most of the time. If I am, say, scrolling through a folder to do an initial cull of a new set of unedited files there is no pattern, so some may load in 5 seconds and other will take 15-20 seconds. it also happens with files i have developed, the time taken doesn't seem proportionate to the amount of work done on them.
A noticeable thing is that when using Task Manager to view the activity there is often little going on with the GPU but the CPU will be at 100% during the time. I understood that ON1 is designed to make more use of the GPU than this suggests.
My PC is Intel i7, 32Gb fast RAM, C drive: 250gb M.2 SSD, D Drive: (Storage location) 1tb m.2 SSD, E(scratch) drive 250gb M.2 SSD connected to motherboard using a Pcie adaptor, Nvidia GTX 1650, 4gb graphics card.
I have (I think) followed all the guidance about separate drive for Scratch space and PBC, altered display settings in W10 to high performance, same in Nvidia control panel, for which I thank Brian Lawson, Rick Sammartino and numerous correspondents over the last 2 years or so for their tips and advice.
over the last few weeks I have done two clean re instalations of Windows 10 and two further clean installations (and a couple of resets) of ON1PR, as well as clean installations of the Nvidia studio drivers. I don't use catalogues and don't have the left hand side window open when editing. I prefer to use the browser and rarely use presets.
I am hoping that there is someone who has experienced this problem and knows the solution and that maybe there is something obvious or maybe an incompatibility that I have overlooked.
For info: I have raised this with technical support but their advice has amounted to update the graphics card driver.
Thanks
-
Have you read Catalogs and Caches - ON1 Photo RAW 2022 & ON1 PHOTO RAW 2021? It goes into the details on how the program manages memory and storage of the previews and how the different Preferences > System settings can affect performance. I had to read it a couple of times before it all sank in.
FWIW, here are my System settings.
0 -
If I had to guess I'd say your Browse Cache Size is what is slowing things down. Note that this is separate from the PerfectBrowseCache which grows to be as large as needed for your library.
The reason you see the previews for different photos appearing in random order has to do with the program needing to put those previews into the Browse Cache. Not using Catalogs is also slowing things down as previews for Cataloged images are not removed from the Browse Cache, only those not cataloged get purged. Because you are not using Catalogs the Browse Cache is constantly being purged of old previews to make room for the previews for where you are currently browsing. Those previews have to be rendered first.
I have a Catalog for my entire Pictures folder. If I'm understanding that Browse Cache section correctly that means all my photos are always previewed which speeds my browsing. I just went to the Pictures folder and turned on Show Subfolder Contents so my entire 15435 image photo library was displayed at once. With the preview size slider at the bottom of the Browser window set to the midpoint giving me 7 landscape photos per row, paging through them, the previews are loaded nearly instantaneously. When I scroll through them quickly, skipping several pages at once before stopping the scroll, it can take a second or two for the page to load all the previews, nothing approaching your load times.
0 -
Hello Brian, thanks for the prompt reply. i'm not sure I've explained myself properly.
My System settings is very similar to yours, a result of your previous posts.
I don't use catalogs because as the article you kindly referred to, which I have seen before, says in the FAQs it is OK to browse. I only use the Photos folder in Windows to store files until I have processed them, they are then exported, as JPEGs, to another internal drive in a folder structure, (where I would then use a windows app to view them) and duplicated to a NAS, as a local backup, where they are also accessible to a Smart TV in the house. So it is possible that if I haven't been anywhere for a time, eg in the lockdown, the Photos folder would be empty. I don't use ON1 as a Viewer.
You can probably guess from this that I am an amateur and would not describe myself as a 'walk about' photographer. I am retired and travel when possible, which is when I take most of my photos.
When I open ON1 I would immediately click on a file to open it in 'Edit' and it is this that is slow to load then when I subsuquently scroll from file to file in the Edit module that they are taking a long time to load. Always at least five seconds but often much longer for the progess bar to disappear then another 2 seconds for the final 'snap' into focus. There seems to be nothing consistent in the delay, a couple of files could take 5-7 seconds then the next ones could take 20.
It occurred to me that it may be just the file size causing the problem, or maybe a glitch with what my camera is producing. So I have just loaded around 50, 25mb raw files from my phone and opened them in my usual way. they have all loaded within a second each. so the file size seems to be a factor but the delay with the larger files is variable and disporportionate.
I have also created a catalog of the Photos folder, which does then show the thumbnails immediately on start up but hasn't improved the time taken to load the files into Edit.
My attempts to minimise this delay has led me to, currently, not using my second display for the gallery of thumbnails and not having Filmstrip visible, so that the only thing on screen would be one photo in Edit, the right hand modules and the left hand Tool Well.I understand the difficulties in trying to find a solution, not least because of the infinite combination of components that could make up a Windows PC
0 -
I may have misunderstood your original post. I approached my answer only in terms of the Browser and its preview speed and I'll start with that again this time.
If you do not use Catalogs, the Browse Cache must be cleared once it has been filled with previews to the max size setting. Any new image previews, even if the images themselves are no longer on the drive, will require older reviews to be flushed from the cache to make room for the new previews. That takes time. The new images will then have to be rendered so their previews can be displayed. That takes more time.
Yes, you can use the program without Catalogs but IMO it comes at a cost. Catalogs improve the program's performance, that's why they are there. You can run the program on an 8GB system with an integrated GPU but just barely.
No, catalogs will not affect the time it takes to load an image in the Editor. They only affect the Browser. The Editor's load time is going to be very system and image dependent and if the image has been processed in any way that will slow the load time as well as each different layer (with a lower case el) has to be rendered to produce the final result. If you've use NoNoise AI you'll see an even longer load time as the image has to be reanalyzed and the noise removal applied before any of the rendering can be done. I see variations in load time as well but very rarely do I see one that takes 20 seconds. Your GPU will affect this very much. It looks like you've got a decent video card so that's helping.
In your first post you mentioned using Task Manager to monitor processor and GPU usage and commented that the program doesn't seem to be using it much. I forgot to address that before, sorry. It does rely heavily upon the GPU but only in bursts as needed. It isn't in constant high load use like the GPU is. You'll see it spike while you are brushing during masking and as you are making adjustments while editing.
What happens if when you come across a photo that is taking a long time to load into the Editor you restart your computer then load it again? If you see a difference in load time I would suspect possible memory issues like a possible leak. Do you see memory use keep climbing higher and higher as you work with the program? I make it a habit to Quit the program and relaunch it after every couple of photos and especially when I've finished a large edit.
Other than that I don't know what to tell you. While I'm familiar with Windows I haven't used it regularly in several years so I don't have any advice in that arena. Perhaps one of our Windows users has some insight for you.
0 -
I have found that the Uncompressed RAW images from the A7III also load very slowly during decoding. Try shooting a few images in Compressed RAW and see if that makes a difference.
It seems that RAW processing, that is decoding and demosaicing of the Uncompressed RAW images, with ON1 Photo RAW has issues, maybe because they are double the size of the Compressed RAW.
0 -
Doesn't the compressed RAW have to be decompressed before they can be demosaiced?
0 -
Hi Brian it isn’t quite that simple as the Sony compressed RAW file is lossy compression, most Sony 6 and 7 series only use this which can cause artifacts because of the compressed data … similar to JPG images. I suspect that the same decoder is used for both formats which may lead to slower processing of the uncompressed RAW … just speculation.
I am really not sure why ON1 Photo RAW takes longer to decode the uncompressed RAW than compressed RAW but I know from experience that it is noticeably slower with my A7III images.
0 -
If your camera uses a lossy compressed format then no, it does not need to be decompressed. The compression comes from throwing away data to achieve compression. That results in lower quality images which do not need to be decompressed.
If the format is not lossy then yes, it must be decompressed before it can be demosaiced. You gain lost resolution at the cost of longer load times.
https://photographylife.com/compressed-vs-uncompressed-vs-lossless-compressed-raw
0 -
Brian, Thanks again for such a comprehensive reply. I didn't notice the memory going above 50% of the 32 gb available but things are all always better after a reboot, which, because of SSDs isn't as much of a pain as it used to be. Your suggestion of quiting and restarting is something I am happy to adopt, (and also using catalogs) as I am otherwise happy with ON1 and certainly don't have the time (even in retirement!) or inclination to learn how to use something else.
Your suggestion about memory leaks likely means that it would be helpful to restrict any other startup programms and running any other apps whilst using ON1.
Don, thanks for your suggestion. From my experience earlier today trying smaller (25gb) raw files from my phone as an excercise I think you are on to something. I hadn't thought of using compressed files but will try in the next day or two and report back.
0 -
Hi Brian & Don, further comments noted, thanks.
I will read the Photographylife article and also give the compressed files a try to see how they look and how load times are affected.
0 -
Something else I just thought of, and this is for both platforms, is to be sure to turn off any active anti-virus monitoring of the folders holding your photos. All the updates to the .on1 sidecars would be scanned after each update and that will impact your performance. I have active monitoring turned off for my Pictures folder but it still gets scanned every week when I do a full system scan.
0 -
Thanks Brian, I only use Windows Security which I don't think does active scans.
0 -
Hi Guys. FWIW.
I just ran a little test. I was working on a 2 frame panorama with each frame composed of a 3 exposure HDR. It's a 209.9 MB .onphoto file. This is a file I had started working on last year with 2021 and had not opened in 2022 before.
It opened pretty quickly the first time I took it into the Editor where I added some NNAI, Dynamic Contrast, and a Vignette. I moved back and forth between the editor and the browser while I was working on it and it usually reopened pretty quickly. I've found that if I just press G D to bounce out to the Browser and back to the Editor there is no lag in reopening the file. After some amount of time it seems as if some buffer or cache gets cleared and reopening it takes a little longer.
Anyhow, after finishing the edits I quit the program then used a memory manager utility to flush all the OS RAM buffers and caches so only what was currently running was still in memory. It returns all other stuff to the available stack. (The Mac normally holds the memory from a just quit program until that RAM is actually needed again so the program can be reopened faster.) I relaunched PR and opened that same file in the Editor. This time it took almost 1 minute as recorded in the log file. Without making any new edits I did the B R thing again and it loaded right up without delay again.
After writing the above while PR was still running in the background, when I returned to it and opened the photo back into the Editor again it took 25 seconds.
FWIW
0 -
Note that if you have NNAI active on a photo, the no noise process has to be redone EVERY time that photo is opened - and that includes when doing an export to a new format. Those long load times may be due to NNAI.
1 -
Hi Lou, Thanks a lot for this information, I didn't know that this happened and hadn't made the connection with the really long loading times. I have just spent some time experimenting with a 100 or so new files and the ones with NNAI applied were the ones that took a long time to load, 25-30 seconds.
For general info, I used 50% lossy compressed and 50% uncompressed. The compressed files were a little quicker to load, obvious I guess but it was marginal, they are half the size (around 62mb) and took about 3-4 against 5 seconds for the uncompressed.
I have no other experience to tell me if this is reasonable and whether it is similar for all brands of raw files and therefore a question of the file size.
0 -
Those times seem reasonable to me, Mark.
0 -
Great, thanks Brian
0
Please sign in to leave a comment.
Comments
17 comments