Skip to main content

Sky replacement: needs improving

Comments

22 comments

  • Ray Miles

    The masking is completely haywire and doesn't seem to often recognise what is sky in the original image. Note the arc between the side of the building and the steps:

     

    Is there something wrong with my system or are others experiencing the same issues?

    0
  • Rick Sammartino Community moderator

    From my experience, Sky AI will get you into the ball park, but I always have to clean it up by hand.

    0
  • Ray Miles

    Stevie in Support messaged me today to say, "We are aware of some current limitations with Sky swap and will be working to improve results and functionality in update releases. I don't have any specific ETA however."

    Quite frankly this should have been sorted prior to release. Where were the beta testers?

    0
  • David Tillett
    Great answers

    Not sure why you are blaming beta testers? I reported issues with recognising skies with buildings etc sticking up right at the start, commenting that people would complain. I sent several test images as examples.

    As it is the creation of the mask is failing rather than the functionality of the sky swapping I would hope that improvements will come from further training of the AI model rather than the need to fix program bugs.

    0
  • Ray Miles

    I am not blaming the beta testers, but I AM blaming On! It is so obvious that there is a problem with sky swap when there is a building jutting up into the sky that this should have been dealt with prior to the release of On1 PR 2022.

    Anthony Morganti released a video today highlighting this issue, although other "influencers" like Jim Nix have stayed silent on the issue. I guess their income might be at risk if they criticise On1.

    0
  • Brian Lawson Community moderator

    You were blaming the beta testers when you said, "Where were the beta testers?" I was also one of those beta testers and also reported my findings during the testing phase.

    It takes time to train an AI. Getting the sky replacement AI to handle more than just landscape type horizons will require more training for the AI.

    I watched Anthony Morganti's video too. There are far better tools for replacing the sky in his lighthouse photo that would handle that photo without any problems. I would have used a Color Range mask. He also has a bad habit of saying Photo RAW cannot do things that it can.

    0
  • Ray Miles

    Hola Brian. I realise you are a beta tester, and I am also grateful to you for the invaluable help you have given me and others on this forum.

    Was the buildings issue raised during beta testing? If so, why did On1 go ahead and release sky swap anyway? 

    Thanks for the tip about using colour masking. I have tried it on a number of images and it certainly achieves better results than I was getting without it. Hopefully, in the next iteration, this kind of tweaking won't be necessary.

    0
  • Brian Lawson Community moderator

    You're asking us to explain the company's reasoning when we don't have anything to go on. They don't consult with us about when to release the updates. All we can do is report our findings.

    For the photo with all the columns I would use the AI Masking tool.

    For the top photo load your replacement sky image as a new Layer. Move it to the bottom of the Layers stack then select the original photo on the top of the stack. Use the Color Range tool along with the masking tools Levels sliders to fine tune it so only the sky is selected. You may need to do some cleanup with the brush, then Invert the mask so the sky is masked out.

    0
  • Ray Miles

    I took a look at the mosque image again. Colour range on its own didn't work (although it has done for others). For the time being, it's a lot easier/quicker to invoke Luminar as a plug-in! Thanks anyway.

    0
  • Brian Lawson Community moderator

    You should use what ever works best. This is true if one wants to stay entirely in Photo RAW, or work with other software. Just because something is new, or AI, doesn't mean it is always going to be the best choice for the task at hand.

    The recent announcements from the company on what is coming in the update next month looks promising. I can't locate it right now but I'm pretty sure I read or saw something that said more training for the sky replacement AI was being done.

    0
  • Alexander Doroshev

    Brian,

    I used ON1 for a long time, and actually I like it, however, with all my respect to you and the company, the feature does not work as it is specified (and marketed) on the site. I tried trial 2022 (considering to upgrade to it) versus PS and Luminar, and it failed on most of the images that include man made structures (tall buildings, light poles, etc., btw. as it was mentioned by Morganti). In the same time PS and Luminar 4/AI do not have such problems at all. As an IT person I would agree with topic author that this is a mistake of testers equally as developers, and current feature state could be considered as a major issue up to removal it from the specs.
    Please look at the RAW image with applied Weather filter and mask which was created from it to limit effect only to sky. No comments...
    I cannot tell about anyone else but for me it is a deal-breaker (and the most expected feature from 2022) and as result I will stay with the previous version.
    Regards.

    0
  • Brian Lawson Community moderator

    I disagree about this being the testers' fault in any way. All we can do is report our findings. We do not set the schedule for the release of the program or for the order of fixing the bugs that get reported. We certainly don't have the ability to make the fixes ourselves.

    If you're not satisfied, that's fine. Like I said, you should use the tool that you find works best for the task at hand.

    0
  • Ray Miles

    Brian is being discreet and not revealing whether this issue with Sky Swap came up during beta testing. I am sure it must have and am somewhat surprised that On1 was prepared to release it in such an unsatisfactory form: it has not worked for about 70% of the images I have tried since v2022 was released.

    However, Brian did mention that there is un update to v2022 due next month and On1 has been publicising its new features. I sincerely hope they also deal with glitches in older ones!

    0
  • Alexander Doroshev

    Ray,

    I actually understand him, since I can suppose that he has signed NDA and cannot directly confirm (or deny ;)) any knowledge about the issue.

    Speaking about the issue itself, I'm not such optimistic since (due to my personal experience with ML/AI) it looks like an engine problem and much less than just an incomplete AI training database.

    Anyway, I would be happy to be wrong ;)
    Cheers

    0
  • Brian Lawson Community moderator

    But training is how you create the AI engine. The engine, or for less familiar with how these things work, the part the does the actual analysis of your images for what ever tool is being automated, can only work properly for the type of images it was trained with.

    When I took my AI class last year we had one assignment to write a program to recognize different types of balls. If it wasn't fed pictures of round fruit and taught to reject them it didn't know how to properly identify that object as a piece of sporting equipment and it took its best guess based on what it appeared most like.

    It's my opinion that this engine was trained primarily with landscape type horizons without a lot of tall buildings or skyscraper skylines. It takes time to train the engines. You have to feed them thousands of photos to be analyzed then you have to review the analysis so you can tweak the recognition parameters for better discrimination without dialing it in so tightly that it overanalyzes everything leading to more incorrect results. We spent a week just running the training programs for these types of assignments. That is running the analysis 24 hours per day for up to 7 days just to do the training portion of the assignment.

    FWIW 

    0
  • Alexander Doroshev

    Brian,

    I suppose, this forum is not right place for technical discussions. Long story short. Even partially trained engine should give statistically better results (I supervised from start to finish few big projects based on AI and I know what I'm talking about). Can it be fixed, yes, it can; almost everything can be fixed. Only question is in time and effort. 
    PS. If you took attention to my example (one of very many that I've tested). The light pole should be recognized since it has many properties equal to nature objects like trees and tree branches. Also, in this case, it was high level of contrast against the sky. It should work, otherwise it is a bug in the algorithm.

    Kind regards

    0
  • Brian Lawson Community moderator

    I'm not going to argue with you about this. What we have is what we have. All we can do is wait to see what happens with the next release and keep reporting to support when it doesn't work properly.

    0
  • Alexander Doroshev

    Brian,

    Here I agree with you. Lets wait and see what will be in the update ;)

    0
  • Ray Miles

    A quick look at the new Sky Swap and yes there is a big improvement.

    However, what DOES still need improvement, possibly with a new slider is, when there is a hazy horizon. Sky Swap can fade the sky itself, but something is needed so that sky and horizon blend better and it's not so sharp. Maybe others have a work-round. 

    But here is an image I tried. Notice how unnatural the horizon looks, even with the fade edge and shift edge sliders at 100%

    Luminar still does a better job at blending, with its Mask Refinement tools:

     

     

    0
  • James Kohler

    I've been an ON1 user for years. I was really excited about the sky replacement tool. While I can have some good luck I can also have bad luck with some photos. When I have an issue I will edit the photo then go to Luminar to add a sky then back to ON1. A lot of other features in ON1 are way better than Luminar. I know ON1 is a small company and I am sure they will fix all the issues. For me it is trees and buildings. Also, Side note, erase is still better in Luminar.

    0
  • David Tillett
    Great answers

    When I was beta testing the original release of 2022 I found problems with sky masking and reported these to On1, along with some images.

    I am sure that I tried again when 2022.1 came out and found that although things had improved the masks would still have needed a lot of work before they became usable.

    The release notes for the new patch release don't mention any change to sky replacement but having tested my problem images again I have found that the masking has improved considerably and is quite usable.

    I compared some of the On1 attempts with the technology preview of Luminar Neo and at first glance there is very little difference between the masking.

    For one difficult image of a ruined pier with lots of small gaps in structure On1 didn't find the gaps but neither did Luminar Neo.

    Still not perfect but On1 sky masking seems to have come along way in a few months. I rarely want to replace skies but being able to separate sky from foreground to target adjustments is a very useful feature.

    0
  • James Kohler

    Many thanks for the reply and comments. I did just download and instill the latest patch. I did find also that sky replacement is much better with buildings. I took the same Lighthouse image that, before this patch, sky replacement was not good. With this patch it worked great. Not so good with trees. I’m still having issues but Like you I hope they fix that issue as well. I know they will make improvements over time. Being a small company it will take a bit longer than Adobe but they will make it great!

    James K

    0

Please sign in to leave a comment.