What's new
Photoshop Gurus Forum

Welcome to Photoshop Gurus forum. Register a free account today to become a member! It's completely free. Once signed in, you'll enjoy an ad-free experience and be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!

Does generative fill without prompt restrict itself to my image?


GracieAllen

Active Member
Messages
37
Likes
7
Photoshop 2024...
I'm thinking this has been asked, but I don't see it anywhere... I put it in the Photoshop forum, but so far, it's just sitting there...

Now that generative fill is no longer beta, if I use it to fill an area of an image without using a text prompt, does it restrict itself to source pixels in the image? Or does it grab pixels from other images that aren't mine? Or create them from nothing?

Same question for the generative expand - does it restrict itself to the image for source pixels?
 
I use it to fill an area of an image without using a text prompt, does it restrict itself to source pixels in the image?
As an educated guess..............I would say, unless you are asking (via text prompt) GF to add any object that's not in the active source image, it would more than likely only use the native source pixels of the active open image.

However, it is AI driven and it would not surprise me if it borrowed similar pixels from it's library of images or generated similar pixels to complete the task.

Why would this be a concern?
 
The initial image is always a resource for AI. But Generative Fill, Generative Expand, and Remove tool, are not perfect. It often takes several tries to get close to what you want. This is why an AI tool like Generative Fill, offers you several options to choose from. And if you don't like those, you can keep hitting generate until you get close to what you want/need. When you use fill, it's taking its prompt from your image and also drawing from it's own algorithmic image base. Often, you still have to go back to "old school tools" like clone stamp and brushes to finish the image.

AI does not create from nothing. It needs your resource photo to do what it "thinks" you are asking. It will replicate, or get close to replicating, your colors, textures, shapes etc. There are times when the response is perfect - there are times where it seems to pick "answers" out of its ass. That's the best way I can put it since there are times when AI leaves me scratching my head in disbelief.

Keep in mind that these tools are "conveniences" - they help you with quick results. Things that might take you an hour in research and manipulation will take less than a minute. True creativity still comes from the user. There are people here on this forum using CS 5 version and getting incredible results.

BTW, I've used both the beta and full versions. I don't see many differences. But I'm not testing it technically. I'm just looking at results.

So to answer your questions, "..does it restrict itself to source pixels in the image? Or does it grab pixels from other images that aren't mine? Or create them from nothing?"

The honest answer is yes. :)
 
Last edited:
Often, you still have to go back to "old school tools" like clone stamp and brushes to finish the image.
Exactly!

Note for all and not just Jeff: Not "old school tools" LOL! Most of the time the skill of the Ps tools user beats anything in our AI driven "one click world". Learn the trusted tools of Ps and you shouldn't have any problems. If you're relying on a one click fix, you will always have notably questionable results.
 
A couple months ago I asked this question in here. At that time it was just to get info. Now it’s a fer-real question…………

I belong to a photo club. The photo club has monthly salon competitions. Up ‘til the new release of Lightroom and Photoshop, the rules were very simple – essentially “do your own work” and “don’t add anything to an image that you don’t own”…

AI changed things. So, I created a proposal to add to the existing rules. Here’s the relevant piece

Images generated wholly, or in part, by user-initiated prompt Generative AI are prohibited, however,
all available tools, including AI tools (i.e., generative fill), may be used to optimize one’s own captures
provided no perceptible elements not captured by the photographer are introduced.


My BASIS, in the almost complete absence of information on how the tools work in context of a photograph was this:
zz gen fill tip.jpg

When you run generative fill without a prompt, it displays tips. This is one of them. Seems to indicate that without a prompt generative fill WILL, if possible, use the surroundings – which I take to mean the current image – to fill.

Needless to say (actually I was a bit surprised) the proposal caused an UPHEAVAL. Mostly by one person who purports to have a heavy understanding of AI. His claim is that gen fill ALWAYS goes out and comes back with “stuff” from what Firefly has been trained on. And no matter HOW MUCH it matches what’s in your image, it actually has filled with stuff from this magical Adobe Library in Firefly. AND, that by the mere fact that you RAN the Generative Fill, YOU NOW OWN anything it returned.

My statement that that the members of the club were ethical enough to stay within the guidelines and NOT to use a visible new element returned by unprompted gen fill was met with derision and the assertion that members encompassed a “wide range” of ethics and therefore COULD NOT be allowed access to technology they would surely misuse.

The person was ADAMANT that Generative fill IN ANY FORM must not be allowed to be used in ANY context for salons. Backed by a small bloc of like-minded people, the proposal was defeated. The proposal, however, is not dead and CAN be sent to the membership for a vote to override the board’s decision.

SO, long trip to the question: Does anyone in here have actual knowledge of whether or not generative fill, used without a prompt WILL or WILL NOT restrict itself if it can to the current image (the surroundings) and not pull all manner of non-allowed elements into your image? Or is it actually out to destroy photography by filling anything it touches with elements from images in some library from which the AI was trained? In which case the tip while running gen fill is “misleading”?
 
This is an ongoing debate in the creative community - how much room should we give AI in our creative work. To address your question more directly, Generative Fill/AI will work within the context of your image. BUT - it may add additional elements, that even though might be within the context of the image, were not present when the original photograph was taken. It may add objects and people that were not there before. And it might even add an out of context element which is why it will give you several choices in each round as you continue to generate changes until you pick one that is most appropriate to your own image. It may seem to create creative decisions when it primarily using algorithms.

Now the photographer/creative can manipulate when Generative Fill has created or even delete elements. Part of the new creative process that has been created. It's a tool that we might use in the same way we may use cloning or content aware fill. And consider the other tools in the PS toolbox and what the software other than AI is capable of. My own opinion is that if you restrict AI, then maybe you should restrict all post-processing and stick only with out-of-camera images.

As for myself, I resisted AI since I felt I was being somewhat unfaithful to the craft. But it's capabilities have greatly expanded what I can edit or even create. It's not perfect and often creates quite laughable results like the six fingered hand it once offered. Again, it's a tool that has to used with some intelligence and ethics.

What about your salon competitions? You may end up with 3 categories - straight-out-of-camera, post-processed, or AI generate/assisted images. Or judge the image by the end result as a creative act. What I think those who oppose the inclusion of AI is because they feel it's "deceptive" - yet, unless you're a portrait photographer, it's all deceptive in a creative way. :)
 
Thanks for the reply...
This is an ongoing debate in the creative community - how much room should we give AI in our creative work. To address your question more directly, Generative Fill/AI will work within the context of your image. BUT - it may add additional elements, that even though might be within the context of the image, were not present when the original photograph was taken. It may add objects and people that were not there before. And it might even add an out of context element which is why it will give you several choices in each round as you continue to generate changes until you pick one that is most appropriate to your own image. It may seem to create creative decisions when it primarily using algorithms.
The problem for me ISN’T that it may add obvious objects like people or other elements. In cases where that happens it’s EASY to look and recognize that it’s clearly added them and they’re not based on the photographer’s captures.

The problem is the assertion having been made last night that generative fill ALWAYS, even when it “might” remain within the context of the image, uses foreign elements that match virtually perfectly with what IS in the image, but are actually taken from some external data source in the AI. And that MUST NOT BE ALLOWED!

Bizarrely, the SAME group that so steadfastly clung to how gen fill without a prompt was in fact USING this other data, have absolutely NO problem with Lightroom denoise creating pixels to improve sharpening or Remove doing the same thing or Lightroom Lens Blur creating flattened pixels (or whatever it may actually be creating, if anything) to provide the blur. Or that ANY other AI-driven tool may be doing a similar thing. The whole argument became rather absurd.

Now the photographer/creative can manipulate when Generative Fill has created or even delete elements. Part of the new creative process that has been created. It's a tool that we might use in the same way we may use cloning or content aware fill. And consider the other tools in the PS toolbox and what the software other than AI is capable of. My own opinion is that if you restrict AI, then maybe you should restrict all post-processing and stick only with out-of-camera images.
I agree, and MY belief was that like MOST of us, if we suddenly have a sunflower pop up in a street scene because the unprompted generative fill adds it while cleaning up some dirt, is immediately going to recognize that it’s NOT something we captured and do the minor work to remove the unwanted addition.
From my perspective, If these contemporaries of mine are so offended by, and fearful of, the capabilities of the generative fill, and so convinced other competitors are of such low moral character as to take advantage of this, why not require every competitor to reinstall the 2018 versions of Lightroom and Photoshop so there’s no danger of an image being contaminated.

As for myself, I resisted AI since I felt I was being somewhat unfaithful to the craft. But it's capabilities have greatly expanded what I can edit or even create. It's not perfect and often creates quite laughable results like the six fingered hand it once offered. Again, it's a tool that has to used with some intelligence and ethics.
I've also found that when the gen fill runs amok the results are laughable and clearly don't belong in the image. Unfortunately, my arguments were to no avail.

What about your salon competitions? You may end up with 3 categories - straight-out-of-camera, post-processed, or AI generate/assisted images. Or judge the image by the end result as a creative act. What I think those who oppose the inclusion of AI is because they feel it's "deceptive" - yet, unless you're a portrait photographer, it's all deceptive in a creative way.
This particular club has never had specific restrictions on particular types of images – Nature images just have to LOOK realistic, so people like me are free to add a wing to a bird as needed. Even travel images can be fully optimized. Other than the occasional “photojournalism” topic where they loudly proclaim that we must "PRESERVE THE SANCTITY OF THE ORIGINAL IMAGE", there’s nothing to stop anyone from making their images as good as possible by any means available.

Until the scourge of generative fill came to town.

What of the image I displayed earlier – if ADOBE says the tool is using the “surroundings”, what DOES that mean other than that it’s working with what’s in the image?
 
Last edited:
Does anyone have, or can someone point me at, a list of those tools in Lightroom and Photoshop that use the AI. Specifically, those that have the potential to create new pixels, either in the tool or by taking them from the images in the AI. For example, I THINK that though Select Subject uses the AI to help determine what the subject is, it WOULDN'T add any pixels. But, Denoise in Lightroom WOULD seem likely to create pixels for sharpening. Or content-aware fill generate pixels (or possibly elements) to do a good job filling.
 
@GracieAllen If you use phrases like 'scourge of generative filll', even cynically, you're already defeated. The only compromise that I think might be made is that any photo submitted must originate out-of-camera. You just can't deny the existence and impact of generative AI much longer.
 
Last edited:
Good grief. What does that have to do with getting a list of tools? Would a new title make you feel better? And yes, the use WAS cynical because so many organizations are hysterically throwing out all manner of restrictions to keep user from actually USING the tools... Same question as before, Can somebody who understands better than I what's going on inside Photoshop and Lightroom provide a simple list of the tools the have the potential to create pixels in an image.
 

Back
Top