What's new
Photoshop Gurus Forum

Welcome to Photoshop Gurus forum. Register a free account today to become a member! It's completely free. Once signed in, you'll enjoy an ad-free experience and be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!

refine edge problem


Tom Mann

Guru
Messages
7,223
Likes
4,343
The suggestions by Larry, Sam, Paul, etc. to use a brush to refine your mask are right on the mark, especially with fine structures like hair, as well as when the color and/or tonal contrast between the subject and the original background are small. I'm sure you will get the hang of this soon.

To get to the point where you only need to refine a small number of edge pixels with the brush technique, one can make an initial selection using any of several selection tools including the pen tool (a favorite of many), the "Quick selection" tool (which I often start with), and you can then try to refine the initial selection from any of these tools using the "Refine Edge" tool. Sometimes "Refine Edge" works well, sometimes it's not that great, as you have discovered.

However, there is one trick that really helps out both the "Quick Selection" and the "Refine Edge" tools, and that is to make a temporary image that has greatly exaggerated local tonal contrasts, as well as greatly exaggerated color contrasts, and then use the Quick Selection and Refine Edges tools on that temporary image to quickly construct a reasonably good initial selection for your real image. I find this approach is often much faster and gives one a much more accurate starting point for the final tweaks of the selection with a brush than some of the other methods.

One can increase local tonal contrast by using tools such as medium radius Unsharp Masking or third party tools such as Topaz's "Detail", NIK's "Tonal Contrast" in their Color Efx Pro package, or many other methods. One can increase the color contrast (ie, contrasts in both saturation AND in hue) by increasing the vibrance, and by temporarily converting the image to the LAB color space and then pulling in the endpoints on the "a" and "b" channels. These are the approaches I took in the example below, and then only needed to touch up a few fly-away hairs with a brush at the end of the process. To show what parts of my initial selection looked like before any touch up work with a brush (ie, with only Quick Select and Refine Edges), I intentionally did not do any brush work on a few of loops of hair in the (viewer's) upper right hand corner of the subject's head.

One final comment -- it is rare to be able to use the exact same cutout on a black background as well as a white background. Almost always, the mask needs to be adjusted slightly depending on the brightness of the background. Complex, textured backgrounds are the easiest to drop in. Almost anything will work with them, but solid, very dark or very light final backgrounds will test anyone's skill.

I start off the sequence of attached images with the temporary version with the exaggerated local contrast and colors that I used only to quickly (ie, 3 or 4 minutes) construct a reasonably good initial selection. Starting with this approach, I then needed only another couple of minutes of painting to bring the selections to this point that you see. More time with the brush would have resulted in much more realistic edges.

HTH,

Tom M
 

Attachments

  • Cutout_example-_1000_Original.jpg
    Cutout_example-_1000_Original.jpg
    482 KB · Views: 0
  • Cutout_example-_1001_Exaggerated colors_contrast.jpg
    Cutout_example-_1001_Exaggerated colors_contrast.jpg
    971.6 KB · Views: 0
  • Cutout_example-_1002_On solid black bkgnd.jpg
    Cutout_example-_1002_On solid black bkgnd.jpg
    295.1 KB · Views: 0
  • Cutout_example-_1004_On solid white bkgnd.jpg
    Cutout_example-_1004_On solid white bkgnd.jpg
    286.8 KB · Views: 0
  • Cutout_example-_1005_On complex bkgnd.jpg
    Cutout_example-_1005_On complex bkgnd.jpg
    588.5 KB · Views: 0

Junqi Ng

Well-Known Member
Messages
82
Likes
4
The suggestions by Larry, Sam, Paul, etc. to use a brush to refine your mask are right on the mark, especially with fine structures like hair, as well as when the color and/or tonal contrast between the subject and the original background are small. I'm sure you will get the hang of this soon.

To get to the point where you only need to refine a small number of edge pixels with the brush technique, one can make an initial selection using any of several selection tools including the pen tool (a favorite of many), the "Quick selection" tool (which I often start with), and you can then try to refine the initial selection from any of these tools using the "Refine Edge" tool. Sometimes "Refine Edge" works well, sometimes it's not that great, as you have discovered.

However, there is one trick that really helps out both the "Quick Selection" and the "Refine Edge" tools, and that is to make a temporary image that has greatly exaggerated local tonal contrasts, as well as greatly exaggerated color contrasts, and then use the Quick Selection and Refine Edges tools on that temporary image to quickly construct a reasonably good initial selection for your real image. I find this approach is often much faster and gives one a much more accurate starting point for the final tweaks of the selection with a brush than some of the other methods.

One can increase local tonal contrast by using tools such as medium radius Unsharp Masking or third party tools such as Topaz's "Detail", NIK's "Tonal Contrast" in their Color Efx Pro package, or many other methods. One can increase the color contrast (ie, contrasts in both saturation AND in hue) by increasing the vibrance, and by temporarily converting the image to the LAB color space and then pulling in the endpoints on the "a" and "b" channels. These are the approaches I took in the example below, and then only needed to touch up a few fly-away hairs with a brush at the end of the process. To show what parts of my initial selection looked like before any touch up work with a brush (ie, with only Quick Select and Refine Edges), I intentionally did not do any brush work on a few of loops of hair in the (viewer's) upper right hand corner of the subject's head.

One final comment -- it is rare to be able to use the exact same cutout on a black background as well as a white background. Almost always, the mask needs to be adjusted slightly depending on the brightness of the background. Complex, textured backgrounds are the easiest to drop in. Almost anything will work with them, but solid, very dark or very light final backgrounds will test anyone's skill.

I start off the sequence of attached images with the temporary version with the exaggerated local contrast and colors that I used only to quickly (ie, 3 or 4 minutes) construct a reasonably good initial selection. Starting with this approach, I then needed only another couple of minutes of painting to bring the selections to this point that you see. More time with the brush would have resulted in much more realistic edges.

HTH,

Tom M

Hi tom,

are you able to guide me step by step how to do it as i new to photoshop.

kindly appreciate if you can guide me along. because i do not understand how to do the contrast thing. And what do you mean by unsharp masking and the endpoint on A and B. i did the outcome become like this. The last photo of the right shoulder look abit transparency.

testing111.jpg
testing222.jpg
testing333.jpg
testing444.jpg
 
Last edited:

IamSam

Administrator
Staff member
Administrator
Messages
22,721
Likes
13,258
Junqi Ng ----- Right now it would be better if you learned about the masking process. Watch the tutorials I linked you with.

What Tom Mann is proposing is a bit more advanced than I think your ready for at this time. Learn to walk before you run.
 

Junqi Ng

Well-Known Member
Messages
82
Likes
4
Junqi Ng ----- Right now it would be better if you learned about the masking process. Watch the tutorials I linked you with.

What Tom Mann is proposing is a bit more advanced than I think your ready for at this time. Learn to walk before you run.

my outcome like this. is it okay ?

outcome.jpg
 

IamSam

Administrator
Staff member
Administrator
Messages
22,721
Likes
13,258
The problem is that you have not matched the backgrounds light source to the subject.

Take a look at the light source on the subject....
Screen Shot 2015-04-06 at 11.04.42 AM.png

Screen Shot 2015-04-06 at 11.05.03 AM.png
 

IamSam

Administrator
Staff member
Administrator
Messages
22,721
Likes
13,258
This is the light source direction of your background....

Screen Shot 2015-04-06 at 11.10.20 AM.png
 

IamSam

Administrator
Staff member
Administrator
Messages
22,721
Likes
13,258
i know that i just take it as a reference to put it into the image.

but does the person of the image cut correctly from your experience ?
Ok, I understand.

Post your subject on a black background.
 

Tom Mann

Guru
Messages
7,223
Likes
4,343
Hi Junqi - Sam's comments about the lighting direction are right on the mark. One has to match the lighting on the cutout image with the lighting of the new background in terms of direction, intensity, color, hard or soft, etc. This is why he posted image #2 in (his) post #25 in this thread, and is also exactly why I used a photo with backlit leaves as one of my backgrounds. In fact, sometimes it can be helpful to add some extra effects to dominate the existing lighting and color and make any mis-matches unnoticeable. For example, here I added a lens flare effect as well as a warm, almost vintage color cast to the whole image. Sometimes, doing so may not be very appropriate (eg, my warm romantic backlit effect on an Iron Man character, LOL), but if you can, it can save the Photoshoper quite a bit of work.

In any case, notice how any small errors in my cutout are completely covered up by the choice of background and added efx.

Tom M
 

Attachments

  • Cutout_example-_1006_added_lens_flare_warm_vintage_color_cast.jpg
    Cutout_example-_1006_added_lens_flare_warm_vintage_color_cast.jpg
    271.9 KB · Views: 1

Tom Mann

Guru
Messages
7,223
Likes
4,343
Hi Sam, this the image. is it okay ?

View attachment 54596

Hi Junqi - A few thoughts ...

a) Posting screen shots can be very useful, and I don't want to discourage your from doing this. However, since we are discussing edge problems on the length scale of a pixel or two, it's next to impossible to use a capture of your entire Photoshop window to diagnose such problems. If, in addition to the screen shot, you could also post the actual image (eg, as a jpg), it would be very helpful.

b) Whenever one works on an image, they should always inspect it for hard to see faults. In this case, I brightened up your last screen shot (see attached image), and, all of a sudden, all these flaws start appearing in the black background. Obviously, they would not be acceptable in an end product.

c) The edges almost everywhere on your cutout are way too soft and inaccurate. Just look at the (viewer's) right hand side of the man's jacket. This is because you used much too large a brush radius to do the painting, and you are painting everwhere instead of using it to touch-up small areas. Instead, I would suggest that when you are doing this sort of work, you view the image at 300% or more magnification, and use a brush that is at most 1 or 2 pixels.

d) I can't tell for sure, but I have a suspicion that you might be painting on the image itself, not on the mask. Please let us know.

Best regards,

Tom M
 

Attachments

  • OP's-attempt-screen_grab_of_screen_grab-acr0-ps01a_brighter-01.jpg
    OP's-attempt-screen_grab_of_screen_grab-acr0-ps01a_brighter-01.jpg
    211.8 KB · Views: 10

Top