Inpaint

Sometimes, you have a perfect picture, except for one part of it that came out wrong. For those situations, Inpaint is the perfect tool. It allows you to select just specific portions of an image, and then replace only those portions with NovelAI Image Generation.
You can access this handy tool above any generated image, or straight from the Image2Image UI

Where to Inpaint

Goose Tip: You don't need to generate an AI image to use as a base for Inpaint.                                      You can use it on any uploaded image for the Image2Image tool.

Masking

When you access this tool, the first thing you'll be greeted with is the Inpaint Canvas.
It is similar to the usual Image2Image Canvas, but with a completely different usage.
Instead of adding strokes of color to the image, you'll be drawing one or more blue-tinted selection areas over the image, called a Mask, that will determine what portions of the image you want redone.
Anything that is not marked by the Mask, will remain the same.

Where is the Mask

Goose Tip: Careful if drawing the Mask's edge too close to what you don't want changed, as little pixels might sometimes "still be seen" by the AI.                                                                                                              If you feel like stuff from outside the Masked area is "leaking" into it, try expanding the Mask.

Generation

After you're done Masking your image, you can then proceed with the usual Image Generation process of prompting and picking options.
You can prompt for what's already in the image if you just want to reimagine it, or prompt for completely new things if you want to add details.
When you press the Generate Image button, you will get image generations that only apply the prompt to the Masked area.

Where is the Mask

Goose Tip: Even if there are multiple imperfect parts in one image, it's easier to fix them one by one.
• By Masking each part at a time, not only there's only one thing that can go wrong per generation attempt, you can also prompt for what you want of each part more precisely.
• Alternatively, if you're generating with more than one Masked area at once, you can always restart the Inpaint process with a newly generated image that fixed at least one of the issues, potentially saving generation attempts.

Inpainting Strength

It's possible to change the Strength of the Image2Image applied to the masked part of your image. This can be useful in two cases:

If you like what's under the mask (like the overall hand pose), but it's not quite right, so you want it slightly different.
Or, if you scribble something over the image and want to have the model finish it. This can be done by setting the imported image as the base image, then using the Img2Img editor to scribble over it, and then clicking on Inpaint to draw a mask on top of the rough scribble. In the example below, you can see how at 0.55 Strength the design of the sword sticks really close to the rough drawing done over the image, while at 1 Strength the design is entirely dictated by the changes done in text to the prompt.

Note: The Inpainting Strength slider is only available on V4 or higher models.

Strength Slider Example

Focused Inpainting

The Focused Inpainting button next to the brush and eraser tools allows you to select a region of your image with a rectangular selection. Then you can either paint your inpainting mask as usual inside this selection box or leave it empty, in which case the whole content of the box will be inpainted.
The key difference with regular inpating is that, before inpainting, this section gets upscaled to approximately one megapixel (the size of the selection is limited to a smaller size). This allows the inpainting process to add more details at a higher resolution.

Additionally, if you are an Opus subscriber, Focused Inpainting also lets you inpaint regions on large images at zero Anlas cost.

Focused Inpainting

The Minimum Context Area slider, positioned where the brush size slider is for the regular inpainting brush, allows you to adjust the context around the selected area, which will not get inpainted and allows the model to see the surroundings of your selected region. This context region is marked with the translucent, red border inside the selection box.

Focused Inpainting may sound a little complicated, but in practice, it is quite easy to use. If you would like to improve the face details of a character in your image, you just press the selection tool button and put the selection rectangle around the character’s face. Generate, and the level of detail should improve.

Advanced: Outpainting

With a little ingenuity, it's possible to use the Inpaint to perform an Outpaint generation. By uploading a picture with an empty space added to where you want it to be expanded, and masking that empty space, you can essentially perform Inpainting outside the base image.

Outpainting Example An example of adding empty space to the source image, and then prompting for high-heels.

Goose Tip: You can use NovelAI's Canvas' Resize Canvas feature to create empty spaces in images that are already selected for Image2Image. Don't forget the "Crop to closest valid size" button!

Advanced: Reference Inpainting

A more advanced technique is to take advantage of the way Inpainting works to give the AI a visual reference to use when generating. This can be a useful tool to recreate characters, items of clothing, or simply something the AI doesn't know.

First, you have to make a blank white canvas in the size of the image you want to generate (e.g. 832x1216) in an external image editing program like Paint, or Photoshop.
Then, find one or more images to use as reference, these can be anything from a character's face, a full body shot or even an article of clothing. For good measure, it's recommended to add a black border around each of your reference images. Put these reference images in a corner or one side of the image, they don't need to be huge.

Once your reference canvas is ready, import the image into NovelAI and select Image2Image. Then, press the Inpaint Image button and draw a blue mask over all the white empty parts of your canvas that aren't the reference images. Once that's done, write your prompt. Ideally your prompt will contain tags for both the reference and what you want the AI to generate in as much detail as possible, but most importantly, make sure you include the reference inset tag somewhere in your base prompt.

Reference Inpainting steps

With the mask done, make sure reference strength is set to 1 (unless you have a good reason not to) and generate. If you notice the AI is struggling to recreate a specific thing in your references, try tweaking your prompt to nudge it in the right direction.

Some other miscellaneous tags you may be interested in when doing Reference Inpainting include:

  • alternate headwear can be helpful if you're trying to prompt for a specific headwear but one of your reference images already shows a character wearing a different headwear than the one in your prompt (this also applies to all the other alternate tags targeting different things).
  • multiple views will generate your character in multiple different angles.

Reference Inpainting example