Mask a turning object before RealityScan aligns its photos

Photograph an ornament on a turntable and its background remains in every picture. RealityScan’s AI Masking uses a trained segmentation model to separate the object from that background. Introduced with the desktop app’s 2.0 release, it gives the subsequent reconstruction a more selective set of image areas to work with.[1]

The useful checkpoint in this guide is a saved project containing inspected masks and aligned cameras. It follows the current documentation; no photo set has been processed for this article. The finished surface and its texture come later.

Bring a suitable photo set into the desktop app

Use desktop RealityScan on a compatible 64-bit Windows computer with an NVIDIA GPU; check the current requirements against your hardware. Install it through the RealityScan tab in the Epic Games Launcher’s Unreal Engine section, using an Epic account. The download page lists free use for students, educators and individuals or companies below US$1 million in annual gross revenue, with separate paid licensing options.[2]

Start with clear, overlapping views of one object. RealityScan’s capture guidance asks for every surface point to appear clearly in at least two photographs. Preserve the original photographs in a dedicated folder. A mask can choose which visible pixels contribute; an unseen surface still needs another view.[3]

In a new project, use WORKFLOW > Images to add photographs, or Folder to import the set. For an arrangement that makes checking easier, choose WORKFLOW > Layout > 1+1+1. This places the image list, a 2D image and the help view alongside one another. Clicking a photograph in the list changes the displayed image.[4][5]

Generate the mask before alignment

Open ALIGNMENT > Process > Mask Images and choose AI Masking. This is the option that can operate before cameras have been aligned. The generated mask layers attach to the project automatically. Masking from Mesh is a different option for a model that already exists.[6]

Display a photograph in the 2D view. Under IMAGE 2D/VIEW > Source, set Input Layer to Mask. White areas contribute to processing; black areas are excluded. Tab cycles through available image layers. If Mask is unavailable, that input has no mask layer to display.[6]

Inspect several angles before processing the whole set further. Look at narrow handles, the base touching the turntable and any opening through the object. A retained strip of tabletop and a missing handle are different errors: one admits unwanted material, while the other removes part of the subject you meant to reconstruct.

For a mask that needs manual repair, RealityScan accepts external mask images. Its documented naming pattern keeps the full photograph name and adds the mask suffix: ornament-01.jpg pairs with ornament-01.jpg.mask.png. Import the matching photograph and mask together through WORKFLOW > Images. Keep their pixel geometry aligned; a differently cropped mask no longer describes the same image positions.[10]

Tell alignment to use the mask

Select the relevant inputs in the image list and inspect Selected Input(s) > Image layers. Enable masks for alignment must be enabled for this stage. Enable masks for meshing and Enable masks for texturing and coloring are separate controls: decide those when you reach those later operations. The presence of a mask layer and its use in a particular process are separate things to check.[7]

Choose ALIGNMENT > Registration > Align Images, or press F6. Alignment detects and matches common image features, estimates camera positions and orientations, and produces a sparse point cloud. This is the stage at which you find out whether the retained views connect.[8]

Inspect the alignment, then save both parts

Switch a view to 3Ds using its upper-right view-type control. In SCENE 3D/VIEW > Source, choose the component to inspect. A component is a group of images that aligned together; the 1Ds view shows how many inputs it contains. Check the point cloud and camera arrangement, including whether an important side of the object has been left out.[9]

Several components can reflect blurred photographs, weak surface texture, insufficient overlap or large viewpoint changes. Inspect the missing views and add better overlapping images where needed before aligning again. Generating another mask cannot create correspondence that the photographs never captured.[9]

Return to the Mask Images dropdown to export the masks. Use Ctrl+S to save the project, then keep it beside the original photo set and exported masks. The project save command also appears under the application logo at the upper left.[6][9]

Reopen the saved project and inspect a mask from one awkward angle again. Confirm that the intended component and its cameras are present. You now have a defined starting point for meshing: the object areas you chose to retain, the photographs behind them and an alignment you have actually examined.

Report an error or suggest a correction