Export a review folder of visually similar images with FiftyOne

Two photos can look almost identical while recording different information: a crack seen from another angle, or a changed label on the same box. FiftyOne’s near-duplicate search compares image embeddings, numerical representations of visual features. Its results are candidates for inspection, not proof that either file is redundant.[5]

This documentation-based workflow was not executed. It assumes terminal familiarity, a Python 3.10–3.13 virtual environment already activated, and a small folder of images for a first trial. Install FiftyOne in that environment with python -m pip install fiftyone.[1]

For the model used here, also run python -m pip install torch torchvision. MobileNetV2’s model card lists those packages, supports CPU processing and provides image embeddings. The model identifier is mobilenet-v2-imagenet-torch.[2]

Create the persistent image collection

Replace the quoted paths below with absolute paths on your computer. Use an unused dataset name instead of near-review if that name already exists. ImageDirectory imports image files from a directory, including subdirectories by default.[3]

fiftyone datasets create --name near-review --dataset-dir "/absolute/path/to/images" --type fiftyone.types.ImageDirectory[3]

Install the official Brain plugin with this command:[4]

fiftyone plugins download https://github.com/voxel51/fiftyone-plugins --plugin-names @voxel51/brain[4]

Launch the collection with fiftyone app launch near-review. Keep that terminal open while using the App.[7]

Find candidates, then inspect them

Click Browse operations and choose find_near_duplicates. In its configuration form, choose mobilenet-v2-imagenet-torch for the embeddings and inspect the distance threshold before running. The operator creates saved views named near duplicates and representatives of near duplicates; the latter contains one representative per group.[4]

The threshold depends on both the collection and the model. A wider threshold admits more weak matches; a tighter one can miss repeats. Start with the form’s initial setting and use the inspected results to decide whether to rerun it.[5]

Open the saved-view menu at the upper left to load the candidate and representative views. Click images for a larger view and compare the details that matter for your collection. Keep both members available when a second angle or changed text could be useful.[6]

Select the images you want to examine further with their checkboxes. Click the tag icon above the grid, add the sample tag check-duplicate and apply it. A tag action affects the current view unless samples are selected, in which case it affects that selection.[6]

Filter by check-duplicate in the sidebar and note the visible sample count. This is the batch you will export. Tagging a candidate is a request for review; it does not decide which photograph to keep.[6]

Export copies of the review set

In a second terminal with the same environment activated, run this export command. Choose a new output folder so its contents can be checked as one batch:[7]

fiftyone datasets export near-review --filters tags=check-duplicate --export-dir "/absolute/path/to/duplicate-review" --type fiftyone.types.ImageDirectory --kwargs export_media=True[7][8]

ImageDirectory produces a folder of image files. The explicit copy option leaves the originals in place. Compare the exported file count with the tagged sample count, then open several copies and check them against the App’s selection.[8]

Keep the persistent dataset and saved views for the next review session so you can revisit the selected images.[6][3]

Report an error or suggest a correction