Rooftop dataset records how people revise AI proposals for solar mapping

A new rooftop dataset preserves the stage that many AI training collections hide: what the machine proposed before a person corrected it. RoofeNet-Multi contains 5,359 Czech aerial-image tiles, each paired with a Google Gemini proposal and at least one human review. It offers researchers a way to study how roof planes and obstructions are interpreted, and where the original proposal continues to influence the result.[1]

The Scientific Data release, published on 24 October, records 12,761 annotation sets: 5,359 machine proposals and 7,402 human review outputs. Reviewers could retain, move, delete or add rectangular boxes. The people doing the work were supervised university students, rather than a recruited panel of photovoltaic or remote-sensing specialists.[1]

A human-edited label is not automatically an independent label

Every participant began by seeing the image together with an AI-produced proposal. The human reviewer could keep boxes, move them, delete them or add missing ones. That produces a record of how people correct a model's interpretation, which can be valuable for building human-in-the-loop tools. But it also creates an important dependency: if the AI boxes nudged several reviewers toward the same mistaken interpretation, their agreement would not prove that independent observers recognized the same roof feature unaided.[1]

The team is explicit about that limitation. Of the 5,359 image tiles, 1,502 received at least two human review outputs, while 3,857 received only one. Researchers can therefore study variation among reviewers in a defined subset, but they cannot treat every human annotation as multiple independent votes. Nor should they use such a dataset to declare that a model's agreement with its own seeded annotation establishes performance on a blind external test.[1]

The annotations use axis-aligned rectangular boxes for rooftop planes and features that obstruct PV placement. Rectangles are convenient for object detection and fast review. Yet a real roof is a three-dimensional structure whose slopes, ridges and irregular boundaries are seldom perfect rectangles. Accurate panel layouts require more geometric information. A label locating a likely usable plane does not establish its orientation, shading across seasons, structural suitability or expected electricity production.[1]

From image labels to an engineering survey

The released tiles do not retain the per-image georeferencing or polygon geometry needed for engineering-grade solar-potential estimates. Real panel layout also depends on roof dimensions, orientation, shading and structural suitability. The collection is therefore most directly useful for detecting and reviewing roof features before those engineering measurements are added.[1]

The dataset instead addresses a prior step: creating auditable training and evaluation material for identifying roof planes and obstructions. It can support experiments on how much human review changes machine proposals and how model performance varies across architectural styles. Because the data includes annotation history rather than only a single 'approved' label, researchers may also identify ambiguous classes and design more focused instructions for reviewers.[1]

Making review work visible

Many AI datasets conceal human work in the final label. RoofeNet-Multi makes part of that work visible. Reviewers are not merely cleaning a finished answer; their changes provide evidence about what the original machine suggested and where interpretation diverged. That creates an opportunity to measure review cost and the types of error that matter most for a practical solar workflow. A missed chimney may deserve different attention from a slight boundary adjustment.[1]

For researchers, the unusual asset is the annotation history rather than a single polished answer. It makes possible a more revealing evaluation of review workflows: which objects people add, which boundaries they change, and which machine proposals survive. A better solar-mapping pipeline begins with knowing how its training labels were made.[1]

Report an error or suggest a correction