Estimating Polyp Size From a Single Colonoscopy Image Using a Shape-From-Shading Model

Colonoscopy (CO) is the most useful procedure to estimate the polyp size as part of surveillance and therapeutic management to prevent Colorectal cancer. Studies have reported a high rate of misestimated lesions by experts, reaching a relative accuracy of 67%. This work presents a method to estimate polyp size from a raw CO frame. A shape-from-shading model trained with synthetic images estimates a depth map to reconstruct the three-dimensional (3d) colon structure. Polyp is segmented in RGB image with a U-Net, to compute diameter in pixels. This diameter is projected onto the 3d surface to compute polyp size in millimeters. The method obtained a mean absolute error of 2.16 mm and relative accuracy of 88.17% in 2802 synthetic images, and 2.06 mm in 100 real images. In a binary classification task (< 10 mm or > 10 mm), the method achieved macro F1 scores of 89% and 72% in the synthetic and real databases respectively.