Abstract:Indices such as tumor cell density,nucleocytoplasmic ratio,and average size have important implications for cancer grading and prognosis.Therefore,segmentation of nuclei is the fundamental prerequisite for tumor microenvironment analysis in computational pathology.Additionally,the exploration of new tumor markers is of great significance through statistical analysis of segmentation results.However,the morphology of nuclei in the background of pathological images is irregular,the staining of nuclei is uneven,and adhesion occurs between the edges of nuclei.While the segmentation error of the edge will make no difference on the overall loss as long as the main body of the nucl is correctly segmented,so the adhering nuclei can easily be regarded as the same segmentation target by existing deep learning algorithms.To address the overlapping nuclei,a new segmentation algorithm based on the Transformer and distance map,abbreviated as TDM-Net,is proposed,which integrates the core of multi-head self-attention mechanism in Transformer with contextual information to fully explore the proximity relationship and enhances the learning ability of image details by introducing distance map to emphasize the interior of nuclei and weaken the boundary of nuclei.The algorithm's Dice coefficient,precision,Aggregated Jaccard Index (AJI) and Hausdorff distance are 0.797 9,0.756 1,0.667 2,and 10.11,respectively.The results show that the proposed TDM-Net outperforms other segmentation algorithms,effectively improves nuclei segmentation accuracy and solves overlapping of different nuclei.