Abstract:Abstract: Indices such as tumor cell density, nucleocytoplasmic ratio, and average size have important implications for cancer grading and prognosis. Therefore, nuclear segmentation is the basis of tumor microenvironment analysis in computational pathology. In addition, the exploration of new tumor markers is of great significance through statistical analysis of segmentation results. However, the morphology of nuclei in the background of pathological images is irregular, the staining of nuclei is uneven, and there is a problem with adhesion between the edges of nuclei. In the existing deep learning algorithm, when the main body of the nucleus is correctly segmented, the segmentation error of the edge will not have much impact on the overall loss, so the adhering nucleus can easily be regarded as the same segmentation target. In order to solve the nuclear overlapping problems, a new segmentation algorithm based on the Transformer and Distance map, TDM-Net, is proposed, which integrates the core of multi-head self-attention mechanism in Transformer with contextual information to fully explore the proximity relationship and enhances the learning ability of image details by introducing distance map to emphasize the interior of nuclei and weaken the boundary of nuclei. The algorithm's dice coefficient, precision, aggregated jaccard index and hausdorff distance are 0.7979, 0.7561, 0.6672, and 10.11, respectively. The results show that TDM-Net performs better than other segmentation algorithms, effectively improving nuclei segmentation accuracy and solving overlapping problems with different nuclei.