Brain tumor is one of the most fatal cancers in the world.Its image classification has become the hot spot due to the diverse characteristics of brain tumors.In recent years,Deep Neural Networks (DNN) are commonly used for medical image classification,but the problem of gradient vanishing and over fitting will appear with the increase of depth,while the Residual Network (ResNet) can solve this problem by introducing identity mapping.Therefore,this paper proposes an MDM-ResNet approach for brain tumor classification,which is composed of multi-size convolution kernel module,dual-channel pooling layer and multi-depth fusion residual block.The experiment was carried out on Figshare dataset,the image was preprocessed by data enhancement operation,and the performance of network was evaluated based on five-fold cross validation.The experimental results prove that the MDM-ResNet approach can effectively classify meningioma,glioma and pituitary tumor.