Here,we propose a pruning and optimization approach based on Gradient Weight Pursuit (GWP) to address the overfitting in unsupervised domain,which manifests as significantly lower accuracy on downstream tasks compared to that on training sets.To tackle the overfitting challenge in unsupervised domain,we employ the dense-sparse-dense strategy,focusing on both difference-based and adversarial adaptive methods.First,the network is pretrained intensively to identify crucial connections.Second,during the pruning stage,the optimization algorithm in this paper distinguishes itself from original dense-sparse-dense strategy by jointly considering both weight and gradient information.Specifically,it leverages both weight (i.e.zero-order information) and gradient (i.e.first-order information) to influence pruning process.In the final dense phase,the pruned connections are restored and the dense network is retrained with a reduced learning rate.Finally,the obtained network achieves desirable outcomes in downstream tasks.The experimental results show that the proposed GWP approach can effectively improve the accuracy of downstream tasks,offering a plug-and-play capability compared with original difference-based and adversarial domain adaptation methods.