Due to the rapid development of human-computer interaction, affective computing has attracted more and more attention in recent years. Electroencephalogram (EEG) signals are easy to record and difficult to camouflage, it plays an important role in emotion recognition. However, due to the diversity of human emotions and individual variability, EEG-based emotion recognition is still a difficult problem in the field of affective computing. Aiming at this problem, a multi-source domain adaptive dictionary learning and sparse representation approach is proposed in this study. To reduce the difference of data distribution between the source domains and the target domain, the data of all domains are projected into the shared subspace, and a common dictionary is learned in the subspace to establish the relationship between the source domains and the target domain. The learning criterion of common dictionary is to minimize the intra-class sparse reconstruction error and maximize the inter-class sparse reconstruction error, which can ensure that the learned sparse representation has more recognition ability. In addition, to avoid the occurrence of negative transfer, each source domain corresponds to a domain adaptive weight, and the optimal value of the weight can be obtained in the adaptive learning. The model parameters are solved by the method of parameter alternate optimization, and all parameters can reach their optimal solutions at the same time. Experiments on the DEAP dataset verify the effectiveness of our method.