To improve the accuracy and reliability of driving fatigue detection,three typical color characters are selected to construct the Fisher classifier for extracting lip region based on chroma distribution diversity.Region connecting algorithm is used to filter the noise in binary lip area.Improved integral projection algorithm is adopted to locate the lip boundary.Whether drivers are fatigued is recognized by the open-close degree of mouth and yawning frequency.Experiments show that the Fisher classifier based on three color characters has obvious superiority over that based on single color character in lip color extraction,the improved integral projection algorithm can enhance the accuracy and speed of locating lip boundary,and the driving fatigue detection based on yawning frequency has better accuracy and reliability.Fusing multi-color characters can promote the detection result of driving fatigue on the premise of ameliorating the robustness and reliability of lip color extraction.