Evidence from developed countries shows that there is a significant gender gap in STEM occupations. Girls may begin to underperform in math early as primary school. One possible explanation is the negative stereotype threat towards girls. However, this has been understudied in rural China. In this paper, we describe the math performance gender gap in rural China, compare the gender gap between rural and urban China, and finally compare the Chinese situation with other countries. We further examine possible explanations for the math performance gender gap from comparative perspectives. Using first hand datasets of 3,789 primary students and 12,702 junior high students in northwest China, combing with OECD 2012 Program for International Student Assessment (PISA) survey data, we find that in both rural and urban China, boys outperform girls in math. As students grow older, the gap widens. The size of the gender gap in rural China is larger than that in urban China, and larger than in many other countries. We further find that both the gender gaps in math self-concept and math anxiety and discriminatory family investment towards girls are not sufficient to explain the wide math performance gaps. Our study suggests the inequality of education in rural China still merits concern and calls for further work to explain the observed gender gap in math performance.