This study focuses on how social reform and social changes affect the educational inequality between social groups, specifically the rural and urban Chinese. Existing sociology theories address the class difference drives educational inequality, and treat social group differences as a byproduct of class differences. By analyzing the evolution and components of the educational gap between rural and urban residents in China, this study found that the main driver of the educational gap between rural and urban residents varies in different historical periods. In the low social stratification period, if the effect of institutional discrimination is small, the educational gap is small; in the high social stratification period with more significant institutional discrimination, the inferior rural residents are double jeopardized. These results imply the educational inequality between social groups is driven by both structural changes and discriminatory policies, but if institutional discrimination is the main driver, policies aim to reduce the structural inequality will have little effect on reducing inequality.