Executive function (EF) refers to a set of cognitive abilities involved in self-regulated behavior. Given the critical role of EF in cognition, strategies for improving EF have attracted intensive attention in recent years. Previous studies have explored the effects of abacus-based mental calculation (AMC) training on several cognitive abilities. However, it remains unclear whether AMC training affects EF and its neural correlates. In this study, participants were randomly assigned to AMC or control groups upon starting primary school. The AMC group received 2 h AMC training every week, while the control group did not have any abacus experience. Neural activity during an EF task was examined using functional MRI for both groups in their 4th and 6th grades. Our results showed that the AMC group performed better and faster than the control group in both grades. They also had lower activation in the frontoparietal reigons than the control group in the 6thgrade. From the 4th to the 6th grade, the AMC group showed activation decreases in the frontoparietal regions, while the control group exhibited an opposite pattern. Furthermore, voxel-wise regression analyses revealed that better performance was associated with lower task-relevant brain activity in the AMC group but associated with greater task-relevant brain activity in the control group. These results suggest that long-term AMC training, with calculation ability as its original target, may improve EF and enhance neural efficiency of the frontoparietal regions during development.