NEW DELHI—India's new government chose incremental change over a major revamp for the country's ailing economy, saying Thursday it plans to simplify the tax system, make subsidy programs more efficient and raise the limit on foreign investment in two industries.

Presenting the national budget to Parliament, Finance Minister Arun Jaitley said the cap on foreign holdings in Indian defense and insurance companies would rise to 49% from 26%. But he gave mixed signals on retroactive taxation that has hit major foreign firms...