Two years ago federal prosecutors won a delinquency finding against a boy accused of engaging in sex acts when he was 10 years old with other young boys on an Army base in Arizona—one of the youngest defendants ever pursued by the U.S. Justice Department.

The case, now being reviewed by the Ninth Circuit U.S. Court of Appeals, could open a new front in a long-running debate about how to handle juvenile sex offenders, whose cases generally have been tried in state, not federal, courts. The records are sealed because the...