Former House speaker Salvatore F. DiMasi was removed from the federal prison in Brooklyn, N.Y., early yesterday, raising speculation that he was being escorted to the federal courthouse in Worcester to testify before a grand jury. Officials at the courthouse said they had not seen DiMasi yesterday, though grand jury testimony is closed to the public anyway. DiMasi, currently serving an eight-year prison sentence for public corruption, is listed as “in transit’’ by the US Bureau of Prisons, which means that he is traveling between prisons. He was removed last week from the Federal Medical Center in Lexington, Ky, where he was initially incarcerated.