There's no "ground" for the current to "leak" to. If you used the test light from the GFCI protected hot, to the ground or grounded conductor of an extension cord, plugged into a grounded receptacle, the GFCI would trip.

You've got to watch those 2 wire GFI's when they have a metal cover. I stuck a bugeye tester in one and pressed the test button and that energized the faceplate. The GFI passed the trip test as I completed the path to ground because I was touching the faceplate.

Keep in mind that a GFI receptacle is not a cure all for 2 wire replacements. 250-114 has a list of equipment that is required to be grounded and a GFI on a 2 wire circuit doesn't provide the required grounding.

Few things are harder to put up with than the annoyance of a good example.