Looking through the quest stages, it looks like you have to fully complete the quest for the random dragons to start appearing, this means going back to the Jarl and accepting the reward for killing Mirmulnir the dragon.An interesting sidenote is that just as you are about to kill Mirmulnir, it shouts out "Dovakiin, No!"