Psychology & Neuroscience Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It only takes a minute to sign up.

I was reading through an article about research design. The authors gave an example on the relation between internal and external validity which I thought might be an important area of consideration when designing research proposals.

"A restricted study population (and exclusion criteria) may limit bias
and increase the internal validity of the study; however, this
approach will limit external validity of the study and, thus, the
generalizability of the findings to the practical clinical setting.
Conversely, a broadly defined study population and inclusion criteria
may be representative of practical clinical practice but may increase
bias and reduce the internal validity of the study."

Broadly speaking, what questions can be asked to help decide one should strike a balance between the internal and external validity in our research proposal designs? Is it necessary then to carry out a pilot study first to establish some level of internal validity before moving on to establish external validity in a much bigger and formal follow-up study?