If the tests are not well-matched to the task, individual, and situation at hand, we will have a case of mismatched validity.

Some tests are useful in diverse situations, but no test works well for all tasks with all people in all situations. In his classic 1967 article, Gordon Paul helped psychotherapists move away from the oversimplified search for effective therapies toward a more difficult but meaningful question: 'What treatment, by whom, is most effective for this individual with that specific problem, and under which set of circumstances?'


Selecting assessment instruments involves similarly complex questions, such as: 'Has research established sufficient reliability and validity (as well as sensitivity, specificity, and other relevant features) for this test, with an individual from this population, for this task (i.e., the purpose of the assessment), in this set of circumstances?' It is important to note that as the population, task, or circumstances change, the measures of validity, reliability, sensitivity, etc., will also tend to change.

* * *

Sources:
http://www.kspope.com/fallacies/assessment.php