Validity and Research Design

Questions: 1. compare and contrast the characteristics of external, internal, and construct validity. 2. Identify the threats to external and construct validity.3. Discuss how validity issues could impact your envisioned research?4. Evaluate the differences between facts and theory, as well as concepts and variables. Answers: Introduction: Validity can be referred as one of the important factors that need to be taken into consideration during the research process. In the field of statistics and science, validity is the extent to which a measurement, conclusion or a concept is well-founded and it should also correspond to the real world accurately. The degree, to which the measurement tool can measure what it claims to measure, can be assessed with the help of validity of the tool. Test validity is applied in the case of psychometric tests and the validity measures the extent to which evidence can support the test score interpretation. The validity in a scientific research design can measure if the research can answer the questions that are asked at the beginning of the research. So it can be said that the validity addresses the nature of reality and to estimate whether truth can be achieved from the research. The main aim of this paper is to compare and contrast the characteristics of various forms of validity in the c ontext of a psychometric test like measuring the memory of a person or personality testing. Compare and contrast of the characteristics of external, internal, and construct validity: External Validity Internal Validity Construct validity External validity can be referred as the causal inference validity in the research process; External validity is the extent to which the generalisation of the results of a situation can be done to other people in other situations; The pre-test impact, the post-test impact and reactivity need to be addressed; In external validity all situations must be specific as it can limit the generalisation (Blagov, Bi, Shedler Westen, 2012); The cause and effect relationship should be established in the pre-test as well as in the post test situation. Internal validity can be referred as the inductive estimate of the degree to which causal relationship conclusions can be made; The internal validity represents the extent to which the warranty of a causal conclusion can be given; If the systematic error can be minimised then the generalisation can be warranted (Cakmur, 2012); It is very important to demonstrate a causal relationship between the two variables. For example in case of memory test the difference in recall capacity can be affected by the type of list that is given during the experiment. The recall output can be higher if it is a controlled list and the recall output can be lower if it is a chunk list. Construct validity can be referred as the degree to which the test measures can claim what it intended to measure; It is also contended as the overarching concern of the research validity and all types of validity evidences are subsumed in the construct validity; The appropriateness of the test measures are assessed on the basis of the construct validity; It can estimate whether the test constructs what it intended to construct. For the overall validity measure of a test, construct validity must be maintained; The theoretical ideas and the rationales are the main basis of the judging the appropriateness of the results of the test scores. The trait under consideration is supported by the tests and it can present the construct validity. In the case of psychometric tests there are various aspects like the intelligence, personality, memory that can be measured by using the psychometric tests and there are various theories that are presented for each of these factors (Doran, Safran, Waizmann, Bolger Muran, 2012); The construct validity basically helps whether the results can comply with the theories and the rationales. There are basically six items that can measure quality of the construct validity; These are consequential, that measures the potential risks of invalidity of the measures. The second factor is the content of the measure. The third factor is the substantial nature of the theoretical foundation of the construct validity; The fourth factors is the structural nature and the sixth factors is the external qualities of the measure like the discrimination, convergence etc. The last factors are the generalisation measure. Threat to External Validity: There are several threats that can be faced in case of external validity especially in terms of making the generalisation. When the independent or the cause variable relies on other factors then there can be limitations in the generalisation. So the interactions of all the threats are with the causal or independent variable. Some if the threats to external validity can be mentioned here. The experimental treatments can interact with the pre-testing and it can create certain impact that may hamper the generalisation on the untested population. There can be experimental treatments and selection biases that can interact in the process and thus the results can be affected which threatens the validity. It can create a reactive impact on the experimental arrangements and there can be interference as well for the multiple-treatment. Threats to Construct Validity: There can be several threats to construct validity as well. The threats to the construct validity can be stated here. The preoperational explication of the constructs can be inadequate. It can create severe issue as it can impact the results of the experiment. There may exist the mono-operational bias in terms of the treatment, programme, cause or independent variables of the study and thus it can also affect the outcomes. In this context it can be said that the operationalisation can be flawed in terms of the construct validity which can create major issues. There can be threat due to the mono-method bias. This is related with the observations or the measures of the research study. There can be threat when there is interaction between the treatments and the testing as it can make the groups more receptive or sensitive towards the treatments. Threat can be in terms of restrictions for the generalisations across the constructs. These are also referred as unintended consequences and thus may lead to side effects in the research (Jensen, Wang, Potts Gould, 2012). Impact of Validity Issues on the Research: Here it can be said that the validity issues and the threats that are presented here can lead to severe consequences for the psychometric tests as well. In case of psychometric tests of personality or memory, the validity of the experiment is very important. In case of psychometric tests the validity is referred as test validity (Neophytou, 2012). If there are validity issues then the results cannot be generalised and thus it will not measure what it intends to measure. The validity in the experiments can exist if there are issues in terms of bias selection of the respondents. It is known that the human nature is different and thus there can be some other factors that can affect the results. Difference between Facts Theory and Concepts Variables: It can be said that facts and theories can be used in different context. The facts are those that can be observed readily by an individual. Any objectives can be pertained by the facts. The facts are undeniable and obvious phenomena under a controlled environment. On the other hand the theories are the explanations to the observations or the facts (Jensen, Wang, Potts Gould, 2012). Hypotheses can become a theory after the establishing relationship between the facts. A concept is basically an abstraction on the basis of the characteristics of the reality that is perceived. Thus the concepts are basically abstract ideas that human beings use and they are systematically independent (Nestor Schutt, 2012). On the other hand the variables are the representation of a concept. The variables are basically constrained by implementation and the development of the systems and these are linked to a concept for ensuring the representation. Conclusion: In conclusion it can be said that the validity and the reliability are the two main indicators of the quality or usefulness of an experiment. The validity measures whether the experiment has successfully measured what it intends to measure and the reliability indicates whether there consistency of the construct across time. It is evident that for validity, reliability is a necessary but not the sufficient condition. In this assignment various aspects of validity measures for the psychometric tests and the issues that can be raised in the process are discussed. References: Blagov, P., Bi, W., Shedler, J., Westen, D. (2012). The Shedler-Westen Assessment Procedure (SWAP): Evaluating Psychometric Questions About Its Reliability, Validity, and Impact of Its Fixed Score Distribution.Assessment,19(3), 370-382. doi:10.1177/1073191112436667 Cakmur, H. (2012). MEASUREMENT-RELIABILITY-VALIDITY IN RESEARCH.TAF Prev Med Bull, 1. doi:10.5455/pmb.1-1322486024 Doran, J., Safran, J., Waizmann, V., Bolger, K., Muran, J. (2012). The Alliance Negotiation Scale: Psychometric construction and preliminary reliability and validity analysis.Psychotherapy Research,22(6), 710-719. doi:10.1080/10503307.2012.709326 Jensen, M., Wang, W., Potts, S., Gould, E. (2012). Reliability and Validity of Individual and Composite Recall Pain Measures in Patients with Cancer.Pain Medicine,13(10), 1284-1291. doi:10.1111/j.1526-4637.2012.01470.x Neophytou, L. (2012). Examining the validity and reliability of the Greek version of the Bar-Ons Emotional Quotient Inventory.Educational Research Ejournal,1(2), 135-152. doi:10.5838/erej.2012.12.04 Nestor, P., Schutt, R. (2012).Research methods in psychology. Los Angeles, CA: SAGE Publications.