Skip to Main Content

Quantitative data collection and analysis

Reliability and Validity

Reliability in statistical terms can be considered the degree to which the results of a measurement, calculation can be depended on for its accuracy. In research it specifically refers to the consistency of a research study or test measure. If your findings from your research are replicated consistently they are reliable,

A correlation coefficient can be used to assess the degree of reliability. A reliable test would show a high positive correlation.

 

Internal reliability: is the extent to which items which make up a scale or measure are internally consistent i.e. it is looking at the consistency of people’s response.

split-half method: measures the extent to which all parts of the test contribute equally to what is being measured.
This is usually done by comparing the results of one half of a test with the results from the other half. The test can be split in half  in several ways (i.e. first half and second half, by odd numbers and even numbers). If the two halves show similar results then this suggests that the test has internal reliability. (Cronbach’s alpha)
It is not appropriate for a test which measure different things or constructs.

External reliability: is the extent to which a measure varies from one use to another,

test-retest reliability: measures consistency over time. Usually it would involve giving participants the same test on two separate occasions, e.g. on interviews. If similar results are obtained then external reliability is established.

Inter-rater: the degree to which different raters give consistent estimates, for instance of the same behaviour.

Validity is the extent to which a concept is measured in a accurate way. Is the claim well founded?

It can be thought of as does the measurement tool really measure what it says it does.

Content Validity - is an objective look at the items in a test to ascertain whether they all relate to and measure the construct in question

Concurrent Validity - is a comparison between two tests of a particular construct. One has normally been established as a valid measure and the other will be a new one. If the results of both show a significant correlation then the new one can be considered as valid.

Face Validity - is the extent to which a method of measurement appears ‘at face value’ to measure what you are interested in.

 

Internal validity - is the amount of confidence that the causal relationship being tested is trustworthy and is not influenced by other factors or variables.

External validity - is the extent to which the results from a study can be applied (or generalised) to other situations, or groups etc.