Non-Sampling Error

What is Non-Sampling Error?

Non-sampling error refers to an error that arises from the result of data collection, which causes the data to differ from the true values. It is different from sampling error, which is any difference between the sample values and the universal values that may result from a limited sampling size.

Non-sampling errors can come in various forms, including non-response error, measurement error, interviewer error, adjustment error, and processing error.

Mechanics of Non-Sampling Error

Non-sampling error can arise when either a sample or an entire population (census) is taken. It falls under two categories:

1. Random errors

Random errors are errors that cannot be accounted for and just happen. In statistical studies, it is believed that each random error offsets each other, generally speaking, so they are of little to no concern.

2. Systematic errors

Systematic errors affect the sample of the study and, as a result, will often create useless data. A systematic error is consistent and repeatable, so the study’s creators must take great care to mitigate such an error.

Non-sampling errors can occur from several aspects of a study. The most common non-sampling errors include errors in data entry, biased questions and decision-making, non-responses, false information, and inappropriate analysis.

Types of Non-Sampling Errors

There are several types of non-sampling errors, including:

1. Non-response error

A non-response error is caused by the differences between the people who choose to participate compared to the people who do not participate in a given survey. In other words, it exists when people are given the option to participate but choose not to; therefore, their survey results are not incorporated into the data.

2. Measurement error

A measurement error refers to all errors relating to the measurement of each sampling unit, as opposed to errors relating to how they were selected. The error often arises when there are confusing questions, low-quality data due to sampling fatigue (i.e., someone is tired of taking a survey), and low-quality measurement tools.

3. Interviewer error

Interviewer error occurs when the interviewer (or administrator) makes an error when recording a response. In qualitative research, an interviewer may lead a respondent to answer a certain way. In quantitative research, an interviewer may ask the question differently, which leads to a different result.

4. Adjustment error

An adjustment error describes a situation where the analysis of the data adjusts it so that it is not entirely accurate. Forms of adjustment error include errors with weighting the data, data cleaning, and imputation.

5. Processing error

A processing error arises when there is a problem with processing the data that causes an error of some kind. An example will be if the data were entered incorrectly or if the data file is corrupt.

Sampling Error vs. Non-Sampling Error

Often, sampling error and non-sampling error are used in similar contexts, but there are some crucial differences between both concepts. They include:

1. Sampling error can arise even when no apparent mistake’s been made, as opposed to non-sampling error, which arises when a mistake occurs.

2. Sampling error occurs when the sample is not representative of the universal truth, whereas non-sampling error is specific to a certain study design.

3. Sampling error can be reduced greatly as sampling size increases, but non-sampling error requires more methodical processes to reduce.

4. Sampling error is often caused by internal factors, whereas non-sampling error is caused by external factors not entirely related to a survey, study, or census.

How to Reduce Errors

Reducing non-sampling error is not as easily achieved as reducing sampling error. With sampling error, you can reduce the risk of error by simply increasing the sample size. It will not work for non-sampling error, which is often very difficult to detect and eliminate (unless very methodical consideration is given to the source of the error).

To effectively reduce non-sampling error, careful consideration must be taken by those designing the study to ensure the validity of the results. As such, a researcher may design a mechanism into the study to reduce the error while subsequently not introducing another error.

For example, a researcher may pay the individual a bonus depending on the accuracy of their data entry, or they may film all interviews to ensure that the interviewer stays on topic and on script.

Additional Resources

CFI is the official provider of the Business Intelligence & Data Analyst (BIDA)® certification program, designed to transform anyone into a world-class analyst.

In order to help you become a world-class analyst and advance your career to your fullest potential, these additional resources will be very helpful:

• Cluster Sampling
• Parameter
• Sample Selection Bias
• Type I Error
• See all data science resources