Risk of bias requirements
When a study is entered into the HAWC database for use in an assessment, risk of bias metrics can be entered for a metric of bias for each study. Risk of Bias metrics are organized by domain. The following questions are required for evaluation for this assessment.
Requirements by Study Type
| Domain | Metric | Bioassay | Epidemiology | In Vitro |
|---|---|---|---|---|
| Selection225 | Was administered dose or exposure level adequately randomized?500 | ✔ | ✔ | ✔ |
| Selection225 | Was allocation to study groups adequately concealed?501 | ✔ | ✔ | ✔ |
| Selection225 | Were the comparison groups appropriate?502 | ✔ | ✔ | ✔ |
| Selection225 | Did selection of study participants result in the appropriate comparison groups?3347 | - | ✔ | ✔ |
| Confounding226 | Did study design or analysis account for important confounding and modifying variables?3348 | - | ✔ | ✔ |
| Confounding226 | Did researchers adjust or control for other exposures that are anticipated to bias results?504 | ✔ | ✔ | ✔ |
| Performance227 | Were experimental conditions identical across study groups?505 | ✔ | ✔ | ✔ |
| Performance227 | Did deviations from the study protocol impact the results?506 | ✔ | ✔ | ✔ |
| Performance227 | Were the research personnel and human subjects blinded to the study group during the study?507 | ✔ | ✔ | ✔ |
| Attrition228 | Were outcome data incomplete due to attrition or exclusion from analysis?508 | ✔ | ✔ | ✔ |
| Detection229 | Were the outcome assessors blinded to study group or exposure level?509 | ✔ | ✔ | ✔ |
| Detection229 | Were confounding variables assessed consistently across groups using valid and reliable measures?510 | ✔ | ✔ | ✔ |
| Detection229 | Can we be confident in the exposure characterization?511 | ✔ | ✔ | ✔ |
| Detection229 | Can we be confident in the outcome assessment?512 | ✔ | ✔ | ✔ |
| Selective Reporting230 | Were all measured outcomes reported?513 | ✔ | ✔ | ✔ |
| Other231 | Were there any other potential threats to internal validity?514 | ✔ | ✔ | ✔ |
Selection225
Was administered dose or exposure level adequately randomized?500
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Was allocation to study groups adequately concealed?501
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Were the comparison groups appropriate?502
Comparison group appropriateness refers to having similar baseline characteristics between the groups aside from the exposures and outcomes under study.
Confounding226
Did researchers adjust or control for other exposures that are anticipated to bias results?504
Performance227
Were experimental conditions identical across study groups?505
Did deviations from the study protocol impact the results?506
Were the research personnel and human subjects blinded to the study group during the study?507
Blinding requires that study scientists do not know which administered dose or exposure level the human subject or animal is being given (i.e., study group). Human studies also require blinding of the human subjects when possible.
Attrition228
Were outcome data incomplete due to attrition or exclusion from analysis?508
Attrition rates are required to be similar and uniformly low across groups with respect to withdrawal or exclusion from analysis
Detection229
Were the outcome assessors blinded to study group or exposure level?509
Blinding requires that outcome assessors do not know the study group or exposure level of the human subject or animal when the outcome was assessed.
Were confounding variables assessed consistently across groups using valid and reliable measures?510
Consistent application of valid, reliable, and sensitive methods of assessing important confounding or modifying variables is required across study groups.
Note, a parallel question under selection bias addresses whether design or analysis account for confounding.
Can we be confident in the exposure characterization?511
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?512
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups.
Selective Reporting230
Were all measured outcomes reported?513
Other231
Were there any other potential threats to internal validity?514
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include inappropriate statistical methods, etc.
Selection225
Was administered dose or exposure level adequately randomized?500
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Was allocation to study groups adequately concealed?501
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Were the comparison groups appropriate?502
Comparison group appropriateness refers to having similar baseline characteristics between the groups aside from the exposures and outcomes under study.
Did selection of study participants result in the appropriate comparison groups?3347
Confounding226
Did study design or analysis account for important confounding and modifying variables?3348
Did researchers adjust or control for other exposures that are anticipated to bias results?504
Performance227
Were experimental conditions identical across study groups?505
Did deviations from the study protocol impact the results?506
Were the research personnel and human subjects blinded to the study group during the study?507
Blinding requires that study scientists do not know which administered dose or exposure level the human subject or animal is being given (i.e., study group). Human studies also require blinding of the human subjects when possible.
Attrition228
Were outcome data incomplete due to attrition or exclusion from analysis?508
Attrition rates are required to be similar and uniformly low across groups with respect to withdrawal or exclusion from analysis
Detection229
Were the outcome assessors blinded to study group or exposure level?509
Blinding requires that outcome assessors do not know the study group or exposure level of the human subject or animal when the outcome was assessed.
Were confounding variables assessed consistently across groups using valid and reliable measures?510
Consistent application of valid, reliable, and sensitive methods of assessing important confounding or modifying variables is required across study groups.
Note, a parallel question under selection bias addresses whether design or analysis account for confounding.
Can we be confident in the exposure characterization?511
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?512
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups.
Selective Reporting230
Were all measured outcomes reported?513
Other231
Were there any other potential threats to internal validity?514
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include inappropriate statistical methods, etc.
Selection225
Was administered dose or exposure level adequately randomized?500
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Was allocation to study groups adequately concealed?501
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Were the comparison groups appropriate?502
Comparison group appropriateness refers to having similar baseline characteristics between the groups aside from the exposures and outcomes under study.
Did selection of study participants result in the appropriate comparison groups?3347
Confounding226
Did study design or analysis account for important confounding and modifying variables?3348
Did researchers adjust or control for other exposures that are anticipated to bias results?504
Performance227
Were experimental conditions identical across study groups?505
Did deviations from the study protocol impact the results?506
Were the research personnel and human subjects blinded to the study group during the study?507
Blinding requires that study scientists do not know which administered dose or exposure level the human subject or animal is being given (i.e., study group). Human studies also require blinding of the human subjects when possible.
Attrition228
Were outcome data incomplete due to attrition or exclusion from analysis?508
Attrition rates are required to be similar and uniformly low across groups with respect to withdrawal or exclusion from analysis
Detection229
Were the outcome assessors blinded to study group or exposure level?509
Blinding requires that outcome assessors do not know the study group or exposure level of the human subject or animal when the outcome was assessed.
Were confounding variables assessed consistently across groups using valid and reliable measures?510
Consistent application of valid, reliable, and sensitive methods of assessing important confounding or modifying variables is required across study groups.
Note, a parallel question under selection bias addresses whether design or analysis account for confounding.
Can we be confident in the exposure characterization?511
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?512
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups.
Selective Reporting230
Were all measured outcomes reported?513
Other231
Were there any other potential threats to internal validity?514
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include inappropriate statistical methods, etc.