Risk of bias requirements
When a study is entered into the HAWC database for use in an assessment, risk of bias metrics can be entered for a metric of bias for each study. Risk of Bias metrics are organized by domain. The following questions are required for evaluation for this assessment.
Requirements by Study Type
| Domain | Metric | Bioassay | Epidemiology | In Vitro |
|---|---|---|---|---|
| Selection302 | Was administered dose or exposure level adequately randomized?665 | ✔ | - | ✔ |
| Selection302 | Did selection of study participants result in the appropriate comparison groups?667 | - | ✔ | ✔ |
| Selection302 | Was allocation to study groups adequately concealed?666 | ✔ | - | ✔ |
| Confounding303 | Did study design or analysis account for important confounding and modifying variables? 668 | - | ✔ | ✔ |
| Performance304 | Were experimental conditions identical across study groups?670 | ✔ | - | ✔ |
| Performance304 | Were the research personnel blinded to the study group during the study?672 | ✔ | - | ✔ |
| Attrition305 | Were outcome data complete without attrition or exclusion from analysis?673 | ✔ | ✔ | ✔ |
| Detection306 | Can we be confident in the exposure characterization?676 | ✔ | ✔ | ✔ |
| Detection306 | Can we be confident in the outcome assessment?677 | ✔ | ✔ | ✔ |
| Selective Reporting307 | Were all measured outcomes reported?678 | ✔ | ✔ | ✔ |
| Other308 | Were there no other potential threats to internal validity?679 | ✔ | ✔ | ✔ |
Selection302
Was administered dose or exposure level adequately randomized?665
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization). This applies to a concurrent negative control group (i.e., a group for which exposure is to vehicle or media alone or un-treated) which must be included in the study to address randomization as well as any positive control group that may be part of the study.
Was allocation to study groups adequately concealed?666
Allocation concealment requires that research
personnel allocating subjects or animals to treatment groups (including the control
group) could not foresee which administered dose or exposure level is going to
be assigned at the start of a study. Human studies also require that allocation
be concealed from human subjects prior to entering the study.
Performance304
Were experimental conditions identical across study groups?670
Housing conditions and husbandry practices should be identical across control and experimental groups because these variables may impact the outcome of interest. Identical conditions include use of the same vehicle in control and experimental animals and the rating for this risk-of-bias element will depend largely on the consistent use vehicle across treatment groups. That is because under current reporting practices it is unlikely that similarity of conditions will be explicitly reported in most animal studies. Thus, we will assume unless stated otherwise that experimental conditions (other than treatment vehicle) were identical across groups.
Were the research personnel blinded to the study group during the study?672
Blinding requires that study scientists do not know which administered dose or exposure level the human subjects or animals are being given (i.e., study group). Human controlled-exposure studies also require blinding of the human subjects when possible.
Attrition305
Were outcome data complete without attrition or exclusion from analysis?673
Incomplete outcome
data includes loss due to attrition (nonresponse, dropout, or loss to follow-up)
or exclusion from analyses. The degree of bias resulting from incomplete outcome data depends on
the reasons that outcomes are missing, the amount and distribution of missing
data across groups, and the potential association between outcome values and
likelihood of missing data.
Detection306
Can we be confident in the exposure characterization?676
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?677
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups AND that the outcome assessors were adequately blinded to the study group.
Selective Reporting307
Were all measured outcomes reported?678
Selective reporting is present if pre-specified outcomes are not reported or incompletely reported.
Other308
Were there no other potential threats to internal validity?679
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include inappropriate statistical methods, etc.
Selection302
Did selection of study participants result in the appropriate comparison groups?667
Comparison group appropriateness refers to having similar baseline characteristics of factors related to the outcome measures of interest between groups aside from the exposures (and outcomes for case-control studies).
Confounding303
Did study design or analysis account for important confounding and modifying variables? 668
Interpretation of study findings may be distorted by failure to consider the extent to which systematic differences in baseline characteristics risk factors, prognostic variables, or co-occurring exposures among comparison groups may reduce or increase the observed effect. Confounding variables or confounders include any factor that is: 1) associated with the exposure, 2) an independent risk factor for a given outcome, and 3) unequally distributed between study groups. The potential confounder cannot be an intermediate effect on the causal pathway between exposure and the outcome.
Attrition305
Were outcome data complete without attrition or exclusion from analysis?673
Incomplete outcome
data includes loss due to attrition (nonresponse, dropout, or loss to follow-up)
or exclusion from analyses. The degree of bias resulting from incomplete outcome data depends on
the reasons that outcomes are missing, the amount and distribution of missing
data across groups, and the potential association between outcome values and
likelihood of missing data.
Detection306
Can we be confident in the exposure characterization?676
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?677
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups AND that the outcome assessors were adequately blinded to the study group.
Selective Reporting307
Were all measured outcomes reported?678
Selective reporting is present if pre-specified outcomes are not reported or incompletely reported.
Other308
Were there no other potential threats to internal validity?679
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include inappropriate statistical methods, etc.
Selection302
Was administered dose or exposure level adequately randomized?665
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization). This applies to a concurrent negative control group (i.e., a group for which exposure is to vehicle or media alone or un-treated) which must be included in the study to address randomization as well as any positive control group that may be part of the study.
Did selection of study participants result in the appropriate comparison groups?667
Comparison group appropriateness refers to having similar baseline characteristics of factors related to the outcome measures of interest between groups aside from the exposures (and outcomes for case-control studies).
Was allocation to study groups adequately concealed?666
Allocation concealment requires that research
personnel allocating subjects or animals to treatment groups (including the control
group) could not foresee which administered dose or exposure level is going to
be assigned at the start of a study. Human studies also require that allocation
be concealed from human subjects prior to entering the study.
Confounding303
Did study design or analysis account for important confounding and modifying variables? 668
Interpretation of study findings may be distorted by failure to consider the extent to which systematic differences in baseline characteristics risk factors, prognostic variables, or co-occurring exposures among comparison groups may reduce or increase the observed effect. Confounding variables or confounders include any factor that is: 1) associated with the exposure, 2) an independent risk factor for a given outcome, and 3) unequally distributed between study groups. The potential confounder cannot be an intermediate effect on the causal pathway between exposure and the outcome.
Performance304
Were experimental conditions identical across study groups?670
Housing conditions and husbandry practices should be identical across control and experimental groups because these variables may impact the outcome of interest. Identical conditions include use of the same vehicle in control and experimental animals and the rating for this risk-of-bias element will depend largely on the consistent use vehicle across treatment groups. That is because under current reporting practices it is unlikely that similarity of conditions will be explicitly reported in most animal studies. Thus, we will assume unless stated otherwise that experimental conditions (other than treatment vehicle) were identical across groups.
Were the research personnel blinded to the study group during the study?672
Blinding requires that study scientists do not know which administered dose or exposure level the human subjects or animals are being given (i.e., study group). Human controlled-exposure studies also require blinding of the human subjects when possible.
Attrition305
Were outcome data complete without attrition or exclusion from analysis?673
Incomplete outcome
data includes loss due to attrition (nonresponse, dropout, or loss to follow-up)
or exclusion from analyses. The degree of bias resulting from incomplete outcome data depends on
the reasons that outcomes are missing, the amount and distribution of missing
data across groups, and the potential association between outcome values and
likelihood of missing data.
Detection306
Can we be confident in the exposure characterization?676
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?677
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups AND that the outcome assessors were adequately blinded to the study group.
Selective Reporting307
Were all measured outcomes reported?678
Selective reporting is present if pre-specified outcomes are not reported or incompletely reported.
Other308
Were there no other potential threats to internal validity?679
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include inappropriate statistical methods, etc.