Risk of bias requirements
When a study is entered into the HAWC database for use in an assessment, risk of bias metrics can be entered for a metric of bias for each study. Risk of Bias metrics are organized by domain. The following questions are required for evaluation for this assessment.
Requirements by Study Type
| Domain | Metric | Bioassay | Epidemiology | In Vitro |
|---|---|---|---|---|
| Selection2349 | Did selection of study participants result in appropriate comparison groups?4159 | - | ✔ | ✔ |
| Selection2349 | Was allocation to study groups adequately concealed?4158 | ✔ | - | ✔ |
| Selection2349 | Was administered dose or exposure level adequately randomized?4157 | ✔ | - | ✔ |
| Confounding2350 | Did the study design or analysis account for important confounding and modifying variables?4160 | - | ✔ | ✔ |
| Performance2351 | Were experimental conditions identical across study groups?4161 | ✔ | - | ✔ |
| Performance2351 | Were the research personnel and human subjects blinded to the study group during the study?4162 | ✔ | - | ✔ |
| Attrition2352 | Were outcome data incomplete due to attrition or exclusion from analysis?4163 | ✔ | ✔ | ✔ |
| Detection2353 | Can we be confident in the exposure characterization?4164 | ✔ | ✔ | ✔ |
| Detection2353 | Can we be confident in the outcome assessment?4165 | ✔ | ✔ | ✔ |
| Selective Reporting2354 | Were all measured outcomes reported?4166 | ✔ | ✔ | ✔ |
| Other2355 | Were there any other potential threats to internal validity?4167 | ✔ | ✔ | ✔ |
Selection2349
Was allocation to study groups adequately concealed?4158
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Was administered dose or exposure level adequately randomized?4157
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Performance2351
Were experimental conditions identical across study groups?4161
Were the research personnel and human subjects blinded to the study group during the study?4162
Blinding requires that study scientists do not know which administered dose or exposure level the human subject or animal is being given (i.e., study group). Human studies also require blinding of the human subjects when possible.
Attrition2352
Were outcome data incomplete due to attrition or exclusion from analysis?4163
Attrition rates are required to be similar and uniformly low across groups with respect to withdrawal or exclusion from analysis
Detection2353
Can we be confident in the exposure characterization?4164
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?4165
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups.
Complete methods in protocol with some notes below:
- Definitely low risk of bias criteria note: Training on all AGD measurements for all examiners using well-described methods (preferred is calipers), intra-rater, inter-rater reliability assessed, testosterone measured by HPLC, GC-MS, LC-MS or equilibrium dialysis
- Probably low risk of bias criteria note: Description of training for AGD measurement methods with appropriate citation. Testosterone assay by RIA (not as reliable as methods listed in definitely low risk of bias but ok overall)
- Probably high risk of bias criteria note: Indirect evidence that the outcome assessment method is using an insensitive instrument or AGD assessment without intra-rater or inter-rater reliability
- Definitely high risk of bias criteria note: No description of AGD measurement methods or no description of testosterone laboratory assessment
Selective Reporting2354
Were all measured outcomes reported?4166
Other2355
Were there any other potential threats to internal validity?4167
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include appropriateness of statistical methods, adherence to the study-protocol, etc.
Selection2349
Did selection of study participants result in appropriate comparison groups?4159
Comparison group appropriateness refers to having similar baseline characteristics between the groups aside from the exposures and outcomes under study.
Confounding2350
Did the study design or analysis account for important confounding and modifying variables?4160
Note: The following variables should be considered as key/primary and/or effect modifiers in the analysis of the relationship between phthalates and male AGD outcomes: weight/body size at exam, measure of weight or body size at birth, age at exam, and measure of urinary dilution or indication that measure was adjusted for urinary dilution.
The following variables should be considered as additional potential confounders and/or effect modifiers but consideration is not required: maternal age, pre-pregnancy or maternal BMI, maternal education, maternal income, maternal race/ethnicity, and time of day of urine collect.
Attrition2352
Were outcome data incomplete due to attrition or exclusion from analysis?4163
Attrition rates are required to be similar and uniformly low across groups with respect to withdrawal or exclusion from analysis
Detection2353
Can we be confident in the exposure characterization?4164
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?4165
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups.
Complete methods in protocol with some notes below:
- Definitely low risk of bias criteria note: Training on all AGD measurements for all examiners using well-described methods (preferred is calipers), intra-rater, inter-rater reliability assessed, testosterone measured by HPLC, GC-MS, LC-MS or equilibrium dialysis
- Probably low risk of bias criteria note: Description of training for AGD measurement methods with appropriate citation. Testosterone assay by RIA (not as reliable as methods listed in definitely low risk of bias but ok overall)
- Probably high risk of bias criteria note: Indirect evidence that the outcome assessment method is using an insensitive instrument or AGD assessment without intra-rater or inter-rater reliability
- Definitely high risk of bias criteria note: No description of AGD measurement methods or no description of testosterone laboratory assessment
Selective Reporting2354
Were all measured outcomes reported?4166
Other2355
Were there any other potential threats to internal validity?4167
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include appropriateness of statistical methods, adherence to the study-protocol, etc.
Selection2349
Did selection of study participants result in appropriate comparison groups?4159
Comparison group appropriateness refers to having similar baseline characteristics between the groups aside from the exposures and outcomes under study.
Was allocation to study groups adequately concealed?4158
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Was administered dose or exposure level adequately randomized?4157
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Confounding2350
Did the study design or analysis account for important confounding and modifying variables?4160
Note: The following variables should be considered as key/primary and/or effect modifiers in the analysis of the relationship between phthalates and male AGD outcomes: weight/body size at exam, measure of weight or body size at birth, age at exam, and measure of urinary dilution or indication that measure was adjusted for urinary dilution.
The following variables should be considered as additional potential confounders and/or effect modifiers but consideration is not required: maternal age, pre-pregnancy or maternal BMI, maternal education, maternal income, maternal race/ethnicity, and time of day of urine collect.
Performance2351
Were experimental conditions identical across study groups?4161
Were the research personnel and human subjects blinded to the study group during the study?4162
Blinding requires that study scientists do not know which administered dose or exposure level the human subject or animal is being given (i.e., study group). Human studies also require blinding of the human subjects when possible.
Attrition2352
Were outcome data incomplete due to attrition or exclusion from analysis?4163
Attrition rates are required to be similar and uniformly low across groups with respect to withdrawal or exclusion from analysis
Detection2353
Can we be confident in the exposure characterization?4164
Confidence requires valid, reliable, and sensitive methods to measure exposure applied consistently across groups.
Can we be confident in the outcome assessment?4165
Confidence requires valid, reliable, and sensitive methods to assess the outcome and the methods should be applied consistently across groups.
Complete methods in protocol with some notes below:
- Definitely low risk of bias criteria note: Training on all AGD measurements for all examiners using well-described methods (preferred is calipers), intra-rater, inter-rater reliability assessed, testosterone measured by HPLC, GC-MS, LC-MS or equilibrium dialysis
- Probably low risk of bias criteria note: Description of training for AGD measurement methods with appropriate citation. Testosterone assay by RIA (not as reliable as methods listed in definitely low risk of bias but ok overall)
- Probably high risk of bias criteria note: Indirect evidence that the outcome assessment method is using an insensitive instrument or AGD assessment without intra-rater or inter-rater reliability
- Definitely high risk of bias criteria note: No description of AGD measurement methods or no description of testosterone laboratory assessment
Selective Reporting2354
Were all measured outcomes reported?4166
Other2355
Were there any other potential threats to internal validity?4167
On a project specific basis, additional questions for other potential threats to internal validity can be added and applied to study designs as appropriate.
Examples may include appropriateness of statistical methods, adherence to the study-protocol, etc.