BACKGROUND AND AIMS: Systematic reviews (SRs) are considered to provide reliable estimates, but flaws in designs, methods of monitoring effects, and outcomes have the potential to bias results. There are several tools for assessing risk of bias (RoB), most of them designed for SRs of beneficial effects. To our knowledge, there is no tool that is adapted specifically to assess RoB in studies of adverse effects associated with orthodontic treatment. To address this, the aim of this study was first to introduce a tool for assessment of RoB in studies of adverse effects associated with orthodontic treatment and, second, to apply it in an SR of external root resorption (ERR) associated with orthodontic treatment with fixed appliance.

MATERIALS AND METHODS: The approach with domains supported by signalling questions was used for the tool. Domains and signalling questions were tailored to the review questions of the SR of studies of ERR after orthodontic treatment using periapical radiography or cone beam computed tomography. Duplicate study selection, data extraction, and RoB assessment using the tool, followed by meta-analyses, were performed.

RESULTS: Using the tool for the assessment of RoB identified shortcomings and report deficiencies of primary studies concerning the presentation of orthodontic treatment, identification of ERR, and analysis of outcomes. RoB assessment resulted in 12 of 32 studies read in full text being included. Reported severe ERR varied across studies between 2 and 14 per cent for all incisors and 10 and 29 per cent for maxillary incisors. Results of ERR related to patients' age and sex, orthodontic diagnosis, and treatment were contradictory. Quality of evidence evaluated by GRADE was low due to study limitations, imprecision, and inconsistency of study results.

CONCLUSIONS: As the tool and its application highlight important issues to consider when planning, conducting, and reporting research, the tool may have a valuable role for quality enhancement of future studies of outcomes of orthodontic treatment. The tool may also serve for authors when planning SRs. Our SR identified a need for studies that use rigorous methodology and transparent reporting.

REGISTRATION: PROSPERO (ID = CRD42018084725).