812 844
1.
Introduction
The practice of evidence-based medicine means integrating
individual clinical expertise with the best available external
clinical evidence from systematic research
[1] .Treatment recommendations in European Association of
Urology (EAU) guidelines are underpinned, whenever
possible, by the results of systematic reviews (SRs)/meta-
analyses (MAs) and large randomized controlled trials
(RCTs). According to the 2009 Oxford Centre for Evidence-
based Medicine, SRs of RCTs (with or without a meta-
analysis) that are free of worrisome variations (heteroge-
neity) in results between individual studies provide the
highest level of evidence (LE), 1a, whereas individual RCTs
with a narrow confidence interval provide the next highest
LE, 1b
[2]. As SRs can provide a higher LE than RCTs, the
results of SRs are generally considered to take precedence
when developing treatment recommendations.
The quality of the results of an SR/MA depends on the
quality of the studies included. Kjaergard et al
[3]found a
correlation between methodologic quality and discrepan-
cies in the results of large and small RCTs included in MAs.
Intervention effects were exaggerated in small trials with
inadequate allocation sequence generation, inadequate
allocation concealment, and no double blinding.
Discrepancies have also been noted between large RCTs
and previously published MAs on the same subject
[4– 6] .For 12 large RCTs carried out subsequent to 19 MAs
addressing the same question, LeLorier et al
[7]found that
the results of subsequent RCTs
[1_TD$DIFF]
disagreed with those of
earlier MAs 35% of the time.
To illustrate these points and provide guidance to
guideline developers in dealing with conflicting data from
different sources, two examples that have a direct bearing
on EAU Guidelines treatment recommendations are pre-
sented. In the first example, the EAU Guidelines Office has
recently been confronted with the results of a large RCT that
found no beneficial effect of medical expulsive therapy
(MET) on stone passage, contrary to results of previous
meta-analyses that formed the basis for treatment recom-
mendations
[8] .In the second example, which compares the
efficacy of partial and radical nephrectomy for localized
renal tumors, discordance between the results of the meta-
analysis and the only available RCT are investigated
[9,10].
2.
Advantages and limitations of RCTs
As summarized in
Table 1, RCTs have a number of
advantages and limitations.
2.1.
Advantages of RCTs
RCTs are the gold standard for providing evidence on the
effectiveness of interventions
[11,12]. Randomization bal-
ances, on average, the distribution of both known and
unknown prognostic factors at baseline in the intervention
groups, thereby minimizing selection bias when assigning
patients to treatments. Although adjusting for baseline
covariates used in the randomization process can improve
statistical power, complex adjustment procedures such as
propensity score weighting are not usually required when
comparing outcomes.
Patients are selected, treated, followed, and assessed
according to a common protocol testing a specific hypothe-
sis. Blinding of participants and physicians to the allocated
interventionmay be possible tominimize performance bias,
and is especially important when assessing outcomes
[13]. Quality control measures and external review of key
parameters maximize study quality.
2.2.
Limitations of RCTs
RCTs can be challenging to design (randomization and
blinding), conduct (poor recruitment, loss to follow-up),
analyze (missing data), and report (patient exclusions).
RCTs require an adequate sample size and follow-up to
have sufficient power to detect clinically relevant differ-
ences between interventions
[14] .In practice, many clinical
trials do not meet their prespecified power requirements, so
a conclusion of ‘‘no significant difference’’ in outcome
should not be interpreted as meaning that two or more
treatments are equivalent in effect. Sample size estimation
requires data about expected differences and variability of
the primary outcome. Often these data are unknown or only
available from observational studies prone to bias.
Although analyses using the intention-to-treat principle
can provide an unbiased estimate of the treatment effect,
this assumes that there are no differences in follow-up or
missing outcome data that may bias the treatment
comparison
[15]. In some RCTs, not all participants receive
their randomized intervention; they may, for example,
cross over to the other randomized treatment, in which case
a per-protocol analysis may also provide useful informa-
tion. Various analysis strategies exist, depending on
whether the objective is to estimate treatment efficacy
(the intervention effect under perfect conditions, in which
case intent to treat can dilute the size of the treatment
effect) or effectiveness (the real-world intervention effect
with ‘‘imperfect’’ compliance).
Table 1 – Advantages and limitations of randomized controlled trials (RCTs)
Advantages
Limitations
Randomization minimizes the influence of both known and
unknown prognostic variables on treatment outcome
It may be difficult to recruit and follow up patients
RCTs can demonstrate causality
Ethical considerations may make randomization difficult
Patients are treated according to a common protocol
Required study power might not be met
Quality control of treatment and outcome assessment
Generalizability may be low
RCTs provide the strongest empirical evidence of treatment efficacy
RCTs are expensive and resource-intensive
E U R O P E A N U R O L O G Y 7 1 ( 2 0 1 7 ) 8 1 1 – 8 1 9
812