Sajid (2013) concluded that there were significant limitations in the methodological quality of few included RCTs analysed in this review. Methodological limitations included not adequately (reporting of) blinding of the trial participants and outcome assessors, lack of intention-to-treat analysis and potential Risk of Bias in industry-sponsored trials.

Two mistakes in the review by Sajid (2013) were detected. Sajid included the mesh of 55 g/m2 in the study by Bittner (2011)* as a LWM. Moreover, chronic pain in the study of Bringman (2005) was assessed although follow-up was only 2 months. For current analysis both mistakes were adjusted for and the mesh and study were not included in the meta-analyses.

Burgmans(2015 and 2016)/ Roos (2018) conducted a prospective double-blinded RCT in 949 patients. This is the largest sample size in the field up to date. They included adult, male patients with a primary, reducible, unilateral inguinal hernia and no contraindications for TEP repair. All patients underwent TEP repair without the use of any fixation, sealant, adjunct, or glue. Outcomes of chronic pain were assessed up to 2 years and recurrences up to 5 years postoperatively.

Risk of Bias in this study was low.

Prakash (2016) randomised 140 adult patients with uncomplicated inguinal hernia into a HW mesh group or LW mesh group. All the patients underwent LIHR by either TAPP or TEP method. No mesh fixation was used in either group. A total of 131 patients completed a minimum of 3 months follow-up period, 66 in HW mesh group and 65 in LW mesh group. Outcomes were assessed up to 12 months postoperatively.

Risk of Bias in this study was low, the only potential Risk of Bias was due to inadequate blinding of care providers and outcome assessors to treatment allocation.

Kalra (2017) conducted a multi-center RCT comparing TAPP repair with placement of either a LWM or a HWM. Tackers were used for the fixation of the mesh. Patients were 15 to 60 years, with a unilateral, reducible inguinal hernia. Sixty patients were included with a follow-up of 3 months.

Risk of Bias in this study was high due to underreporting. Any form of blinding and loss to follow-up was not reported.

Wong (2017) performed a double-blind RCT in 85 patients with a mean follow-up of 20.3 months (range 12 to 34 months). All patients underwent TEP repair. No device or agent was used to anchor the mesh. Inclusion criteria were age 18 to 81 years and a primary uncomplicated unilateral or bilateral inguinal hernia. The recurrence rate was assessed.

Risk of Bias in this study was low, only the randomization technique was not described.

Results

1. Chronic pain

The meta-analysis from Sajid (2013) could be updated with data from Burgmans (2016). Prakash (2016) and Kalra (2017) presented chronic pain outcome as the mean VAS ± SD. We were unable to include these studies in the meta-analysis. The forest plot is shown in figure 1. Overall, the risk of chronic groin pain was estimated to be reduced by 34% when using a LWM compared with a HWM (RR 0.66, 95%CI: 0.38 to 1.16), but the confidence interval was wide and enclosed 1, indicating a not statistically significant and imprecise effect estimate.

There was substantial heterogeneity, possibly due to variation in the duration of follow-up (chi-square = 13.66, df = 6, P = 0.03, I² = 57%) among the RCTs.

The meta-analysis from Sajid 2013 could be updated with data from Prakash 2016; Wong, 2017 and Roos, 2018. The pooled analysis showed that the risk of recurrences was increased by a factor 2.83 when a LWM was used compared to a HWM (RR 2.83, 95%CI 1.41 to 5.70).

The study by Burgmans (2016) and Roos (2018) showed significantly more recurrences for LWM after direct hernia repair (LWM n=12, HWM n=1, p=0.003), but no significance for recurrences after indirect hernia repair (LWM n=6, HWM n=4, p=0.545). The study by Wong (2017) does not report on the matter for the 2 recurrences. Prakash 2016 showed 1 recurrence in the LWM-group after an indirect inguinal hernia repair. Bittner (2011)* reported all 3 recurrences (2 LWM, 1 HWM) after direct hernia repair. Bittner 2011 found 1 recurrence after direct hernia repair. Chowbey 2010 found out of five recurrences in the LWM-group, 3 patients had large indirect hernias and 2 patients had large direct hernias. Langenbach (2008) found all 3 recurrences were after direct hernia repair. Champault (2007) does not report on the matter for 2 recurrences.

Quality of the evidence

Evidence originated from RCTs and the level of the quality of the evidence comparing LWM versus HWM started therefore at ‘High’. However, the quality of the evidence regarding chronic pain was downgraded with one level to ‘Moderate’ for imprecision (the confidence interval crosses 1). We did not downgrade further for limitations in the methodological quality of the included RCTs (Risk of Bias).

The quality of the evidence regarding recurrence was not downgraded. The lower boundary of the confidence interval was considered clinically relevant.

Description of the studies on Lichtenstein inguinal hernia repair

Sajid (2012) performed a systematic review with a meta-analysis of the literature on lightweight mesh (LWM) compared with heavyweight mesh (HWM) in open inguinal hernia repair. Relevant databases were searched till May 2011. Any trials that compared LMW with HWM were included. A total of 9 RCT were found and described (Bringman, 2006; Champault, 2007; Koch, 2008; Nikkolo, 2010; O’Dwyer, 2005; Paajanen, 2007; Post, 2004; Smietanski, 2008); Smietanski, 2011; Torcivia, 2011) encompassing 1156 patients with a LWM and 1154 patients with a HWM. In all trials the HWM consisted of polypropylene. In all trials all patients underwent a Lichtenstein technique. All trials scored highly enough to suggest good quality of the included trials.

Sajid (2012( concluded that the duration of operation, postoperative pain, postoperative complications, hernia recurrence rate, risk of testicular atrophy and time to return to work were comparable between LWM and HWM. LWM was associated with a reduced risk of developing chronic groin pain and other groin symptoms.

For the current analysis the short-term results published by Paajanen (2007), Nikkolo (2010) and Smietanski (2008) were replaced with the long-term results published by Paajanen (2012), Nikkolo (2012) and Bury (2012), respectively.

A mistake in the review by Sajid was detected. The review wrongly presented Smietanski (2011) as 5 year follow-up of Smietanski (2008). Smietanski (2011) is an original study, with no previously published short-term results, that investigates a different LWM compared to Smietanski (2008). Bury (2012) is the correct 5 year follow-up of Smietanski (2008). In our analysis we adjusted for this mistake.

Apart from the long-term follow-up studies, in 9 more studies after the latest review and meta-analysis by Sajid another 978 patients with a LWM and 964 patients with a HWM were studied. In all studies the groups were comparable at baseline. In all studies patients were aged over 18 years and most studies included primary unilateral inguinal hernias. Only Pielaciński (2013) and Bona (2018) included bilateral inguinal hernias. Pielaciński (2013) was the only study that included recurrent hernias. In all studies a Lichtenstein technique was performed. Most studies used sutures for fixation of the mesh. In four studies (Paradowski, 2009; Sadowski, 2011, Rutegård, 2017), Bona 92018) mesh fixation was not reported. In the study by Canonico (2012) human fibrin glue was used for fixation of the mesh. Follow-up of the studies ranged from 3 to 60 months. Canonico (2012) and Rutegård (2017) were the only two studies that did not report chronic pain as an outcome. Sadowski (2011) was the only study that did not have recurrence as an outcome.

Risk of Bias in half of the studies was low (see: Risk of Bias table). Risk of Bias was high in the following studies: Sadowski (2011), Canonico (2012) and Pielaciński (2013) did not report on the randomization technique and concealment of allocation, and bias due to loss to follow-up was likely or unclear. Moreover in the study of Sadowski (2011) and Pielaciński (2013) bias due to blinding was likely or unclear and Canonico (2012) performed selective outcome reporting because the 6 months results were not described. In the study of Nikkolo (2012) blinding was not reported and loss to follow-up was high. Lee (2017) did not report on the randomization technique, concealment of allocation and blinding of outcome assessors. Bona (2018) did not blind the outcome assessors, loss to follow-up was high and bias due to selective outcome reporting was likely because the 12 months results were not described.

Therefore, 15 trials (n=3394) reported applicable data for meta-analysis on the outcome chronic groin pain. In the LWM-group 234 of 1669 patients reported chronic pain, compared to 325 of 1725 patients in the HWM-group. The risk of having chronic groin pain after a hernioplasty with a LWM was estimated to be 23% lower than with a HWM. The pooled risk ratio estimate was 0.77 (95%CI: 0.61 to 0.96). Inapplicable for meta-analysis, Lee 2017 reported chronic pain as a mean VAS (SD) of 0.7 (1.1) for the LWM and 0.8 (1.4) for the HWM.

Seventeen trials (n=3953) reported data on the outcome recurrences. In the combined results 40 patients who received a LWM had a recurrence, while 27 patients with a HWM had a recurrence. The risk for recurrence was therefore increased with the use of LWM with a pooled risk ratio estimate of 1.48 (95%CI: 0.86 to 2.52).

Evidence originated from RCTs and the level of the quality of the evidence comparing LWM versus HWM started therefore at ‘High’. However, the quality of the evidence regarding chronic pain was downgraded with one level to ‘Moderate’ for imprecision, as the upper boundary of the confidence is very close to 1 and we therefore cannot rule out that the difference might not be clinically relevant.

The quality of the evidence regarding recurrence was downgraded with one level to ‘Moderate’ for imprecision (the confidence interval crosses 1). We did not downgrade further for limitations in the methodological quality of the included RCTs (Risk of Bias).

Description of the SR on Lichtenstein inguinal hernia repair with ProGrip

The 10 RCTs that were included in the SR of Molegraaf 2018 described 2,541 patients (n = 1,216 self-gripping mesh group, n = 1,245 sutured mesh group). The duration of follow-up ranged from 6 to 72 months.

The quality assessment of the studies by Molegraaf 2018 showed that the quality of 2 trials was poor due the absence of an adequate randomization technique or no information about it, absence of blinding, no power calculations, and no baseline score. The other trials were of moderate or good quality, despite the common absence of blinding and poor reporting of the definition or assessment method of chronic pain. In four studies a baseline pain score was lacking, although preoperative pain is a well-known risk factor for chronic pain. Furthermore, some trials compared different types of meshes in the 2 study groups instead of only changing the method of mesh fixation (polypropylene and polyester, and heavy and low weight).

Results

Chronic pain (critical outcome)

Chronic pain was assessed in all trials and 9 of them reported the incidence of chronic pain according to the definition used in their study protocol. Incidence rates were analysed separately for the different moments of follow-up (3, 6 to 12, 24, 36, and 72 months).

At all follow-up time points, there was no significant difference in the incidence of CPIP between the self-gripping mesh and sutured mesh group (3 months OR = 0.89, 95% CI 0.48 to 1.64 (n=425); 6 to 12 months OR = 1.00; 95% CI 0.75 to 1.34 (n=1517); 24 months OR = 1.00; 95% CI 0.39 to 2.61 (n=372); 36 to 72 months OR = 0.77; 95% CI 0.38 to 1.58 (n=464)).

Evidence comparing open inguinal hernia repair with a self-gripping ProGrip mesh and a conventional Lichtenstein hernioplasty originated from RCTs and the level of the quality of the evidence regarding chronic pain and recurrence after inguinal hernia repair with self-gripping meshes started therefore at ‘High’. However, the quality of the evidence was downgraded one level for substantial methodological limitations of the studies to ‘Moderate’. We did not downgrade for the aforementioned clinical heterogeneity between studies, because the subgroup analyses accounting for mesh weight and including only studies that used a light weighted mesh in both the study and control group also showed no difference in CPIP rates between the self-gripping mesh and sutured mesh (Molegraaf, 2018).

The working group decided that chronic pain and recurrences were crucial outcome measures for decision-making. Recurrences after inguinal hernia repair have dropped dramatically with the introduction of tension-free mesh repair and endoscopic preperitoneal approaches. Still, recurrences and chronic pain are the most common long-term complications after inguinal hernia repair. According to the International Association for the Study of Pain, chronic pain is defined as (inguinal) pain lasting a minimum of 3 months. Any level of pain was considered relevant. For the detection of recurrences any time-period was considered relevant for comparison. A distinction between recurrences after direct hernia repairs or indirect hernia repairs was sought for in included endoscopic repair studies.

Searching and selecting (Methods)

A literature search, for open and for endoscopic inguinal hernia repair, was conducted in MEDLINE, EMBASE and the Cochrane Library at June 6th 2018. The search was not limited by publication date or language. The search details can be found in the tab Acknowledgement. Subsequently, a filter for identifying RCTs was used to filter out nonrandomized trials. All duplicates were removed. Literature experts excluded non-relevant studies based on title and abstract and/or full-text screening.

For the comparison of lightweight versus heavyweight meshes after endoscopic inguinal hernia repair the working group selected 19 studies based on title-abstract that could possibly answer the research questions. After reading full text, 2 studies were excluded (see exclusion table in the tab Acknowledgement). Finally, the studies from the latest review and meta-analysis for endoscopic mesh comparison by Sajid 2013 and 6 additional original studies that were not included in Sajid (2013) were included for analysis (Burgmans, 2015; Burgmans, 2016; Prakash, 2016; Kalra, 2017; Wong, 2017 and Roos, 2018).

For the comparison of lightweight versus heavyweight meshes after open inguinal hernia repair the working group selected 25 studies based on title-abstract, that could possibly answer the research questions. After reading full text, 2 studies were excluded (see exclusion table in the tab Acknowledgement). Finally, the studies from the latest review and meta-analysis for open mesh comparison by Sajid (2012) and 12 additional original studies that were not included in Sajid 2012 were included for analysis (Paradowski, 2009; Sadowski, 2011; Bury, 2012; Nikkolo, 2012; Canonico, 2013; Paajanen, 2013; Pielaciński, 2013; Yazdankhah Kenary, 2013; Demetrashvili, 2014; Lee, 2017; Rutegård, 2017; Bona, 2018).

The 2 SRs with meta-analyses (Sajid, 2012 and Sajid, 2013), containing 20 RCTs were included in the literature analysis. These SRs were updated with the recent RCTS.

During the preparation and writing of the guideline text of this module, a relevant systematic review was published that answered PICO 3 (Molegraaf, 2018). Molegraaf (2018) performed a systematic review of the literature to identify RCTs comparing open inguinal hernia repair with a self-gripping ProGrip mesh and a conventional Lichtenstein hernioplasty. The working group decided that this SR and meta-analysis provided the most up-to-date overview of RCTs regarding self-gripping (ProGrip) mesh.

Data extraction and analysis

The most important study characteristics and results were extracted from the SRs or original studies (also, in case of missing information in the review). The most important study characteristics and relevant results are shown in the evidence tables. The judgement of the individual study quality (Risk of Bias) is shown in the Risk of Bias tables.

It was agreed that the blinding of the care providers (operating surgeon) was impossible. The Risk of Bias however was rated as low, since the type of mesh used is an unlikely influence on the success of surgery execution. In the case of not reported intention-to-treat analysis it was considered an unlikely Risk of Bias due to the inability to cross-over. The lack of an adequate randomization technique (or not reported), inadequate blinding (or not reported) and a high or non-proportionally divided loss to follow-up was considered a high Risk of Bias.

The summary of included studies, study characteristics and quality of the SRs by Sajid are presented in the evidence table for SRs and the quality assessment table for SRs.

Relevant pooled and/or standardised effect measures were, if useful, calculated using Review Manager 5.3 (Cochrane Collaboration, Oxford, United Kingdom). If pooling results was not possible, the outcomes and results of the original study were used as reported by the authors.

The working group did not define clinical (patient) relevant differences for the outcome measures. Therefore, we used the following boundaries for clinical relevance, if applicable: for continue outcome measures: RR <0.75 or >1.25 (GRADE recommendation) or Standardized mean difference (SMD=0.2 (little); SMD 0.5 (reasonable); SMD=0.8 (large). These boundaries were compared with the results of our analysis. The interpretation of dichotomous outcome measures is strongly related to context; therefore, no clinical relevant boundaries were set beforehand. For dichotomous outcome measures, the absolute effect was calculated (Number Needed to Treat (NNT) or Number Needed to Harm (NNH)).

Research question: Which mesh is recommended for endoscopic and open inguinal hernia repair?

Study reference

Study characteristics

Patient characteristics

Intervention (I)

Comparison / control (C)

Follow-up

Outcome measures and effect size

Comments

Open inguinal hernia repair

Sajid 2012

SR and meta-analysis of RCTs

Literature search up to May 2011

A: Bringman (2006)

B: Champault (2007)

C: Koch (2008)

D: Nikkolo (2010)

E: O’Dwyer (2005)

F: Paajanen (2007)

G: Post (2004)

H: Smietanski (2008)

I: Smietanski (2011)

J: Torcivia (2011)

Study design:

All RCTs

Setting and Country:

A: Sweden and Finland

B: France

C: Sweden

D: Estonia

E: UK and Germany

F: Finland

G: Germany

H: Poland

I: Poland

J: France

Source of funding:

Not reported for the included trials or review

Inclusion criteria SR:

RCT

Comparison of LWM* vs HWM

Open inguinal hernia repair

10 studies included

Important patient characteristics at baseline:

N, age (y)

A: 494 patients LWM: 55

HWM: 55

B: 232 patients, 54

C: 317 patients

LWM: 56

HWM: 57

D: 135 patients

LWM: 59

HWM: 57

E: 321 patients

LWM: 55

HWM: 57

F: 233 patients

LWM: 56

HWM: 59

G: 108 patients

LWM: 60

HWM: 62

H: 392 patients

LWM: 56

HWM: 56

I: 182 patients

LWM: 55

HWM: 58

J: 47 patients

LWM: 54

HWM: 54

Sex:

A: All male

B: Mixed

C: All male

D: Mixed

E: Mixed

F: Mixed

G: Mixed

H: Mixed

I: Mixed

J: Mixed

Open inguinal hernia repair + Lightweight mesh (LWM)

LWM was defined as surgical mesh with a tensile strength of 16 N/cm, elasticity of 20–35 per cent at a tensile strength of 16 N/cm, pore size more than 1 mm, and containing woven lightweight polymers of biomaterial usually weighing less than 50 g/m2.

Open inguinal hernia repair + Heavyweight mesh (HWM)

In all trials a polypropylene mesh was used.

End-point of follow-up:

A: 37 months

B: 24 months

C: 12 months

D: 6 months

E: 12 months

F: 24 months

G: 6 months

H: 12 months

I: 60 months

J: 30 days

For how many participants were no complete outcome data available?

Not reported

Outcome measure-1

Defined as chronic pain, measured as groin pain

Effect measure: RR, [95% CI]:

A: 0.76 (0.56-1.05)

B: 0.15 (0.04-0.58)

C: 0.57 (0.19-1.65)

D: 0.57 (0.27-1.22)

E: 0.65 (0.49-0.87)

F: 3.02 (0.37-24.64)

G: 0.16 (0.02-1.32)

H: Not reported, due to mistake (see: comments)

I: 0.98 (0.14-6.80)

J: 0.38 (0.18-0.82)

Pooled effect (fixed effects model):

RR 0.61 [95% CI 0.50 to 0.74] favoring LWM.

Heterogeneity (I2): 31%

Outcome measure-2

Defined as recurrence

Effect measure: RR, [95% CI]:

A: 4.36 (0.95-19.96)

B: 1.13 (0.23-5.41)

C: 2.06 (0.19-22.53)

D: Not estimable

E: 7.85 (0.99-62.06)

F: 0.75 (0.13-4.42)

G: 0.83 (0.12-5.66)

H: Not reported, due to mistake (see: comments)

I: 0.33 (0.03-3.08)

Pooled effect (fixed effects model):

RR 1.82 [95% CI 0.97 to 3.42] favoring HWM.

Heterogeneity (I2): 19%

Mistake in review detected:

Smietanski (2011) is not 5 year follow-up of Smietanski (2008) as wrongly interpreted by Sajid. Smietanski (2011) is original, not previously published, data with a different LWM.

An assessment of publication bias should include a combination of graphical aids (For example funnel plot, other available tests) and/or statistical tests (For example Egger regression test, Hedges-Olken). Note: If no test values or funnel plot included, score “no”. Score “yes” if mentions that publication bias could not be assessed because there were fewer than 10 included studies.

Sources of support (including commercial co-authorship) should be reported in both the systematic review and the included studies. Note: To get a “yes,” source of funding or support must be indicated for the systematic review AND for each of the included studies.

Research question: Which mesh is recommended for endoscopic inguinal hernia repair?

Study reference

(first author, publication year)

Describe method of randomisation1

Bias due to inadequate concealment of allocation?2

(unlikely/likely/unclear)

Bias due to inadequate blinding of participants to treatment allocation?3

(unlikely/likely/unclear)

Bias due to inadequate blinding of care providers to treatment allocation?3

(unlikely/likely/unclear)

Bias due to inadequate blinding of outcome assessors to treatment allocation?3

(unlikely/likely/unclear)

Bias due to selective outcome reporting on basis of the results?4

(unlikely/likely/unclear)

Bias due to loss to follow-up?5

(unlikely/likely/unclear)

Bias due to violation of

intention to treat analysis?6

(unlikely/likely/unclear)

Burgmans 2015

Computer generated.

Unlikely

Unlikely; patients were blinded for allocation of mesh type.

Likely; blinding of care providers was not possible.

Unlikely; researchers were blinded for allocation of mesh type.

Unlikely

Unlikely; 3.2%/3.6% in 3 months of follow-up.

Unlikely; all data were analysed on an intention-to-treat basis.

Burgmans 2016

Computer generated.

Unlikely

Unlikely; patients were blinded for allocation of mesh type.

Likely; blinding of care providers was not possible.

Unlikely; researchers were blinded for allocation of mesh type.

Unlikely

Unlikely; 7.9%/9.3% in 2 years of follow-up.

Unlikely; all data were analysed on an intention-to-treat basis.

Prakash 2016

Computer generated.

Unlikely

Unlikely; patients were blinded for allocation of mesh type.

Likely; blinding of care providers was not possible.

Likely; single-blinded study only for patients.

Unlikely

Unlikely; 7.1%/5.7% in 1 year of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Kalra 2017

Sealed envelope system.

Unlikely

Unclear; not reported.

Likely; blinding of care providers was not possible.

Unclear; not reported.

Unlikely

Unclear; not reported.

Unlikely; not reported, however unlikely due to inability to cross-over.

Wong 2017

Not reported.

Unlikely

Unclear; double-blinded, yet no further report on the method used.

Likely; blinding of care providers was not possible.

Unlikely; surgeons assigned to do postoperative assessment were blind to the type of mesh used.

Unlikely

Likely; 9.3%/9.5% in 20 months of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Roos 2018

Computer generated.

Unlikely

Unlikely: Patients were blinded for allocation of mesh type.

Likely; blinding of care providers was not possible.

Unlikely; Telephonic follow-up was performed by 5 independent researchers who were blinded for allocation of mesh type.

Unlikely

Likely; 15.9%/17.6% in 5 years of follow-up.

Unlikely; all data were analysed on an intention-to-treat basis.

Randomisation: generation of allocation sequences have to be unpredictable, for example computer generated random-numbers or drawing lots or envelopes. Examples of inadequate procedures are generation of allocation sequences by alternation, according to case record number, date of birth or date of admission.

Allocation concealment: refers to the protection (blinding) of the randomisation process. Concealment of allocation sequences is adequate if patients and enrolling investigators cannot foresee assignment, for example central randomisation (performed at a site remote from trial location) or sequentially numbered, sealed, opaque envelopes. Inadequate procedures are all procedures based on inadequate randomisation procedures or open allocation schedules.

Blinding: neither the patient nor the care provider (attending physician) knows which patient is getting the special treatment. Blinding is sometimes impossible, for example when comparing surgical with non-surgical treatments. The outcome assessor records the study results. Blinding of those assessing outcomes prevents that the knowledge of patient assignement influences the proces of outcome assessment (detection or information bias). If a study has hard (objective) outcome measures, like death, blinding of outcome assessment is not necessary. If a study has “soft” (subjective) outcome measures, like the assessment of an X-ray, blinding of outcome assessment is necessary.

Results of all predefined outcome measures should be reported; if the protocol is available, then outcomes in the protocol and published report can be compared; if not, then outcomes listed in the methods section of an article can be compared with those whose results are reported.

If the percentage of patients lost to follow-up is large, or differs between treatment groups, or the reasons for loss to follow-up differ between treatment groups, bias is likely. If the number of patients lost to follow-up, or the reasons why, are not reported, the risk of bias is unclear.

Participants included in the analysis are exactly those who were randomized into the trial. If the numbers randomized into each intervention group are not clearly reported, the risk of bias is unclear; an ITT analysis implies that (a) participants are kept in the intervention groups to which they were randomized, regardless of the intervention they actually received, (b) outcome data are measured on all participants, and (c) all randomized participants are included in the analysis.

Research question: Which mesh is recommended for open inguinal hernia repair?

Study reference

(first author, publication year)

Describe method of randomisation1

Bias due to inadequate concealment of allocation?2

(unlikely/likely/unclear)

Bias due to inadequate blinding of participants to treatment allocation?3

(unlikely/likely/unclear)

Bias due to inadequate blinding of care providers to treatment allocation?3

(unlikely/likely/unclear)

Bias due to inadequate blinding of outcome assessors to treatment allocation?3

(unlikely/likely/unclear)

Bias due to selective outcome reporting on basis of the results?4

(unlikely/likely/unclear)

Bias due to loss to follow-up?5

(unlikely/likely/unclear)

Bias due to violation of

intention to treat analysis?6

(unlikely/likely/unclear)

Paradowski 2009

Not reported; during operation after preparation of the hernia sac.

Unclear.

Unlikely; patients were blinded to the mesh they received.

Likely; blinding of care providers was not possible.

Unlikely; examiners of follow-up were blinded to the received mesh.

Unlikely.

Unlikely; 4% in 1 year of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Sadowski 2011

Not reported.

Unclear.

Unlikely; patients were blinded to the mesh they received and remained blinded throughout the follow-up period.

Likely; blinding of care providers was not possible.

Likely; not reported, although it states single-blinded for patients.

Unlikely.

Likely; 10.3% in 3 months of follow-up.

Unlikely; ITT analysis.

Bury 2012

Wichmann-Hill pseudorandom number generator (modified by McLeod).

Unlikely.

Unclear; not reported.

Likely; blinding of care providers was not possible.

Unlikely; surgeons were blinded for allocation of mesh type for follow-up visits.

Unlikely.

Likely; 9.3%/9.0% in 5 years of follow-up.

Unlikely; ITT analysis.

Nikkolo 2012

Blind envelope system.

Unlikely.

Unclear; not reported.

Likely; blinding of care providers was not possible.

Unclear; not reported.

Unlikely.

Likely; 15.9%/12.1% in 3 years of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Canonico 2012

Not reported.

Unclear.

Unlikely; patients were blinded to the mesh they received.

Likely; blinding of care providers was not possible.

Unlikely; observers of outcomes were blinded to the received mesh.

Likely; 6 months follow-up results not described.

Unclear; not reported.

Unlikely; not reported, however unlikely due to inability to cross-over.

Paajanen 2012

Sealed and numbered envelopes.

Unlikely.

Unlikely; patients were unaware of the treatment allocation.

Likely; blinding of care providers was not possible.

Unlikely; dedicated research nurse blinded to the type of mesh.

Unlikely.

Likely; 21.2%/26.7% in 5 years of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Pielaciński 2013

Not reported.

Unclear.

Unclear; not reported.

Likely; blinding of care providers was not possible.

Unclear; not reported.

Unlikely.

Likely; 23%/29% in 6 months of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Yazdankhah Kenary 2013

Computer generated random numbers.

Unlikely.

Unlikely; patients were blinded to the mesh they received.

Likely; blinding of care providers was not possible.

Unlikely; examiners of follow-up were blinded to the received mesh.

Unlikely.

Unlikely; no loss to follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Demetrashvili 2014

Simple random sampling.

Unlikely.

Unlikely; patients were blinded to the mesh they received.

Likely; blinding of care providers was not possible.

Unlikely; the examinations were performed by surgeons who had not been participating previously in the study.

Unlikely.

Likely; 15%/9.7% in 3 years of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Lee 2017

Not reported.

Unclear.

Unlikely; patients were blinded to the mesh they received.

Likely; blinding of care providers was not possible.

Unclear; not reported.

Unlikely.

Unlikely; 4%/8% in 4 months of follow-up.

Unlikely; per protocol analysis (PP). The last observation carried forward method was

used to handle missing values in the PP population.

Rutegård 2017

Concealed allocation sequence was computer generated by statistician with a random number technique and stratified at the hospital level.

Unlikely.

Likely; surgeons were divided into two groups according to mesh preference. Patients were not blinded to the group of surgeons that performed the operation.

Likely; blinding of care providers was not possible.

Unlikely; examiners of follow-up were blinded to the received mesh.

Unlikely.

Unlikely.

4.6%/9.6% in 1 year of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Bona 2018

Centralized. Balanced every 10 patients within centre, using for this purpose the algorithm of Pocock and Simon.

Unlikely.

Unlikely; single-blind.

Likely; blinding of care providers was not possible.

Likely; mesh type was mentioned in operative report.

Likely; 12 months follow-up results not described.

Likely; 20,9% and 20.2% in 1 year of follow-up.

Unlikely; not reported, however unlikely due to inability to cross-over.

Randomisation: generation of allocation sequences have to be unpredictable, for example computer generated random-numbers or drawing lots or envelopes. Examples of inadequate procedures are generation of allocation sequences by alternation, according to case record number, date of birth or date of admission.

Allocation concealment: refers to the protection (blinding) of the randomisation process. Concealment of allocation sequences is adequate if patients and enrolling investigators cannot foresee assignment, for example central randomisation (performed at a site remote from trial location) or sequentially numbered, sealed, opaque envelopes. Inadequate procedures are all procedures based on inadequate randomisation procedures or open allocation schedules.

Blinding: neither the patient nor the care provider (attending physician) knows which patient is getting the special treatment. Blinding is sometimes impossible, for example when comparing surgical with non-surgical treatments. The outcome assessor records the study results. Blinding of those assessing outcomes prevents that the knowledge of patient assignement influences the proces of outcome assessment (detection or information bias). If a study has hard (objective) outcome measures, like death, blinding of outcome assessment is not necessary. If a study has “soft” (subjective) outcome measures, like the assessment of an X-ray, blinding of outcome assessment is necessary.

Results of all predefined outcome measures should be reported; if the protocol is available, then outcomes in the protocol and published report can be compared; if not, then outcomes listed in the methods section of an article can be compared with those whose results are reported.

If the percentage of patients lost to follow-up is large, or differs between treatment groups, or the reasons for loss to follow-up differ between treatment groups, bias is likely. If the number of patients lost to follow-up, or the reasons why, are not reported, the risk of bias is unclear.

Participants included in the analysis are exactly those who were randomized into the trial. If the numbers randomized into each intervention group are not clearly reported, the risk of bias is unclear; an ITT analysis implies that (a) participants are kept in the intervention groups to which they were randomized, regardless of the intervention they actually received, (b) outcome data are measured on all participants, and (c) all randomized participants are included in the analysis.