Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered report describes the proposed replication plan of key experiments from ‘Inhibition of bromodomain and extra terminal (BET) recruitment to chromatin as an effective treatment for mixed-lineage leukemia (MLL)-fusion leukemia’ by Dawson and colleagues, published in Nature in 2011 (Dawson et al., 2011). The experiments to be replicated are those reported in Figures 2A, 3D, 4B, 4D and Supplementary Figures 11A-B and 16A. In this study, BET proteins were demonstrated as potential therapeutic targets for modulating aberrant gene expression programs associated with MLL-fusion leukemia. In Figure 2A, the BET bromodomain inhibitor I-BET151 was reported to suppress growth of cells harboring MLL-fusions compared to those with alternate oncogenic drivers. In Figure 3D, treatment of MLL-fusion leukemia cells with I-BET151 resulted in transcriptional suppression of the anti-apoptotic gene BCL2. Figures 4B and 4D tested the therapeutic efficacy of I-BET151 in vivo using mice injected with human MLL-fusion leukemia cells and evaluated disease progression following I-BET151 treatment. The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange and the results of the replications will be published in eLife.

Introduction

The mixed-lineage leukemia (MLL) gene encodes a large histone methyltransferase that directly binds DNA and positively regulates gene transcription (Marschalek, 2010). MLL is a frequent target of chromosomal translocation events (Meyer et al., 2009). During rearrangement, the N-terminus of MLL fuses to one of more than 60 partners, the most common of which coexist in a super elongation complex (SEC) enriched with transcription elongation factors (Meyer et al., 2009; Smith et al., 2011). The resulting fusion event converts MLL into a potent transcriptional activator often giving rise to aggressive hematological malignancies (Mueller et al., 2009; Slany, 2009). The overall prognosis for pediatric and adult patients with confirmed MLL-fusion leukemia remains extremely poor and necessitates the development of new methodologies and therapeutic agents to improve survival outcomes (Slany, 2009; Tamai and Inokuchi, 2010).

Bromodomain and extra terminal (BET) proteins are transcriptional regulators that epigenetically control the expression of genes involved in cell cycle, growth and inflammation (Darnell, 2002; Wu and Chiang, 2007; LeRoy et al., 2008; Dey et al., 2009; Nicodeme et al., 2010). BETs therefore provide potential therapeutic targets for modulating gene expression programs associated with various human diseases. Dawson and colleagues identified novel interactions between BET family members bromodomain protein (BRD) 3 and BRD4 with components of the SEC and polymerase-associated factor complexes in MLL fusion cells (Dawson et al., 2011). Given that BRD3 and BRD4 may be involved in the recruitment of the SEC and PAF complexes to regions of active chromatin, the authors tested the hypothesis that the dislocation of BET proteins from chromatin constitutes a viable therapeutic strategy in the treatment of MLL-fusion leukemia. For this purpose, Dawson and colleagues developed I-BET151, a BET inhibitor that selectively binds to the bromodomains of BET proteins and prevents their ability to bind acetylated histone residues (Dawson et al., 2011).

In Figure 2A and S11A-B, Dawson and colleagues assessed the ability of I-BET151 to suppress cell growth in a variety of human leukemia cell lines (Dawson et al., 2011). In these experiments, cells were treated with increasing concentrations of I-BET151 and allowed to grow for a further 72 hr. I-BET151 treatment was extremely effective at inhibiting the growth of leukemic cell lines harboring MLL fusions, including MV4;11, RS4;11, MOLM13, and NOMO1 cells, as determined by their low (nanomolar range) IC50 values. In contrast, the proliferation of cell lines using other oncogenic drivers, including gain-of-function kinase activity, was either resistant (K526) or significantly less sensitive (human erythroleukemic [HEL], HL60, and MEG01 cells) to I-BET151, exhibiting IC50 concentrations in the micromolar range and above. This key experiment shows that I-BET151 exhibits potent efficacy against MLL-fusion leukemic cell lines and will be replicated in Protocol 2. More recently, substantial growth inhibition with I-BET151 has been observed in other hematological malignancies, including acute myeloid leukemia (AML) (Dawson et al., 2014), multiple myeloma (MM) (Chaidos et al., 2014), and primary effusion lymphoma (Tolani et al., 2014), as well as non-hematological cancer models (medulloblastoma, melanoma, and glioblastoma) at concentrations ranging from 100 to 500 nM (Gallagher et al., 2014; Long et al., 2014; Pastori et al., 2014). Additionally, the BET inhibitor JQ1 was reported to have a broad growth-suppressive activity, similar to I-BET151, effectively inhibiting leukemic cell lines, such as MV4;11, while K526 cells remained largely resistant (Zuber et al., 2011).

To investigate the mechanism of action for I-BET151, Dawson and colleagues assessed apoptosis and cell cycle progression after drug treatment. Closer examination of the transcriptional pathways controlled by I-BET151 revealed that drug treatment repressed the activity of several known MLL targets, including the oncogene BCL-2. Bcl-2 promotes cell survival and protects cells from a wide range of cytotoxic insults (Cory et al., 2003). In Figure 3D, the authors confirmed the ability of I-BET151 to transcriptionally downregulate BCL-2 expression in the MLL-fusion cell lines MOLM13, MV4;11, and NOMO1, but not in the K526 resistant cell line. This key experiment shows that I-BET151 is effective at silencing BCL-2 gene transcription and will be replicated in Protocol 3. In addition to MLL-fusion cell lines, I-BET151 treatment correlated with enhanced apoptosis and reduced BCL-2 gene transcription in AML patient samples (Dawson et al., 2014). In contrast, while I-BET151 also promoted cell death and/or growth inhibition in HEL cells (Wyspianska et al., 2014), Me1007 melanoma cells (Gallagher et al., 2014), and Sufu−/− cells (mouse embryo fibroblasts deficient in the hedge hog signaling molecule Smoothened) (Long et al., 2014), drug treatment did not significantly impact Bcl-2 at either the gene or protein expression level.

Materials and methods

Protocol 1: Determine the population doubling time of K562 and MV4;11 cells

The doubling time of K-562 and MV4;11 cells is assumed to be approximately 25 and 50 hr, respectively. To empirically determine the doubling time in the replicating lab, this general protocol will be used to determine the treatment time of K562 and MV4;11 cells for Protocol 2.

Sampling

■ This experiment is performed with two cell lines (K562 and MV4;11).

■ Each cell line to be performed with six technical repeats per experiment.

Deliverables

Confirmatory analysis plan

■ n/a.

Known differences from the original study

All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm their identity and will be sent for mycoplasma testing to ensure there is no contamination. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/hcqqy) and made publically available.

This protocol assesses the ability of I-BET151, a small molecule inhibitor of BET family proteins, to selectively and potently inhibit the growth of the human leukemic cell line MV4;11, which is driven by an oncogenic translocation of the MLL gene. As a negative control, human K-562 leukemic cells, which are not oncogenically driven by an MLL-fusion, will also be treated with I-BET151. As a further negative control, both cell lines will be treated with vehicle alone (dimethyl sulfoxide (DMSO)). This protocol will replicate experiments reported in Figure 2A, Supp. Figure 11A, and Supp. Figure 11B.

Sampling

■ This experiment will be performed three separate times (biological replicates) for a final power of ≥80%. The original data reported a single IC50 value for each cell line, thus to determine an appropriate number of replicates to perform initially, sample sizes required based on a range of potential variance was determined. The sample size will also be determined post hoc as described in ‘Power calculations’ and additional replicates will be performed if necessary.

○ See ‘Power calculations’ section for details.

■ Experiment has two cohorts:

○ K562 human leukemic cells (−MLL).

○ MV4;11 human leukemic cells (+MLL).

■ Each cohort has 11 conditions to be performed in technical triplicate per experiment:

c. Calculate viability as a percentage of control (DMSO (vehicle) cells) after background subtraction.

Determine IC50 values for each cell line.

Repeat independently two additional times.

Deliverables

Data to be collected:

○ STR profile and result of mycoplasma testing of cells.

○ Raw absorbance data, I-BET151 values at each concentration normalized to DMSO-treated control values, and analyzed data (sigmoidal dose–response curves for I-BET151), and IC50 values determined for each cell line and repeat. (Compare to Figures S11A and S11B).

○ The replication data (mean and 95% confidence interval) will be plotted with the original reported data value displayed as a single point on the same plot for comparison.

Known differences from the original study

All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm their identity and will be sent for mycoplasma testing to ensure there is no contamination. The doubling time of each cell line was determined in Protocol 1. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/hcqqy) and made publically available.

This protocol evaluates the expression of the BCL2 gene in both MV4;11 (+MLL) and K-562 (−MLL) leukemic cell lines following treatment with the BET inhibitor I-BET151. BCL2 is a key anti-apoptotic gene implicated in the pathogenesis of MLL-fusion leukemias. Treatment with I-BET151 is expected to reduce the expression of BCL2 in MV4;11 cells, but not in the unresponsive K-562 cells. As a control, both cell lines will also be treated with vehicle alone (DMSO only). The expression of BCL2 will be normalized against the endogenous expression of β2 microglobulin (B2M). This protocol will replicate experiments reported in Figure 3D.

Sampling

■ Perform this experiment three separate times (biological replicates) for a minimum power of 80%.

○ See ‘Power calculations’ section for details.

■ Experiment has two cohorts:

○ K562 human leukemic cells (−MLL).

○ MV4;11 human leukemic cells (+MLL).

■ Each cohort has two conditions to be performed in technical duplicate per experiment (qRT-PCR of BCL2 and B2M):

Confirmatory analysis plan

■ One-sample t-test of ΔΔCT values from K562 cells compared to a constant of 1.

■ One-sample t-test of ΔΔCT values from MV4;11 cells compared to a constant of 1.

Meta-analysis of effect sizes:

○ Compute the effect sizes of each comparison, compare them against the effect size in the original paper and use a random effects meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm their identity and will be sent for mycoplasma testing to ensure there is no contamination. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/hcqqy) and made publically available.

This protocol assesses the maximum tolerable dose (MTD) of I-BET151 in a xenograft mouse model of leukemia by intra-peritoneal injection, using a range of I-BET151 compound. The original study reported using 30 mg/kg/day, however, as batch-to-batch variation occurs, the MTD will be assessed in this protocol to avoid toxicity. The MTD determined in this protocol will be used in Protocol 5 to assess the efficiency of I-BET151 in this model.

Sampling

■ Experiment has four cohorts:

○ Cohort 1: NOD/SCID mice treated daily with vehicle only.

○ Cohort 2: NOD/SCID mice treated daily with 10 mg/kg/day I-BET151.

○ Cohort 3: NOD/SCID mice treated daily with 20 mg/kg/day I-BET151.

○ Cohort 4: NOD/SCID mice treated daily with 30 mg/kg/day I-BET151.

■ Experiment will use five mice per treatment group.

○ See ‘Power calculations’ section for details.

Materials and reagents

Reagent

Type

Manufacturer

Catalog #

Comments

MV4;11

Human cell line

ATCC

CRL-9591

–

I-BET151 (GSK1210151A)

Small molecule

Sigma–Aldrich

SML0666

–

DMSO, molecular biology grade

Reagent

Sigma–Aldrich

D1435

Original brand not specified

RPMI-1640 medium, with L-glutamine and sodium bicarbonate

Cell culture reagent

Gibco, Life Technologies

22400-089

Original catalog number not specified

Fetal bovine serum (FBS)

Cell culture reagent

Sigma–Aldrich

F2442

Original brand not specified

Penicillin–streptomycin solution (100×) stabilized

Cell culture reagent

Invitrogen

15140122

Original brand not specified

Phosphate buffered saline (PBS)

Buffer

Gibco, Life Technologies

14190-136

–

Female and male NOD-SCID mice (6–8 weeks old)

Animal model

Jackson Laboratory

001303

–

IV Busulfex (busulfan)

Chemical

Otsuka America Pharmaceutical, Inc.

NDC 59148-070-90

Not originally used

½ cc LO-DOSE U-100 insulin syringe 28G

Labware

Becton–Dickinson

329461

Original brand not specified

APC anti-human HLA-A,B,C antibody

Antibodies

Biolegend

311410

Original catalog number not specified

APC mouse IgG2a, κisotype control antibody

Antibodies

Biolegend

400220

–

Kleptose HPB

Chemical

Roquette Pharma

n/a

Original brand not specified

0.9% NaCl, USP

Chemical

Hospira, Inc

0490-1966-05

Original brand not specified

1cc insulin syringe U-100 27G 5/8

Labware

Becton–Dickinson

329412

Original brand not specified

Flow cytometer

Instrument

Becton–Dickinson

n/a

Canto or LSR II (replaces CyAn ADP from Dako)

FlowJo software

Software

Tree Star, Inc

n/a

–

Procedure

Note

All cells will be sent for mycoplasma testing and STR profiling, as well as screened against a Rodent Pathogen Panel.

The MTD will be determined by identifying the dose at which the group body weight loss does not exceed 20% compared with the vehicle group and at which morbidity is not observed in one or more animals. When the MTD is reached, the next lowest dose will be used in Protocol 5.

○ All flow cytometry plots in gating scheme (including controls), leading to final populations of HLA-A,B,C+ cells before treatment intervention.

Confirmatory analysis plan

■ n/a.

Known differences from the original study

The original study conditioned the recipient mice with a sublethal dose of radiation (300 cGy) prior to injection of MV4;11 cells. The replication attempt will use a single dose of busulfan, which has been reported to be comparable for human cell engraftment in NOD-SCID mice (Robert-Richard et al., 2006). All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure there is no contamination. Additionally, cells used for xenograft injection will be screened against a Rodent Pathogen Panel to ensure no contamination prior to injection. The animals will be randomized prior to treatment. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/hcqqy) and made publically available.

This protocol assesses the efficacy of I-BET151 as a therapeutic agent in a xenograft mouse model of leukemia. Immunocompromised mice will be injected with preparations of MV4;11 cells and disease will progress for 21 days. At day 21, mice will be treated either with I-BET151 or vehicle control. Disease-free progression will be measured and plotted, as reported in Figure 4B. The presence and degree of disease progression will be determined by measuring the number of human leukemia cells present in the PB, spleen, and bone marrow (BM) of leukemic xenograft mice. Leukemic mice treated with I-BET151 will be compared to mice treated with vehicle control. This protocol replicates the experiments reported in Figure 4D and Supp. Figure 16A.

i. Use 0.5% human leukemia cells (HLA-A,B,C+ cells) over the total live nucleated cells (human and mouse cells) in sample as a minimum threshold of engraftment (leukemia detected).

b. Animals are randomized according to a stratified randomization procedure to balance gender and baseline tumor characteristics.

i. Female and male mice are assigned into separate blocks.

ii. In each block, animals are ranked according to disease burden (percent human HLA-A,B,C+ cells) and group assignment is performed with a simple randomization procedure.

Begin once daily intraperitoneal injections with vehicle control or I-BET151 dose determined from Protocol 4 (dose volume is 10 ml/kg).

a. Prepare vehicle and drug as outlined in step 6 of Protocol 4.

b. The same lot of I-BET151 will be used.

Continue dosing mice with either drug or vehicle every day for 21 days.

a. Monitor mice as described in step 7 of Protocol 4.

b. Euthanize mice when they receive a health-monitoring score of 3 or within 3 days of the last treatment.

At sacrifice, collect PB by cardiac puncture into EDTA-treated tubes. Remove spleen and both tibias and femurs per mouse.

a. Prepare cell suspensions from spleen (SPL) by pressing the spleen through a cell strainer in PBS and BM cells by flushing both tibias and femurs with PBS following the replicating lab's standard protocols.

For each mouse, confirm the presence or absence of leukemia. If a mouse is euthanized before the end of the experiment time length, but does not have detectable disease as assessed by flow cytometry, they should be censored from the Kaplan–Meier survival curve.

a. Use 0.5% human leukemia cells (HLA-A,B,C+ cells) over the total live nucleated cells (human and mouse cells) in sample as a minimum threshold of engraftment (leukemia detected).

○ Include raw disease-free survival data for I-BET151 treated and untreated xenografted mice, including any mice censored because of no detectable disease.

○ All flow cytometry plots in gating scheme (including controls), leading to final populations of HLA-A,B,C+ cells before and after treatment intervention. Compare to Figure 4D and Supplemental Figure S16A.

○ Number of HLA-A,B,C+ cells in PB, SPL, and BM in each treatment group.

Confirmatory analysis plan

■ Statistical analysis of the replication data:

○ Comparison of Kaplan–Meier survival curves between vehicle and I-BET151-treated mice using the Log-rank Mantel–Cox test.

■ Meta-analysis of effect sizes:

○ Compute the effect sizes of each comparison, compare them against the effect size in the original paper, and use a random effects meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

The original study conditioned the recipient mice with a sublethal dose of radiation (300 cGy) prior to injection of MV4;11 cells. The replication attempt will use a single dose of busulfan, which has been reported to be comparable for human cell engraftment in NOD-SCID mice (Robert-Richard et al., 2006). The original study counted PB cells using a SciVet abc machine, while the replication attempt will include CountBright absolute counting beads to determine the absolute number of human leukemia cells in each mouse after treatment. The original study lysed red blood cells from samples using Red Blood Cell Lysis Buffer, while the replication attempt will use ammonium chloride solution while performing HLA-A,B,C and 7-AAD analysis. For analysis of leukemia burden using CountBright absolute counting beads, the cells will be lysed using 1× DB Lysis Buffer during manufacturer's instructions. All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm their identity and will be sent for mycoplasma testing to ensure there is no contamination. Additionally, cells used for xenograft injection will be screened against a Rodent Pathogen Panel to ensure no contamination prior to injection. The animals will be randomized prior to treatment. The apoptotic marker dye 7-AAD will be used to exclude populations of dead or dying cells from analysis and an isotype control antibody will be used to confirm the specificity of the HLA-A,B,C antibody. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/hcqqy) and made publically available.

Power calculations

For additional details on power calculations, please see analysis scripts and associated files on the Open Science Framework:

Protocol 1

Not applicable.

Protocol 2

Summary of original data reported in Figures 2A, S11A, and S11B:

Cell line

IC50

K562 cells (−MLL)

>100 µM

MV4;11 cells (+MLL)

26 nM

The original data do not indicate the error associated with multiple biological replicates. To identify a suitable sample size, power calculations were performed using different levels of relative variance.

In order to produce quantitative replication data, we will run the experiment three times. Each time we will determine the IC50. The three replicates and the original reported value will be checked to see if the original value is an outlier using Grubb's test (with a significance level of 0.05). If the original value is detected as an outlier it will not be included with the replication replicates to determine the standard deviation of IC50 values, otherwise it will be included in the standard deviation calculation. The calculated standard deviation will be combined with the reported value from the original study to simulate the original effect size. We will use this simulated effect size to determine the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure that the experiment has more than 80% power to detect the original effect.

Since the original comparison was not statistically significant. This is the effect size that can be detected with 80% power and the indicated sample size. Unlike the above power calculations, the aim of this sensitivity calculation is not to detect the original effect size, but to understand what effect size could be detected. The original effect size is 1.300.

To keep animals per group balanced 16 (4 per group), 20 (5 per group), or 24 (6 per group) total samples keeps E between 10 and 20. 20 total animals (5 per group) will be used to account for any potential exclusion.

Protocol 5

Summary of original data estimated from Kaplan–Meier graph reported in Figure 4B:

Two mice were censored from the I-BET151 cohort during the 21-day treatment period. For the power calculations, the censoring rate was divided in half since the calculation assumes the censoring rate is equal for both groups.

Decision letter

Karen Adelman

Reviewing Editor; National Institute of Environmental Health Sciences, United States

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for submitting your work entitled “Registered report: Inhibition of BET recruitment to chromatin as an effective treatment for MLL-fusion leukaemia” for peer review at eLife. Your submission has been favourably evaluated by Charles Sawyers (Senior Editor), a Reviewing Editor, and three reviewers.

The reviewers have discussed their reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

The manuscript is designed to replicate the major findings of Dawson et al.,‘Inhibition of BET recruitment to chromatin as an effective treatment for MLL-fusion leukaemia’ (Nature, 2011).

There are 3 main experiments chosen for replication. (i) To replicate the sensitivity of MLL rearranged leukaemia cell lines to BET bromodomain inhibition (ii) To assess the downregulation of BCL2, a major target gene and (iii) To assess the efficacy of I-BET against a xenograft model of leukaemia.

Detailed protocols are provided for replicating each of these key experiments, with increased sample sizes above the original manuscript to ensure statistical significance of results.

Essential revisions:

1) In general, the authors summarize the literature adequately. This area of biomedical/cancer research has been very active and there have been many studies that have demonstrated pre-clinical efficacy for BET bromodomain inhibition in various malignancies. Nonetheless the authors should cite Zuber et al. (Nature, 2011); Delmore et al. (Cell, 2011) and Mertz et al. (PNAS, 2011) as these studies were published concurrently to the Dawson et al. study. In particular:

2) From a statistical perspective, the proposal looks very detailed and acknowledges managing some of the uncertainty in the study design. There is also clear discussion of the randomisation processes to be used. However, there are some aspects that necessitate some clarification.

a) Protocol 2 allows for the fact that the original report does not report any error for the average value reported. Was any attempt made to get this data from the original authors? Getting the original data for error would be preferable to generating replicates to get an estimate of the SD from 3 new biological replicates and the original report. If that is not possible then I think the 3 new biological replicates will be added to the original report to make a sample size of 4 to estimate the SD. Then this SD will be used to scale the original (singly reported) value as a scaled effect size and then a new sample size calculation will be performed to see how many more than 3 will be needed to achieve 80% power.

b) Protocol 3. Within this protocol there is a sample size calculation (sensitivity) to see what effect size could be detected with n=3. However, this comparison was not statistically significant in the original report. I think the aim of this part of the protocol should be reworded.

c) Protocol 4. Five mice per group will be used in the MTD analysis and this will be reported using Kaplan Meier plots. However, no sample size justification was made. What is the reasoning for using 5 per group?

Author response

1) In general, the authors summarize the literature adequately. This area of biomedical/cancer research has been very active and there have been many studies that have demonstrated pre-clinical efficacy for BET bromodomain inhibition in various malignancies. Nonetheless the authors should cite Zuber et al. (Nature, 2011); Delmore et al. (Cell, 2011) and Mertz et al. (PNAS, 2011) as these studies were published concurrently to the Dawson et al. study. In particular:

Thank you for bringing these to our attention. We have updated the manuscript to include these references.

2) From a statistical perspective, the proposal looks very detailed and acknowledges managing some of the uncertainty in the study design. There is also clear discussion of the randomisation processes to be used. However, there are some aspects that necessitate some clarification.

a) Protocol 2 allows for the fact that the original report does not report any error for the average value reported. Was any attempt made to get this data from the original authors? Getting the original data for error would be preferable to generating replicates to get an estimate of the SD from 3 new biological replicates and the original report. If that is not possible then I think the 3 new biological replicates will be added to the original report to make a sample size of 4 to estimate the SD. Then this SD will be used to scale the original (singly reported) value as a scaled effect size and then a new sample size calculation will be performed to see how many more than 3 will be needed to achieve 80% power.

We did reach out to the original authors about obtaining the originally reported data. Unfortunately, they were unable to provide the raw data for any of the included experiments. Thank you for the suggestion regarding the approach to used the replication variation and the original (singly reported) value to calculate the effect size to use in a new sample size calculation. The one potential issue that could potentially arise is if the replication data are significantly different than the original value, which would cause the SD to greatly increase. Thus, we propose to include the original value as long as it is not detected as an outlier using Grubb’s test. We have updated the manuscript to include this approach.

b) Protocol 3. Within this protocol there is a sample size calculation (sensitivity) to see what effect size could be detected with n=3. However, this comparison was not statistically significant in the original report. I think the aim of this part of the protocol should be reworded.

Thank you for the suggestion. We have reworded this part to increase the clarity of the calculation.

c) Protocol 4. Five mice per group will be used in the MTD analysis and this will be reported using Kaplan Meier plots. However, no sample size justification was made. What is the reasoning for using 5 per group

We have included the justification in the revised manuscript. The MTD will be determined using body weight loss and morbidity, while also reporting overall survival using Kaplan Meier plots. Sample size was determined using the law of diminishing return (since the primary aim is to find any level of difference between groups), with 5 animals per group sufficient to keep E between 10 and 20, while also accounting for any potential loss due to exclusion.

Funding

Laura and John Arnold Foundation

Reproducibility Project: Cancer Biology

The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Mark Dawson for generously sharing critical information to ensure the fidelity and quality of this replication attempt. We thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type Culture Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich, and System Biosciences (SBI).

Reviewing Editor

Karen Adelman, National Institute of Environmental Health Sciences, United States

In 2015, as part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Fung et al., 2015), that described how we intended to replicate selected experiments from the paper "Inhibition of BET recruitment to chromatin as an effective treatment for MLL-fusion leukaemia" (Dawson et al., 2011). Here, we report the results of those experiments. We found treatment of MLL-fusion leukaemia cells (MV4;11 cell line) with the BET bromodomain inhibitor I-BET151 resulted in selective growth inhibition, whereas treatment of leukaemia cells harboring a different oncogenic driver (K-562 cell line) did not result in selective growth inhibition; this is similar to the findings reported in the original study (Figure 2A and Supplementary Figure 11A,B; Dawson et al., 2011). Further, I-BET151 resulted in a statistically significant decrease in BCL2 expression in MV4;11 cells, but not in K-562 cells; again this is similar to the findings reported in the original study (Figure 3D; Dawson et al., 2011). We did not find a statistically significant difference in survival when testing I-BET151 efficacy in a disseminated xenograft MLL mouse model, whereas the original study reported increased survival in I-BET151 treated mice compared to vehicle control (Figure 4B,D; Dawson et al., 2011). Differences between the original study and this replication attempt, such as different conditioning regimens and I-BET151 doses, are factors that might have influenced the outcome. We also found I-BET151 treatment resulted in a lower median disease burden compared to vehicle control in all tissues analyzed, similar to the example reported in the original study (Supplementary Figure 16A; Dawson et al., 2011). Finally, we report meta-analyses for each result.

eLife is a non-profit organisation inspired by research funders and led by scientists. Our mission is to help scientists accelerate discovery by operating a platform for research communication that encourages and recognises the most responsible behaviours in science.eLife Sciences Publications, Ltd is a limited liability non-profit non-stock corporation incorporated in the State of Delaware, USA, with company number 5030732, and is registered in the UK with company number FC030576 and branch number BR015634 at the address:
eLife Sciences Publications, Ltd
Westbrook Centre, Milton Road
Cambridge CB4 1YG
UK