Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Matters Arising
  • Published:

Reply to: Transparency and reproducibility in artificial intelligence

Nature volume 586, pages E17–E18 (2020)Cite this article

Subjects

The Original Article was published on 14 October 2020

replying to B. Haibe-Kains et al. Nature https://doi.org/10.1038/s41586-020-2766-y (2020)

We thank the authors of the accompanying Comment1 for their interest in our work2 and their thoughtful contribution. We agree that transparency and reproducibility are paramount for scientific progress. In keeping with this principle, the largest data source used in our publication is available to the academic community. Any researcher can apply for access to the OPTIMAM database (https://medphys.royalsurrey.nhs.uk/omidb/getting-access), which our institution helped fund. The broad accessibility of the database was part of the reason we pursued this collaboration. In fact, since our article came out, another group has already published results on this very dataset3.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Haibe-Kains, B. et al. Transparency and reproducibility in artificial intelligence. Nature https://doi.org/10.1038/s41586-020-2766-y (2020).

  2. McKinney, S. M. et al. International evaluation of an AI system for breast cancer screening. Nature 577, 89–94 (2020).

    Article  CAS  ADS  PubMed  Google Scholar 

  3. Kim, H.-E. et al. Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. Lancet Digital Health 2, e138–e148 (2020).

    Article  PubMed  Google Scholar 

  4. Wu, N. et al. Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE Trans. Med. Imaging 39, 1184–1194 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Rodriguez-Ruiz, A. et al. Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J. Natl. Cancer Inst. 111, 916–922 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Lee, R. S. et al. A curated mammography data set for use in computer-aided detection and diagnosis research. Sci. Data 4, 170177 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  7. McKinney, S. M. et al. Addendum: International evaluation of an AI system for breast cancer screening. Nature https://doi.org/10.1038/s41586-020-2679-9 (2020).

  8. Price, W. N., II, Gerke, S. & Cohen, I. G. Potential liability for physicians using artificial intelligence. J. Am. Med. Assoc. 322, 1765–1766 (2019).

    Article  Google Scholar 

  9. Abadi, M. et al. Deep learning with differential privacy. In Proc. 2016 ACM SIGSAC Conference Computer Communications Security CCS’16 308–318 (2016).

Download references

Acknowledgements

We thank A. Dai and E. Gabrilovich for comments.

Author information

Authors and Affiliations

  1. Google Health, Palo Alto, CA, USA

    Scott Mayer McKinney, Daniel Tse, Yun Liu, Greg S. Corrado & Shravya Shetty

  2. Google Health, London, UK

    Alan Karthikesalingam & Christopher J. Kelly

Authors
  1. Scott Mayer McKinney
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alan Karthikesalingam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Tse
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christopher J. Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Greg S. Corrado
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shravya Shetty
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

This Reply was prepared by a subset of the authors of the original Article in addition to Y.L., all of whom have expertise related to this exchange. S.M.M., A.K., D.T., C.J.K, Y.L., G.S.C. and S.S. wrote and revised this Reply.

Corresponding authors

Correspondence to Scott Mayer McKinney or Shravya Shetty.

Ethics declarations

Competing interests

This study was funded by Google LLC. S.M.M., A.K., D.T., C.J.K, Y.L., G.S.C. and S.S. are employees of Google and own stock as part of the standard compensation package. The authors have no other competing interests to disclose.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McKinney, S.M., Karthikesalingam, A., Tse, D. et al. Reply to: Transparency and reproducibility in artificial intelligence. Nature 586, E17–E18 (2020). https://doi.org/10.1038/s41586-020-2767-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41586-020-2767-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing