We now have reviews from the journal. Below is the text of the reviews.

In terms of other elements:

• We have until January 15 to get a revised manuscript back to the journal.

• We need to review, revise, and merge or close each of the pull requests open from the community. We will need help with this from current authors: https://github.com/greenelab/deep-review/pulls

  • Update domain adaptation in TFBS section #532
  • Consolidate and re-frame the Discussion Section #559
  • Made some changes in Intro #566
  • added: Privacy Preserving generative deep neural networks support cli… #577
  • Sub-section on structure determination using cryo-EM #629
  • Additions to EHR section #632
  • Multiples revisions in md files: 02/03/04/05/06  #636
  • Added PPI section with MHC subsection #638
  • Single cell representation learning #648

• We will stop accepting new contributions from those who are not authors of the manuscript or authors of any currently open pull requests as of the end of the day on Friday. This is for the purpose of getting the revised manuscript back by January 15.

I created sub-issues to track our responses to each individual reviewer comment.

• Ref 1.1 Referee 1.1 #683
• Ref 1.2 Referee 1.2 #684
• Ref 1.3 Referee 1.3 #685
• Ref 1.4 Referee 1.4 #686
• Ref 1.5 Referee 1.5 #687
• Ref 2.1 Referee 2.1 #688
• Ref 2.2 Referee 2.2 #689
• Ref 2.3 Referee 2.3 #690

When you address a review in a pull request, please also update response-to-reviewers.md.

Referee: 1

Comments to the Author
The authors discussed opportunities and obstacles for applying deep learning to tackle biomedical problems. The reviewer thinks that this note would be a valuable contribution to the journal, since it provides with a coherent and comprehensive coverage of this interesting research field. However, the reviewer has the following concerns/suggestions.

The authors summarized over 400 literature references purely by narratives. The authors provided synopsis for each important reference, but lacks synthesis of related work. It would be better to synthesize related work into a table and analyze their characteristics.

The authors discussed deep learning models such as sDA, CNN, RNN etc. It would be better to have a figure illustrating their architectures. This way, the reader will have a concrete visualization that will aid the understanding of the discussion points in the manuscript.

The authors gave a case study of LADA to suggest that integrating multiple data sources may lead to breakthrough medical discoveries. However, it is unclear from the authors’ description that deep why learning models possess such integrating capability. In fact, the tensor model seems to be the widely acknowledged model that can easily integrating multiple data sources.
The authors mentioned “One source of training examples with rich clinical annotations is electronic health records”. What does it mean by “rich clinical annotations”? Can the authors provide a definition and a few examples.

Some existing biomedical informatics systems are not cited. For example, please provide a citation to NegBio.

Referee: 2

Comments to the Author
This is a very timely review of recent progress in applying machine learning or more precisely deep learning approaches to medical and biological data. Despite the relative recent surge in the application of deep learning algorithms in these areas the long list of literature references is a testament of the rapid growth of this field. Overall I found the review very well written and extremely interesting to read. It provides a very useful overview and a rich source of references to recent attempts of leveraging the power of deep learning algorithms for biomedical data analysis. All sections are highly informative, but I found the sections dealing with the problems around obstacles to data sharing, privacy issues, and data quality, as well as the sections on the difficulty of meeting the high standards of decision making in a clinical setting particularly thought stimulating.

Any review that tries to tackle such a complex and broad subject will have some shortcomings. The review in its current form represents certainly already a valuable resource and thought stimulating reading and would, in my view, be satisfactory and acceptable for publication. However, I want to point out a few suggestions that could make the review even more useful.

1. There is little explanation of key deep learning concepts: layers, autoencoders, RNNs, etc. It might be impossible to do that within the space limitations of the current review, I wonder whether a link to a dedicated website or supplementary material where the most often used deep learning concepts are explained in a way an uninitiated reader can quickly read through and understand would be a solution. There is certainly a large readership whose interest has been peaked by countless references to deep learning even in the popular press, but who are very confused when autoencoders, LSTMs and RNNs get thrown at them without any even brief explanation what they are. Just sending readers off to fend for themselves through internet searches or to study the excellent but still quite technical book [10] Goodfellow et al. is probably not the most satisfactory answer.

2. There is a slight imbalance in the presentation of various application areas. The section on drug development (p40ff), for example, is quite extensive and provides a lot of technical details which might be less relevant for a reader who tries to get a general overview of deep learning in biomedical research. An area which is little mentioned on the other hand are deep learning approaches to brain data, eg connectivity maps, and the area of learning from structured data, such as graphs.

3. The main issue with machine learning solutions in a medical, particularly clinical or public health setting is the lack of proper measures of uncertainty, as it is traditionally provided either in the framework of hypothesis testing or in the increasing acceptance of posterior Bayesian inference for public health decisions. Although this is mentioned throughout the review, this issue deserves a much more prominent place in the introduction and the discusssion, since it is one of the key obstacles for the acceptance of machine learning approaches outside exploratory analyses in basic biological research.