Dear Reviewers, First of all, we would like to thank you for your feedback, which has really helped in improving the quality of our position paper. All the changes that were made to the paper are highlighted in blue. Below we provide a point by point answer to all the comments. - *Reviewer 1:* The paper should be revised in order to explicitly highlight the relation and relevance of the discussed topic to the neurosymbolic AI field. This should be done both in the abstract/introduction and in the main body of the paper (e.g., by giving some examples of the "logical constraints" mentioned). Making this relation to neurosymbolic AI more prominent at the beginning of the paper would be important especially given that this paper will appear in an inaugural issue of the NAI journal.

 We have better motivated the paper from a Neuro-symbolic AI perspective. To this end, we have added such motivations both in the abstract and in the introduction (page 2 line 50 - page 3 line 21) and added an entire section (Section 4) on the central role that Neuro-symbolic AI can play in machine learning with requirements.
 - *Reviewer 2:* I feel that the authors do not give enough information on the attempts that have already been made to integrate background knowledge into neural models at testing- or training- time. The paper would benefit a lot from such a discussion, especially if the authors stress the limitations of the current neurosymbolic literature w.r.t. their manifested model development pyramid. 

 We would like to thank the reviewer for the provided references. We added a discussion on the work that has already been done in the field and the limitations of the current literature in Section 4.
 - *Reviewer 2:* Another line of research which should be mentioned is the one that studies learnability of neurosymbolic frameworks. 

 As suggested by the reviewer, we have added some considerations on the learnability of Neuro-symbolic frameworks under partial-labelling and/or weak supervision at the end of Section 4.
 - *Reviewer 3:* However, the main contribution of this paper is the pyramid model for the machine learning (system) development process. This calls for an embedding of this novel model into existing models of this process. Many reference models are cyclical rather than linear as suggested in this paper in e.g. Figure 1. And many of which explicitly mention the role of requirements in throughout the process.

 We thank the reviewer for the references. We have added them and discussed the main differences between the proposed development models and our proposed model in Section 2, page 4. 
 - *Reviewer 3:* The paper would also benefit from a discussion on *how* requirements can effectively be modeled (the leftmost arrow in the pyramid model in Figure 4) and an analysis on how this impacts the right-hand side of the model. This may also strengthen the link with neuro-symbolic AI.

 We have thoroughly discussed all the above in the intro, in Section 2 and Section 4. 
 - *Reviewer 3:* in the introduction it is claimed that requirements in any application domain can be obtained from an existing body of knowledge in this application domain and that there will be a continued push for adoption of systems even when these may have unintended consequences (ln15-18). Claims like these need to be supported

. We have added the relevant references. 
 - *Reviewer 3:” the presented pyramid model is contrasted to a "traditional 'performance-driven' [..] pipeline". Where is this traditional pipeline suggested (see note on background). Furthermore, the idea that predictive performance is the only metric to optimize for "traditionally" is immediately contradicted in the work itself, i.e. by mentioning fairness, robustness, explainability ... So the current model is already in use, what novelty remains? 

We thank the reviewer for these points, which made us elaborate further on (i) the difference between our proposed pipeline and the ones proposed in the field of MLOps, and (ii) the importance of having requirements formally defined, hence strengthening the link with Neuro-symbolic AI. We believe this greatly improved the paper. We also hope that the novelty of our proposed approach is now clearer.