By Ben Wright
Review Details
Reviewer has chosen not to be Anonymous
Overall Impression: Average
Content:
Technical Quality of the paper: Average
Originality of the paper: Yes, but limited
Adequacy of the bibliography: Yes
Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Limited
Organization of the paper: Needs improvement
Level of English: Satisfactory
Overall presentation: Average
Detailed Comments:
Overall, I found the concept and discussion relevant to the NAI journal and audience. I did however, find that the paper has a few issues to address before giving a full accept.
The main novel idea discussed here is the concept of the editing distance between two data structures. This aspect is intriguing and leads me to think about it in the context of some ideas not discussed by the authors (how would this editing distance and created data relate to synthetic data measures?). I am a bit surprised to see that no discussion towards path planning problems or planning problems in general were used as the framing for the editing distance plays very much into those classical AI bins, as distance between two state spaces is an often used heuristic.
One of my bigger issues with the current structure of the article is that it stays too far into the abstract without contextualizing the theory discussed. Additionally, while mentioning in the into and abstract that this relates to the context of ``cognitive computational neuroscience" the authors never bring this back in the conclusion to put a bow on the whole concept.
Following this thought, I was underwhelmed by the papers current level of discussion around "implementation". It was my understanding from the abstract that this was "implemented in detail at the programming level" yet by the time I ended section 5 I did not really see any discussion of running an implementation (even a prototype). Adding more specifics and proof-of-concept implementation discussions (I.e. "we ran a small data structure of X through a usual ML algorithm and our results were..... ") would greatly improve the paper's impression and quality.
There are additional issues of clarity that I would hope can be improved. The paper has an overabundance of "i.e." and "e.g." examples, sometimes within the same sentence. If the authors wish to expand on an idea they are discussing, it is perfectly fine to just state that in the next sentence. There is also an inconsistency between the Introduction and Symbolic Data Representation sections and the rest of the paper with the use of footnotes. The use of footnotes are also excessive, and (this is for footnote 11) sometimes referenced within the main body of the article. Footnotes 12 and 13 do not exist.
Following that, I would also say that most of the tables/charts and equations are not numbered or well placed within the context of their discussion (See pg. 8 line 37 equation and then its follow up on pg. 9 line 3). Equations are also introduced in some footnotes (Also page 8), I'm not sure the reasoning here versus a discussion of it in the main body of the article.
Likewise, the overuse of the caption text for Figure 4 (pg 8 lines 23-28) while simultaneously not discussing the figure in the main body of the article. Figure descriptions should be one sentence, at most.
I would also like to say that, in the context of Figures, make sure to discuss the full figure and why it is all in the figure before the figure appears. Figures 2 and 4 especially fall into this, but a look over all Figures would be beneficial.
Stylistically, it seems there are some presentation issues with how the bulleted lists are used in the paper. The readability is off, and sometimes it seems that the tables are being used for the same context as the bulleted lists. (See page 6, line 11; pg 9, line 4; pg 10 line 28; pg 12 line 32 -> pg 13 line ~3 is especially egregious)
The use of citations and background/relevant work is odd for this paper. My interpretation for the NAI Journal is that you will have a wide mixture of audience readers with a variety of backgrounds, so the assumptions taken may not always be correctly assumed. For instance my background is on the Symbolic AI/ Logic side and would need a more concise introduction to some of the ``usual ML algorithms" (pg 14, line 31)
Also, especially in the context of section 1, there is a high risk of this paper beginning to cite papers just to cite them. A number of the citations are used as a "see this paper for a thing[X]" and then does not have followup or contextual nuance for why it is really being discussed within the context of *this* paper. Pg 3 lines 4-7 are an example of this to me.
The overall presentation of the sections could use a bit of a revamp as well. The idea of combining the ending of your paper along with the "discussion" sits wrong with me. The authors are still in the `introducing new ideas to the conversation' segment during the discussion and that I find is best reserved for sections outside of conclusions and last sections of the paper. To that end, I would have section 5 not be the concluding section of the paper.
Section 2 feels very aligned with sections 1.1 and 1.2. I would probably combine them all together for your Section 2 *BACKGROUND* and then move onto the full discussion of the editing distance stuff in Section 3.
I also think Appendix A.1 would be fine as just part of the main body of the writing to give better context to the discussion around data structures.
Other Minor Things,
- References aren't an Appendix, they are just References.
- Pg 25, Line 14 -- Why is there a different use of citation here?
- Give everything another reading pass, give it to a colleague who didn't write it and have them read it out loud for a bit and see if it matches with your intent on the writing.