By Anonymous User
Review Details
Reviewer has chosen to be Anonymous
Overall Impression: Good
Content:
Technical Quality of the paper: Good
Originality of the paper: Yes
Adequacy of the bibliography: Yes
Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Good
Organization of the paper: Satisfactory
Level of English: Satisfactory
Overall presentation: Good
Detailed Comments:
Review:
The authors state that developiong neurosymbolic systems is challenging
to developers due to lack of development environments, including libraries, reducing the potential impact of the area.
This also makes it harder to compare the systems' performances uniformly
and to identify research directions building on current research.
To address these issues, the authors present a
comparative analysis that
identifies the main components of existing NeSy frameworks,
compares frameworks across the identified facets,
and highlights the requirements for the future of NeSy frameworks
building upon limitations of current systems and
possible interplays between the neural and symbolic components.
To perform the analysis, they work with three systems
DeepProbLog, Scallop, and DomiKnowS over four different tasks.
The paper covers an overview of neurosymbolic frameworks,
covering their main features: flexibility for modeling both neural and symbolic components, symbolic representation language that can be seamlessly connected to neural components, flexibility of connecting to various architectures, including various loss functions, sources
of supervision, and training paradigms, provision of a modeling language for specification and seamless integration of the two components.
A distinction between Nesy techniques and frameworks is described,
as the first is task-specific.
NeSy frameworks are characterizes by a Symbolic knowledge representation language, the representation and flexibility of meural modeling, a model declaration, he interplay between symbolic and sub-symbolic systems, and the use of LLMs.
Next, the authors describe the core componenets of a NeSy framework: symbolic knowledge and neural models representations used by the frameworks. An analisys w.r.t. DeepProbLog, Scallop, and DomiKnowS features is described.
The interplay between the symbolic and subsymbolic interplay of the
three chosen frameworks is presented next, and in the sequel,
the role of LLMs as a "promise for overcoming the bottleneck" of
acquiring symbolic representation. They are described both as sources of symbolic knowledge, but also to generate inputs to symbolic engines.
The authors cover examples of how LLMs are this integrated with NeSy
frameworks.
In the more experimental section, the authors show that in
DomiKnowS, the symbolic reasoning part is formulated as a
logical constraint solving, inDeepProbLog, the symbolic reasoning problem is interpreted a probabilistic logic program,
in Scallop the problem is viewed as a combination
of the neural and the symbolic components, and in LEFT
which is included for the visual question answering task,
the problem is limited to the application of concept learning and grounding language into visual modality.
The tasks experimented in the paper include: MNIST Sum,
Shapes VQA, Toy Named-Entity Recognition, and Math Equation Inference.
The results of the experimentation suggest that the
frameworks tested still present behavior that hinder the application and flexibility of the frameworks. As regards symbolic representation in the frameworks, the authors argue that knowledge representation for neurosymbolic AI requires a language designed for the integration purpose with adaptable semantics, in which learning is the key concept.
Regarding neural modelling, the authors claim that most examined frameworks leave the neural modeling and the
of connecting the symbolic and sub-symbolic components up to the user,
lacking user-friendly libraries.
With respect to the role of LLMs, the authors suggest that
they can reduce the classical issues in symbolic
processing as their large knowledge can be used to reduce the need for rebuilding neural components. Also, thei allow for flexible connections
with symbolic components.
In summary, the paper illustrates how the coming NeSy frameworks should focus on providing flexible implementation tools, user-friendly interfaces, catering for improved scalability, and present seamless integrations with foundation models/LLMs.
Minor issue:
Page 9: In DeepProbLog, the symbolic reasoning problem is interpreted *as* a probabilistic logic programs in ProbLog.
Insert "as"
