By Anonymous User
Review Details
Reviewer has chosen to be Anonymous
Overall Impression: Weak
Content:
Technical Quality of the paper: Average
Originality of the paper: Yes, but limited
Adequacy of the bibliography: Yes
Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Good
Organization of the paper: Needs improvement
Level of English: Satisfactory
Overall presentation: Average
Detailed Comments:
* Summary
This paper analyzes the impact of adding additional information to
knowledge graphs on link prediction tasks. The contributions also
include new datasets and mechanisms to extend knowledge graphs. The
evaluation is carried out over several knowledge graph embeddings.
From the previous version, the paper contains several
changes. However, I believe the main comment I raised in the previous
version has not been properly addressed.
* Strengths
- The paper addresses an important gap in understanding KG design
choices and downstream performance
- The synthetic dataset creation methodology provides controlled
experimental conditions
- The work contributes new datasets (FB15k isotopes and SKG variants)
that may benefit the community
* Comments
- I believe there is a mismatch between the hypothesis and the
evaluation. In my understanding, the central point of the paper is
to study the impact of extending knowledge graphs with additional
semantic information. However, I am concerned about how the study is
carried out. For example, if we consider the subclassOf relation
that is added to SKG-4 and FB237, we know that the subclassOf
relation is transitive and therefore the graph could contain chains
like "A subclassOf B subclassOf C subclassOf D". Most (and maybe
all) methods, especially TransE, cannot handle the semantic
properties of transitive relations. TransE would model the triples
(A, subclassOf, B), (B, subclassOf, C), (A, subclassOf, C) in the
embedding space with the following restrictions:
v(A) + v(subclassOf) = v(B)
v(B) + v(subclassOf) = v(C)
v(A) + v(subclassOf) = v(C)
In order to fulfill those restrictions v(subclassOf) would converge
to 0, which is a trivial and maybe non useful embedding.
This limitation extends beyond TransE to other methods in the
evaluation. DistMult is designed for symmetric relations but is
evaluated on asymmetric relations like instanceOf (P31) and
subclassOf (P279). ComplEx and RESCAL also have specific
architectural constraints that limit their ability to properly
encode hierarchical and transitive properties. The methods selected
are theoretically limited in handling the semantic properties of the
additional information being tested.
Therefore, choosing these methods as representatives can lead to
misleading conclusions about the impact of semantic information on
KG embeddings. Instead, the analysis should consider embedding
methods that can leverage the semantic properties of the new
information (e.g., hierarchy-aware methods for subclassOf,
cardinality-aware methods for instanceOf) in order to match the
hypothesis with the experimental design.
The conclusion the authors claim "While adding complexity enhances
semantic depth, it also makes prediction more difficult" is not
justified since the selected embedding methods cannot handle the
semantic properties of the extra information. Without methods
capable of utilizing the added semantics, poor performance becomes
predictable rather than informative about the impact of structural
complexity.
This fundamental methodological issue prevents the paper from
providing reliable insights about the relationship between KG design
choices and embedding performance, which undermines its primary
contribution to the field.
