A Neurosymbolic Benchmark for Temporal Knowledge-Graph Memory in Partially Observable Environments

Tracking #: 997-2032

Flag : New Paper Submitted

Authors: 

Taewoon Kim
Vincent François-Lavet
Michael Cochez

Submission Type: 

Article in Special Issue (note in cover letter)

Full PDF Version: 

Supplementary Files: 

Cover Letter: 

RESUBMIT of #966-1995 Dear Editors, Please find a revised version of "A Neurosymbolic Benchmark for Temporal Knowledge-Graph Memory in Partially Observable Environments", submitted to the Special Call for Neurosymbolic Benchmark Papers. We thank the three reviewers: the reviews were detailed, and in several places they identified real errors. The revision addresses every point raised. The largest changes are: 1. A new Section 3.1 defines the agent interface and task protocol before any agent is introduced: what the agent observes, what it must answer, what private memory it may keep, which agent classes the benchmark supports, and that the reward is primarily an evaluation signal (Reviewer 1). 2. Section 2.1 adds a discussion of reification and RDF 1.2 with a side-by-side Turtle example, and Section 4.1 adds memory-operation semantics covering insertion, repeated observation, short-term precedence, query-time and traversal updates, contradictions, and eviction (Reviewer 3). 3. A new Section 5.2, "Calibration Bounds", with Table 3, anchors the absolute scores between an analytic upper bound and an observation-only lower bound. It also answers the perfect-memory baseline directly: it is not a separate system here, since K=1024 is a capacity the existing sweep already passes through (Reviewers 2 and 3). 4. Section 5.1 reports the spread over all 27 TKG policy variants alongside the training-selected one, and reports the neural family by a bound over all eight trained configurations rather than agent by agent (Reviewer 2). 5. New temporal neural controls: size-matched LSTM and Transformer variants whose observation tokens carry an explicit arrival-timestep feature, trained under the same protocol, which is 200 additional training runs (Reviewer 3). 6. Prompted by Reviewer 2's point about unsupported claims, we re-checked every comparative claim in Section 5 against the underlying data and corrected those that did not hold. The repository now carries a script that recomputes numbers the paper reports from the released results and confirms that the paper and this response still state them. The supplementary file holds two documents: a point-by-point response to all reviewer comments, and a copy of the paper with the changes marked against the submitted version, additions in green and removals in red. Both artifacts are public and archived. The benchmark environment is at https://doi.org/10.5281/zenodo.18721817, and the baseline agents are at https://doi.org/10.5281/zenodo.21967778, together with the raw results for the TKG policy sweep and the neural runs. Kind regards, Taewoon Kim, Vincent François-Lavet, and Michael Cochez

Previous Version: