\section{Experimental Setup}\label{sec:experiments}
\begin{comment}
- Present the experimental setup, including the tools and frameworks used.
- Describe the characteristics of the benchmark dataset, such as size, complexity, and types of noise introduced.
- Present the results of the experiments, including performance metrics and analysis.
\end{comment}

\subsection{Datasets}

%Dataset
We used \textit{Pizza}~\citep{manchester_pizza_ontology}, \textit{Family}~\citep{Stevens2008AFH} and \textit{OWL2Bench}~\citep{owl2bench} ontologies. Using the \textit{Pizza} ontology, we created an ABox generator to support experiments with synthetic data. The process for generating ABox data for the \textit{Pizza} ontology begins by loading the \textit{Pizza} TBox (Terminological Box) axioms. A custom instance generation step then automatically creates a specified number of individuals (ABox data), and their object properties are defined in a configuration. For this study, only \texttt{NamedPizza} class and \texttt{hasTopping} property are described in the configuration. Crucially, this generation leverages the TBox's inherent OWL restrictions (e.g., \texttt{only} or \texttt{some} constraints) to dynamically determine the appropriate target classes for object properties, thereby guaranteeing the generated ABox is semantically consistent with the ontology's definition. The final output is the complete ontology, comprising the original TBox and the newly populated ABox. In this study, we developed two datasets, \textit{Pizza\_100} and \textit{Pizza\_250}, comprising $100$ and $250$ individuals, respectively. The corresponding number of axioms for these datasets are $2,151$ and $2,628$, respectively.

Furthermore, this work incorporates the \textit{Family} ontology, a well-known ontology designed to represent family relationships and genealogical information. The \textit{Family} ontology provides a foundational framework for reasoning about kinship terms, familial roles, and relationships such as parent-child, sibling, and spouse connections. \textit{Family} contains $2,516$ axioms.

\textit{OWL2Bench} encompasses a wide variety of axioms, including Class Expression Axioms, Object Property Axioms, Data Property Axioms, and Assertions. It serves as a benchmark to evaluate the coverage, scalability, and query performance of ontology reasoners across the four OWL 2 profiles: EL, QL, RL, and DL. \textit{OWL2Bench} was developed as an extension of the well-known University Ontology Benchmark (UOBM), producing four distinct TBoxes—one for each OWL 2 profile. For this study, we used the TBox of \textit{OWL2Bench1-DL}, expressed in the OWL ``DL''  profile. We generated a relatively smaller ABox than those in the \textit{OWL2Bench} dataset as our work aims to assess the reasoning capabilities of neurosymbolic systems rather than their scalability. Our ABox generation for this ontology was restricted to the classes \texttt{University}, \texttt{Department}, \texttt{Person}, and \texttt{Course}, along with key object properties such as \texttt{hasDepartment}, \texttt{hasDoctoralDegreeFrom}, \texttt{teachesCourse}, and \texttt{takesCourse}. The final \textit{OWL2Bench} ontology contains $1,841$ axioms in total. Table~\ref{axioms} details the frequency of each axiom type within the dataset.

% https://www.w3.org/TR/owl2-syntax/
\begin{table}[htbp]
    \centering
    \caption{Number of axioms in \textit{Pizza\_100}, \textit{Pizza\_250}, \textit{Family} and \textit{OWL2Bench}.}
    \label{axioms}
    \resizebox{\textwidth}{!}{%
    \begin{tabular}{l|c|c|c|c}
        \toprule
        & Pizza\_100 & Pizza\_250 & Family & OWL2Bench \\
        \midrule
        \textbf{Class Expression Axioms}    &       &         &           & \\
        Subclass Axioms                     & $250$ & $250$   & $9$       & $122$ \\                
        Equivalent Classes                  & $13$  & $13$    & $5$       & $19$ \\
        Disjoint Classes                    & $13$  & $13$    & $9$       & $11$\\
        \hline  
        \textbf{Object Property Axioms}     &       &         &           & \\
        Object Subproperties                & $4$   & $4$     & $20$      & $59$ \\
        Equivalent Object Properties        & $0$   & $0$     & $1$       & $4$ \\
        Disjoint Object Properties          & $1$   & $1$     & $1$       & $1$ \\
        Inverse Object Properties           & $3$   & $3$     & $15$      & $22$ \\
        Object Property Domain              & $3$   & $3$     & $11$      & $62$ \\
        Object Property Range               & $4$   & $4$     & $13$      & $57$ \\
        Functional Object Properties        & $4$   & $4$     & $3$       & $2$ \\
        Inverse-Functional Object Properties& $3$   & $3$     & $0$       & $1$ \\
        Reflexive Object Properties         & $0$   & $0$     & $0$       & $1$ \\
        Irreflexive Object Properties       & $0$   & $0$     & $0$       & $2$ \\
        Symmetric Object Properties         & $0$   & $0$     & $2$       & $2$ \\
        Asymmetric Object Properties        & $0$   & $0$     & $0$       & $1$ \\
        Transitive Object Properties        & $2$   & $2$     & $2$       & $5$ \\
        Role Composition                    & $0$   & $0$     & $4$       & $4$ \\
        \hline  
        \textbf{Data Property Axioms}       &       &         & $0$       & \\
        Data Subproperties                  & $0$   & $0$     & $0$       & $2$ \\
        Equivalent Data Properties          & $0$   & $0$     & $0$       & $1$ \\
        Disjoint Data Properties            & $0$   & $0$     & $0$       & $1$ \\
        Data Property Domain                & $0$   & $0$     & $0$       & $7$ \\
        Data Property Range                 & $0$   & $0$     & $0$       & $1$ \\
        Functional Data Properties          & $0$   & $0$     & $0$       & $3$ \\
        \hline  
        \textbf{Assertions}                 & $0$   & $0$     &           & \\
        Individual Equality                 & $0$   & $0$     & $0$       & $0$ \\
        Individual Inequality               & $0$   & $0$     & $1$       & $0$ \\
        Class Assertions                    & $232$ & $287$   & $3$       & $362$ \\
        Positive Object Property Assertions & $922$ & $1,289$ & $1,337$   & $488$ \\
        Negative Object Property Assertions & $0$   & $0$     & $0$       & $0$ \\
        Positive Data Property Assertions   & $0$   & $0$     & $0$       & $0$ \\
        Negative Data Property Assertions   & $0$   & $0$     & $0$       & $0$ \\
        \bottomrule
    \end{tabular}
    }
\end{table}

%Dataset Split
The majority of prior research has concentrated on ontology completion tasks (i.e., prediction) rather than on ontology reasoning tasks (i.e., inference)~\citep{yang2025languagemodelsontologyencoders}. Ontology or link completion involves identifying plausible relations that enrich the original ontology, as demonstrated in the study by Chen et al.~\citep{owl2vec*}. In prediction tasks, the training, validation, and testing datasets are typically created by randomly splitting the ontology axioms.

In contrast, our goal is to infer knowledge that logically follows from a given ontology. To this end, we adopt a method similar to that of Makni and Hendler~\citep{noise-tolerant}, following their approach to construct our training, test, and validation sets, thereby enabling proper ontology reasoning. Let $G$ denote the original ontology and $I$ the ontology inferred using Pellet reasoner~\citep{SIRIN200751}. We use the Pellet reasoner to compute the complete set of logical inferences over the original ontology, which serves as the ground-truth reference for evaluating the inferred assertions produced by the evaluated systems. Since our approach is unsupervised, the graph $G$ is ultimately added to $G_{train}$, while $I$ is randomly assigned to $G_{train}$, $G_{test}$ and $G_{val}$. The TBox is further added to $G_{test}$ and $G_{val}$, ensuring that the reasoning tasks are based on a shared conceptual framework. 

In practice, obtaining perfectly clean data is often impractical or costly, especially for ontologies derived from unstructured sources. Real-world datasets frequently contain errors, inconsistencies, and irrelevant information. To better reflect these conditions, we introduce noise to $G_{test}$. This enables us to rigorously evaluate the resilience of reasoners under realistic, noisy scenarios and to develop systems that are more robust to the imperfections commonly found in real-world data. Table~\ref{tab:stat_triples} summarizes the number of triples, membership assertions, object property assertions, and remaining triples\footnote{Remaining triples refer to structural triples, e.g., those used for rdf:first/rdf:rest list encodings.} for each dataset split across the four datasets used in this study.

\begin{table}[!ht]
\centering
\caption{Summary of Triples, Membership, Object Property Assertions, and Remaining Triples for \textit{Pizza\_100}, \textit{Pizza\_250}, \textit{Family} and \textit{OWL2Bench}.}
\label{tab:stat_triples}
\resizebox{\textwidth}{!}{%
\begin{tabular}{llrrrrr}
\toprule
Dataset   & Split & Triples & Membership & Object Property Assertions & Remaining Triples \\
\midrule
Family    & Train & 137,856 & 1,694 (1.2\%)   & 134,997 (97.9\%) & 1,165 (0.8\%) \\
Family    & Test  & 28,987  & 310 (1.1\%)     & 28,532 (98.4\%)  & 145 (0.5\%) \\
Family    & Val   & 28,984  & 309 (1.1\%)     & 28,530 (98.4\%)  & 145 (0.5\%) \\
\midrule
Pizza\_100 & Train & 24,150  & 2,180 (9.0\%)   & 20,478 (84.8\%)  & 1,492 (6.2\%) \\
Pizza\_100 & Test  & 4,701   & 364 (7.7\%)     & 4,192 (89.2\%)   & 145 (3.1\%) \\
Pizza\_100 & Val   & 4,698   & 363 (7.7\%)     & 4,190 (89.2\%)   & 145 (3.1\%) \\
\midrule
Pizza\_250 & Train & 30,413  & 2,637 (8.7\%)   & 26,286 (86.4\%)  & 1,490 (4.9\%) \\
Pizza\_250 & Test  & 5,941   & 438 (7.4\%)     & 5,358 (90.2\%)   & 145 (2.4\%) \\
Pizza\_250 & Val   & 5,938   & 437 (7.4\%)     & 5,356 (90.2\%)   & 145 (2.4\%) \\
\midrule
OWL2Bench  & Train & 4,123   & 2,049 (49.7\%)  & 1,296 (31.4\%)   & 778 (18.9\%) \\
OWL2Bench  & Test  & 1,462   & 510 (34.9\%)    & 174 (11.9\%)     & 778 (53.2\%) \\
OWL2Bench  & Val   & 1,459   & 508 (34.8\%)    & 173 (11.9\%)     & 778 (53.3\%) \\
\bottomrule
\end{tabular}
}
\end{table}

\subsection{Metrics, Tasks and Reasoners}

%Metrics
We used Mean Reciprocal Rank (MRR) and Hits@N to compare the performance of different neurosymbolic reasoners. MRR represents the average reciprocal rank, calculated by taking the reciprocal of the rank (1/rank) of the first relevant item retrieved. Hits@N measures the percentage of positive examples that appear in the top-$k$ ranked predictions.

%Tasks
%classification/subsumption, determining whether one class is a subclass of another based on the logical axioms in the ontology.
To assess how reasoners respond to noise, we focused on specific reasoning tasks: the first involves class assertions (also known as realization or membership), which determine whether an individual belongs to a specific class based on the logical definitions and constraints within the ontology, for example, \texttt{Alice rdf:type Person}. The second task involves object property assertions, that infer new relationships between two individuals in the ontology, such as \texttt{Alice hasSibling Bob}.

%Reasoners
This experimental framework analyzes the impact of noise on reasoning outcomes, as well as to evaluate the performance and robustness of ontology reasoning under different levels and types of noise. For our exploration into neurosymbolic reasoning, we have selected state-of-the-art neurosymbolic reasoners such as \textit{Box2EL}~\citep{box2el} and \textit{OWL2Vec*}~\citep{owl2vec*}. This work used the implementation of these methods provided by the mOWL library~\citep{mowl}. In addition, we employed a purely neural approach based on Relational Graph Convolutional Networks (R-GCN)~\citep{rgcn}, implemented using the torch-geometric library~\citep{fey2019fastgraphrepresentationlearning}.