\section{Introduction}\label{sec:introduction}
\begin{comment}
- Introduce the concept of neurosymbolic ontology reasoners and the importance of benchmark datasets.
- Highlight the specific focus of our paper on designing a benchmark dataset with introduced noise.
- State the objectives of the research and provide an overview of the paper's structure.
\end{comment}

%Background and context: NeSy, Reasoning
%Explain the difference between neuro, symbolic and neurosymbolic reasoners + limitations of neurosymbolic reasoners 
Neurosymbolic computing has emerged as a prominent area of Artificial Intelligence in recent years, combining the robust learning capabilities of neural networks with the reasoning capabilities and interpretability of symbolic systems~\citep{garcez2015neural, YU2023105}. Symbolic reasoners rely on formal logic, rules, and knowledge bases, such as ontologies to make inferences. An ontology is a formal and explicit specification of a shared conceptualization of a domain. It describes the concepts, categories, relationships, and rules that structure knowledge within that domain, facilitating common understanding and interoperability among systems and people~\citep{gruber1993ontology}. Ontologies distinguish between the Terminological Box (TBox), which defines classes, along with their relationships and restrictions, and the Assertional Box (ABox), which contains assertions about individuals such as property assertions or class memberships. Ontologies are often reliable and interpretable, offering traceable mechanisms for their inferences. However, they are sensitive to noise and struggle to handle incomplete or ambiguous data. Symbolic reasoners could fail to perform when faced with missing knowledge or errors in their knowledge base. Moreover, their reliance on a large number of predefined rules and axioms limits their scalability~\citep{10148662, sota2}. In contrast, neural reasoners leverage deep learning models, which can generalize from large volumes of data, are robust to noise. However, their primary limitation lies in their lack of interpretability~\citep{10.1007/s10489-020-02165-6} and handling tasks that require explicit logic or when dealing with rare or unseen examples. Neurosymbolic reasoners can address these shortcomings inherent in each paradigm~\citep{YU2023105}. By integrating symbolic reasoning with neural systems, these reasoners achieve a trade-off between interpretable logical reasoning and the scalable, data-driven capabilities of neural networks~\citep{neurosymbolic_AI, sota2}. Despite these advantages, neurosymbolic systems face unique challenges, particularly in the incorporation of domain ontologies while ensuring resilience against the noise and uncertainty that characterize real-world data.

%Why noise? Noise injection: TBox, ABox -> see methodology: James Hendler
%Why ABox noise? most of the time, errors are in the ABox (especially KGs), real-life scenarios + citation
%Why is it important to handle noise?
Noise in ontologies encompasses various forms of disturbance that can affect their integrity, coherence, and interpretability. Makni et. al.~\citep{noise_taxonomy}, presented a Semantic Web noise taxonomy, which distinguishes between two main categories of noise: TBox noise and ABox noise (i.e., propagable and non-propagable). TBox noise is the type of noise that resides within the ontology, such as in the class hierarchy, or domain and range properties. This type of noise will affect the inference over the entire dataset. While ABox noise is about corrupting an existing triple in an ontology by changing one of the triples' resources. This either changes the inference graph (i.e., propagable noise) or does not have any impact on the inference graph (i.e., non-propagable noise). 

%Research Gap: Importance of Benchmark, Lack of Benchmarch for NeSy reasoners
This work aims to develop \textit{NSORN} (Neurosymbolic Ontology Reasoning with Noise), a framework designed to introduce noise into ontologies and create challenging benchmark datasets to test the effectiveness of neurosymbolic reasoners in handling noise. While numerous benchmark datasets exist for various AI tasks, such as image classification (i.e., MNIST~\citep{deng2012mnist}, CIFAR-10 and CIFAR-100\footnote{\url{https://www.cs.toronto.edu/~kriz/cifar.html}}), natural language processing (i.e., GLUE~\citep{wang2019glue}), and reinforcement learning (i.e., OpenAI Gym~\citep{1606.01540}), there is limited work on benchmark frameworks specifically tailored to evaluating neurosymbolic reasoning under noisy conditions. Such benchmark frameworks can support more systematic and comparable evaluation in this area~\citep{raji2021ai}. Existing neurosymbolic benchmark datasets are predominantly designed to assess the performance of symbolic reasoners~\citep{gunjan}. Furthermore, most reasoning systems are evaluated using various publicly available ontologies~\citep{semrec2021, semrec2022, semrec2023}, which do not address the unique challenges of neurosymbolic integration. Similarly to previous work \citep{liu2024knowledge}, we developed three techniques to introduce noise into ontologies: logical, statistical, and random noise. 

\textbf{Random noise} serves as a baseline, representing data-agnostic, unpredictable errors that may arise accidentally in real-world ontologies. We simulate this by corrupting existing triples—replacing either the subject or the object with a random entity. This allows us to probe the robustness of reasoning processes against general perturbations that do not depend on the underlying data. \textbf{Statistical noise} is generated adversarially using Graph Neural Networks (GNNs), reflecting low-probability links that emerge from predictive uncertainty or bias in automated knowledge graph construction. Although synthetic, this form of noise models realistic mistakes produced by machine-learning systems, and mirrors the types of errors that error-detection models are typically asked to identify during KG construction. \textbf{Logical noise} captures violations of semantic constraints, such as disjointness axioms or domain and range restrictions. Because many real-world ontology errors stem from semantic confusion rather than random corruption, this type of noise directly stresses the logical structure of the ontology and provides a more targeted challenge to reasoning systems. By combining these three types of noise, we aim to approximate a spectrum of potential error patterns observed in real-world ontology and knowledge graph construction, from accidental and statistically plausible mistakes to deliberate logical conflicts.

%Research question or objective
In this work, we explore the following research questions: how noise in ontologies can be characterized operationally, how controlled noise can be introduced into these structures, and how the impact of such perturbations can be evaluated in neurosymbolic reasoners. By exploring these questions, we aim to develop a framework for generating noisy benchmark datasets. This framework will facilitate the assessment of reasoners' robustness and effectiveness in handling noisy data, contributing toward more systematic evaluation practices in neurosymbolic AI~\citep{singh_benchmarking_iswc23, Singh_Mutharaju_Bhatia_Tommasini}.

We apply two neurosymbolic reasoners and one purely neural approach based on Graph Neural Networks to datasets with different noise levels to demonstrate their limitations in handling various types of noise. We then evaluate their performance under these noisy conditions. Many existing studies emphasize ontology completion or link prediction tasks, whereas our focus is on evaluating reasoning performance under noisy conditions. The goal of ontology/link completion is to discover plausible relations that complement the original ontology, as was the task performed in the work of Chen et al.~\citep{owl2vec*}. In contrast, our goal is to infer knowledge that logically follows from the given ontology.

%Outline of the paper
The remainder of the paper is organized as follows: the existing literature on neurosymbolic ontology reasoners and benchmark datasets is reported in \secref{sec:related_work}. \secref{sec:methodology} presents the developed framework for injecting random, statistical, and logical noise. \secref{sec:experiments} presents the experimental setup. \secref{sec:results} shows the results of the experiments, including performance metrics and analysis. Finally, \secref{sec:discussion} discusses the strengths and limitations of the designed framework and explores potential extensions or improvements for future research, followed by \secref{sec:conclusion} to conclude our work. The source code of the benchmark is available at \url{https://github.com/jloe2911/NoisyBench} under MIT License.