Reviewer has chosen to be Anonymous

Overall Impression: Weak

Content:
Technical Quality of the paper: Weak
Originality of the paper: Yes, but limited
Adequacy of the bibliography: Yes

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Limited
Organization of the paper: Satisfactory
Level of English: Satisfactory
Overall presentation: Good

Detailed Comments:

# Summary

The paper looks at incorporating LLMs in various Ontology Engineering Methodologies (OEMs) for constructing an ontology for patient care tasks like monitoring and alerting in Parkinson’s disease. The paper compares classes against a gold standard ontology, Wear4PDmove. The various OEMs described in the paper range from fully LLM based generation using One shot (OS) and Chain of Thought (CoT) methods and then collaborative processes involving varying levels of human input during the initial construction and refinement phase. A role playing approach was also tested where three roles, Knowledge Worker, Domain Expert and Knowledge Engineer, are defined, and the LLM and human can play different roles. The paper argues through its results how human collaboration and iterative refinement improve the quality of the derived ontology.

# Recommendation

Reject

# Reasons

## Strengths

The idea to incorporate roles based on OEMs in the process of collaborative ontology construction seems interesting and certainly has utility. Also, studying varying levels of human assistance in the process can provide insight into where human contribution is key.

## Weaknesses

The paper is essentially constructing ontologies using various kinds of prompts and the novelty of such work is questionable. The collaborative OEMs proposed here may be interesting, but it is entirely based on various ways to prompt the LLM and incorporate human feedback to varying levels. It can essentially be looked at as guided prompting, which is what we all do when the problem is not trivial enough for the LLM to solve in one or two steps. The paper relies on the LLM's ability to simply generate a valid ontology in the Turtle format. It may have looked at other ways to extract classes and relations in a text form. This would have allowed the authors to try different LLMs and have a fair comparison.

Other issues which are present in the paper have to do with evaluation. There appears to be no comparison of the relations identified in the generated ontology with the gold standard, only metrics based on class discovery seem to be covered.

Reviewer has chosen to be Anonymous

Overall Impression: Weak

Content:
Technical Quality of the paper: Average
Originality of the paper: Yes, but limited
Adequacy of the bibliography: No

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Limited
Organization of the paper: Needs improvement
Level of English: Satisfactory
Overall presentation: Weak

Detailed Comments:

Strengths

S1. Clinically meaningful focus on a PD monitoring/alerting ontology. (See C6)
S2. Useful comparative angle: four methodologies to test LLM-driven ontology creation.
S3. Interesting results and general analysis and domain
S4. Generally clear

Weaknesses

W1. Intro claims aren’t backed by LLM-relevant sources; “inadequate reasoning abilities” is unsupported. (See C1)
W2. Related work is thematically unfocused and incomplete (BERT vs LLMs, GPT vs ChatGPT, coverage stalls at 2023). (See C2)
W3. Methods lack specificity (one-shot/CoT not referenced; prompt sequence unclear; figure typo; RQs not set up in Intro). (See C3)
W4. Structure of the paper can be refined for clarity, broken down into multiple parts. (See C4)
W5. Reproducibility gaps: broken GitHub link and no code in the repo. (See C5)

Detailed comments

C1. Introduction (claims, sourcing, and clarity). The sources used to connect LLMs with ontological frameworks are not directly relevant to the paper’s claim; for instance, the second sentence cites a foundational ontology paper to support a trend about LLM-driven ontology construction, which doesn’t fit. Please replace with citations that fit the narrative. Also, “inadequate reasoning abilities” (line 21) needs grounding: inadequate relative to what, and with which source?

C2. Related work (scope, themes, and currency). This is the main weakness of the work. The literature review should be restructured by clearer themes. Right now it jumps from BERT-based ontology generation to LLM uses in other KE tasks (e.g., entity linking) without separation, and it does not discuss the important distinction between GPT (model) and ChatGPT (product/interface). Coverage effectively stops at 2023, yet there’s already an ESWC 2024 citation where multiple papers on ontology generation were published, and the area has since expanded. I recommend broadening the review via Google Scholar with filters from 2021 using terms such as “ontology generation” and “ontology learning,” following up related works, and incorporating representative examples (e.g., the first ones that appear with these filters: Lippolis et al., 2025; Qiu et al., 2024; Aggarwal et al., 2026; Elnagar et al., 2022). Many of these automate steps of existing OEMs and some are domain-specific (CS, life sciences, music, etc.), so they’re important to include. Also, because you use competency questions (CQs) in the methodology, briefly introduce what CQs are in Related Work before relying on them later.

C3. Research methodology (naming, sequencing, and figure). Please explicitly reference the one-shot and Chain-of-Thought methodologies you use. Provide a concise overview of the sequential prompts (what comes first, how you refine, and how CQs feed back), so the overall flow is unambiguous. The research questions appear in the figure but are not outlined in the Introduction: bring them forward so readers know what you’re answering before entering Methods.

C4. Experimental setup and evaluation could benefit from a separate section. Right now these details are embedded in Methods and are easy to miss. Please create a clearly labeled section that states who the human participants are, whether/which API is used, and other core setup details. Isolate the evaluation procedures there so readers can quickly understand how you assessed the hypotheses and methodologies. Limitations could also be a separate section from the conclusions.

C5. Reproducibility and artifacts. The GitHub link includes an extra trailing “t”, which breaks the URL. Remove it so the link opens correctly. Also, the repository appears to contain no code; please add the implementation if it is possible, along with usage guidelines, so the community can verify and reuse your approach.

C6. Overall contribution and domain fit. The approach and results are interesting, and the PD domain makes the work especially valuable. With a revision of all the areas of improvement identified, the paper’s practical impact will come through much more strongly.

Minor issues (typos & small fixes)

Line 16 (and onwards): quotation marks inverted.

Figure caption: “papers” → “paper’s”.

GitHub: remove the extra trailing t in the URL.

Line 25: “confirme” appears to be a typo (for "confirm")

Reviewer has chosen to be Anonymous

Overall Impression: Bad

Content:
Technical Quality of the paper: Bad
Originality of the paper: Yes, but limited
Adequacy of the bibliography: No

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Bad
Organization of the paper: Poor
Level of English: Satisfactory
Overall presentation: Bad

Detailed Comments:

This paper performs human-LLM collaboration in ontology development. This work has not been done yet, and it is new. Citing:
“The primary objective is to determine whether LLMs alone can create comprehensive ontologies and, if not, whether human-LLM collaboration can achieve this goal.”

Weaknesses:
Related work -> all cited papers are before 2024, without properly indicating where your research is standing, it is hard to accept this paper. There has been a lot of work in 2024 and 2025 on ontology generation and evaluating them. I recommend searching this in Google Scholar: put a filter to find papers after 2023, and search for “ontology generation”, “ontology evaluation”, “ontology assistant with LLMs” and also look at their related work.

Methodology section -> In line 21, you wrote “In a one-shot scenario, the model lacks multiple learning examples and must accomplish the task with minimal context. This is a straightforward approach where the model uses its pre-trained knowledge to infer the most likely answer.” However, this is very similar to the definition of zero-shot prompting, which has no example in the prompt, while one-shot has one example of the task. Also, Cot says, “For the purposes of this paper, CoT refers to a methodological approach where the OS is segmented into two sequential prompts. This segmentation allows for a structured progression in the reasoning process, whereby each prompt is designed to focus on a specific element of the overall task.” This is not chain of thoughts, it is decomposed prompting, look up 'sub-task decomposed prompting”.

Methodology section -> Page 5, line 30 -> Comparing ontology against a gold standard has been explored in the literature. The problem with it is that if LLM finds better or different names for classes, it is penalised. Also, you need to evaluate your ontology not just by F1 (and precision or false positives) and OOPS!, you need to include functional evaluation of the ontology to see if the generated ontology is useful for a real task or not. Note: i read “ similarity matching” but you should consider that there are several ways of modelling practices. By introducing more classes, which are proposed in ontology design patterns to have a flexible ontology, your metric gives a lower score. The GitHub link was not working. “This approach aims to provide a comprehensive evaluation of the LLM’s performance” This part “comprehensive evaluation” is not true at all.

Prompts -> On page 6, when I read the prompts, I did not see any input to the problem. Some work generates ontologies based on competency questions. In the prompts, there was no mention of any requirements. I put them into ChatGPT, and it gives me an ontology. It is strange, since it is based on nothing.

Hypothesis 4 -> It is not clear how a human interacts with LLM to generate an ontology. “autonomously develop ontologies using NLP and knowledge extraction, under human expert supervision” explains what is the role of the supervisor. Line 43 and onwards, is not clear what a supervisor or expert does. “iterative refinement” is not explained.

Section 4 line 20 mentions ChatGPT-3.5. Is it done in the chat interface? If so, the paper should not mention GPT-3.5 and mention ChatGPT-3.5 instead. Since GPT-x and ChatGPT-x are different. Also, section 4.1 has almost all numbers mentioned in the table. It should summarise the important ones.

Table 3 -> recall is 122% and F1 is 110%. There are some misscalculations.

The paper does not have conclusion.

Typos:
Line 4 of the abstract: The dot is connected to the next sentence.
“lowchart”, “confirme”, “47,6%” is not consistent with “47.6%”
GPT3 -> GPT-3, GPT3.5 -> GPT-3.5
Try to give your paper to ChatGPT and ask for proofreading and spell warnings.