Reviewer has chosen to be Anonymous

Overall Impression: Good

Content:
Technical Quality of the paper: Excellent
Originality of the paper: Yes
Adequacy of the bibliography: Yes, but see detailed comments

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Good
Organization of the paper: Satisfactory
Level of English: Satisfactory
Overall presentation: Excellent

Detailed Comments:

The paper is a survey/review of methods for explaining concepts in neural networks. These methods are classified as neural-level and layer-level explanation methods. Within these two broad categories the methods are further classified into methods that exploit similarities and causal relationships between concepts and activations, post-hoc methods such as TCAV and probing, and concept bottleneck models.

The paper is well-written and easy to read. It overviews the aforementioned methods in a clear way.

I do not have specific comments apart from pointing out the recent work in [1] that might be related.
I recommend acceptance for its publication in the inaugural special issue.

[1] Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Mateo Espinosa Zarlenga, Lucie Charlotte Magister, Alberto Tonda, Pietro Lió, Frederic Precioso, Mateja Jamnik, and Giuseppe Marra. 2023. Interpretable neural-symbolic concept reasoning. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), Vol. 202. JMLR.org, Article 76, 1801–1825.

Minor points:

- pp.2, line 41: the two probabilities P(d|E) and log P(d) -> the two probabilities P(d|E) and P(d)
- pp.5, line 42: (see Figure 3.3) -> (see Figure 4)
- pp.6, line 29: please define the operator \circ
- pp.7, line 25: paper -> paper.

Reviewer has chosen not to be Anonymous

Overall Impression: Good

Content:
Technical Quality of the paper: Good
Originality of the paper: Yes
Adequacy of the bibliography: Yes

Presentation:
Adequacy of the abstract: Yes
Introduction: background and motivation: Good
Organization of the paper: Satisfactory
Level of English: Satisfactory
Overall presentation: Good

Detailed Comments:

The paper reviews recent methods for inspecting concepts learning inside the neural networks and categorizes these approaches into two types: neuron-level and layer-level. For the neuron level, the paper reviews methods driven by concept-activation similarities and causal intervention on neurons. For the layer level, it discusses concept activation vectors, probing, and bottleneck models, showing their weaknesses and potential applications in inspecting input and model debugging.

Strength:
The paper covers a broad range of existing methods. For Each category, the discussion includes methods from early to most recent and forms a line to illustrate their differences and improvement.

The paper discusses each category in detail. It presents each method in general, then shows their weakness and further application in other domains.

The topic of the NN-learned concept is crucial for neuron-symbolic methods. It can be a link between dense neuron networks and symbolic reasoning.

Weakness:
The paper does not conclude a possible direction for future concept learning or explaining.

Some reviewed methods can also be viewed as explainable AI. The paper can discuss the relations between concept explaining and other domains.