Toward human-level concept learning: Pattern benchmarking for AI algorithms

Andreas Holzinger; Anna Saranti; Alessa Angerschmid; Bettina Finzel; Ute Schmid; Heimo Mueller

doi:10.1016/j.patter.2023.100788

Toward human-level concept learning: Pattern benchmarking for AI algorithms

Patterns (N Y). 2023 Jul 5;4(8):100788. doi: 10.1016/j.patter.2023.100788. eCollection 2023 Aug 11.

Authors

Andreas Holzinger^{1

2}, Anna Saranti^{1

2}, Alessa Angerschmid^{1

2}, Bettina Finzel³, Ute Schmid³, Heimo Mueller²

Affiliations

¹ Human-Centered AI Lab, University of Natural Resources & Life Sciences Vienna, Vienna, Austria.
² Medical University Graz, Graz, Austria.
³ University of Bamberg, Bamberg, Germany.

Abstract

Artificial intelligence (AI) today is very successful at standard pattern-recognition tasks due to the availability of large amounts of data and advances in statistical data-driven machine learning. However, there is still a large gap between AI pattern recognition and human-level concept learning. Humans can learn amazingly well even under uncertainty from just a few examples and are capable of generalizing these concepts to solve new conceptual problems. The growing interest in explainable machine intelligence requires experimental environments and diagnostic/benchmark datasets to analyze existing approaches and drive progress in pattern analysis and machine intelligence. In this paper, we provide an overview of current AI solutions for benchmarking concept learning, reasoning, and generalization; discuss the state-of-the-art of existing diagnostic/benchmark datasets (such as CLEVR, CLEVRER, CLOSURE, CURI, Bongard-LOGO, V-PROM, RAVEN, Kandinsky Patterns, CLEVR-Humans, CLEVRER-Humans, and their extension containing human language); and provide an outlook of some future research directions in this exciting research domain.

Keywords: artificial intelligence; benchmarks; concept learning; diagnostic datasets; pattern analysis.

Publication types

Review