Counting people inside a region-of-interest in CCTV footage with deep learning

PeerJ Comput Sci. 2022 Sep 22:8:e1067. doi: 10.7717/peerj-cs.1067. eCollection 2022.

Abstract

In recent years, the performance of people-counting models has been dramatically increased that they can be implemented in practical cases. However, the current models can only count all of the people captured in the inputted closed circuit television (CCTV) footage. Oftentimes, we only want to count people in a specific Region-of-Interest (RoI) in the footage. Unfortunately, simple approaches such as covering the area outside of the RoI are not applicable without degrading the performance of the models. Therefore, we developed a novel learning strategy that enables a deep-learning-based people counting model to count people only in a certain RoI. In the proposed method, the people counting model has two heads that are attached on top of a crowd counting backbone network. These two heads respectively learn to count people inside the RoI and negate the people count outside the RoI. We named this proposed method Gap Regularizer and tested it on ResNet-50, ResNet-101, CSRNet, and SFCN. The experiment results showed that Gap Regularizer can reduce the mean absolute error (MAE), root mean square error (RMSE), and grid average mean error (GAME) of ResNet-50, which is the smallest CNN model, with the highest reduction of 45.2%, 41.25%, and 46.43%, respectively. On shallow models such as the CSRNet, the regularizer can also drastically increase the SSIM by up to 248.65% in addition to reducing the MAE, RMSE, and GAME. The Gap Regularizer can also improve the performance of SFCN which is a deep CNN model with back-end features by up to 17.22% and 10.54% compared to its standard version. Moreover, the impacts of the Gap Regularizer on these two models are also generally statistically significant (P-value < 0.05) on the MOT17-09, MOT20-02, and RHC datasets. However, it has a limitation in which it is unable to make significant impacts on deep models without back-end features such as the ResNet-101.

Keywords: Convolutional neural networks; Deep learning; People counting; Region-of-Interest.

Grants and funding

This study is funded by the Directorate of Research and Community Service, Directorate General of Research and Development, Indonesian Ministry of Research, Technology and Higher Education (Grant No. 234/E4.1/AK.04.PT/2021) as a part of the 2021 Penelitian Terapan Unggulan Perguruan Tinggi Research Grant. The hardware resources (NVIDIA Tesla P100) were provided by NVIDIA—BINUS AIRDC for the experiment in this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.