A comparative analysis of deep learning-based solutions for image-based hcaptcha challenges.

Abstract

Completely Automated Public Turing Tests to Tell Computers and Humans Apart (CAPTCHA) play a crucial role in securing online platforms from automated bot attacks. Among various CAPTCHA types, image-based CAPTCHAs, such as hCAPTCHA, present unique challenges such as disruption, accessibility issues, browser compatibility and usability concerns that have not been extensively addressed using deep learning-based object detection models. Moreover, in the software development domain, continuous integration and automated pipelines can be disrupted because quality assurance teams struggle to automate tests when CAPTCHA challenges are present. This study presents a comparative analysis leveraging deep learning models, including You Only Look Once (YOLO) (specifically YOLOv3 and YOLOv10) and Real-Time Detection Transformer (RT-DETR), to automate solving hCAPTCHA challenge. While approaches leveraging Residual Networks (ResNet) and YOLO have demonstrated promising results in other domains, there remains a need for more efficient and accurate solutions specifically for hCAPTCHA. This research, therefore, explores the application of these advanced deep learning models for solving hCAPTCHA challenges, presenting the implementation of YOLOv3, YOLOv10 and RT-DETR for this purpose, with a focus on accuracy, inference speed, and computational efficiency. Web scraping was used to systematically collect a relevant dataset on standard hCAPTCHA challenges spanning ten distinct categories—bicycle, boat, bus, car, crosswalk, motorcycle, traffic light, train, truck, and van—resulting in a dataset of 13,000 images used for training, testing, and validation. To ensure robustness, the models were evaluated using both a self-curated dataset and the widely used COCO dataset for comparison. The self-curated dataset through web scraping led to significantly improved performance, highlighting the benefits of domain-specific data over generic COCO datasets. The models were tested against more than 600 image-based CAPTCHA challenges per day via a web-integrated hCAPTCHAsystem. The experiment was conducted on a machine with an Intel Core i5-4300U processor and 16 GB of RAM, demonstrating feasibility in resource-constrained settings without GPU acceleration.