Abstract: Abnormal states of cotter pins in tower crane steel-structure joints can undermine connection reliability and further threaten operational safety. To address the challenges in UAV inspection images, including extremely small target size, complex metallic backgrounds, and the difficulty of quantitatively characterizing pin states, a two-stage cascaded vision framework based on localization–segmentation–geometric assessment is proposed. In the localization stage, a PEAB network is developed on the basis of YOLOv12 by integrating a P2 shallow detection branch, an EMA attention mechanism, an ADown downsampling module, and a BiFPN feature fusion structure, so as to accurately extract the region of interest (ROI) of cotter pin assembly areas. In the segmentation stage, taking DeepLabv3+ as the baseline, a receptive-field-reconstructed MobileNetV3 is adopted as a lightweight backbone, while the CA module is introduced into both the ASPP module and the shallow feature path. Meanwhile, a CEDiceBoundaryLoss is designed to strengthen boundary constraints for the slender tail region of cotter pins. On this basis, geometric features, including connected-component area, skeleton structure, and tail-branch opening angle, are extracted from the predicted masks to achieve automatic state identification of cotter pins. A dataset containing 3,000 original tower crane inspection images was established, from which 6,420 ROI samples of cotter pin assembly regions were further extracted for validation. Experimental results show that the proposed PEAB localization network achieves an mAP@0.5 of 94.30%, while the improved segmentation model improves mIoU and mF1 by 4.85 and 3.90 percentage points, respectively, compared with the original DeepLabv3+. With relatively low model complexity, the proposed method enables micro-target localization, fine-grained segmentation, and quantitative state assessment of cotter pins under complex working conditions, providing technical support for intelligent UAV-based inspection and safety evaluation of tower crane joint connections.