In order to solve the problems that the traditional manual methods have great interference and low efficiency on the classification and rating of scrap, a deep learning network model for scrap classification and rating is proposed based on the squeeze -exception (SE) attention mechanism, and the collected images of scrap unloading process are trained and verified. Firstly, a 1:3 physical model for scrap quality inspection was built, and high-resolution visual sensors were used to simulate and collect the morphological characteristics of different scrap under the scene of truck unloading scrap; Then, cross phase local network is used to extract features from the collected scrap images, spatial pyramid structure is used to solve the problem of feature loss, and attention mechanism is used to pay attention to the correlation between channels; Finally, the model is trained and validated in two datasets containing eight label classifications. Experiments show that the model can effectively carry out automatic rating and judgment for different grades of scrap. The rating accuracy is 94% and the map is 88.8%. In terms of accuracy, it can completely surpass the traditional manual quality inspection method, and solve the fairness problem of quality evaluation in the process of scrap warehousing.