Abstract: The intelligence of magnesium smelting equipment requires a fast and accurate digital model as the basis. Providing a digital model through "multifield coupled numerical simulation, fast-prediction model, and intelligent optimization design" is an important method. This paper, based on the discrete element method (DEM), constructs a natural and disordered stacking model of the briquette layer in the retort of magnesium production. Through single-factor numerical simulation and multifactor orthogonal simulation design, 1894 valid data points were obtained. Then, four machine learning models were trained, tested, and validated on this dataset. Finally, the optimal machine learning model was obtained and used to predict the magnesium reduction rate under the new design conditions. The following conclusions were obtained. (1) In response to the limitations of traditional ordered stacking models, a natural disordered stacking model of the briquette layer is established based on the DEM. The free-fall motion of briquettes is simulated to obtain the true porosity and contact characteristics. The bridging treatment is used to solve the problem of grid division, and a coupled model of the heat transfer reduction reaction in the briquette layer is constructed. The natural disordered model results show a reaction time of approximately 10 h, which is closer to actual field measured data. The simulation results also show the reduction process on the briquette layer in the retort under high temperature and vacuum conditions. (2) Single factor simulation results on the disordered stacking model show that factors such as briquette size, briquette stacking layer thickness, and retort wall temperature have a significant impact on the magnesium reduction rate of the retort. Keeping other factors constant, increasing briquette size, decreasing briquette stacking layer thickness, or increasing retort wall temperature can significantly reduce the reaction time. This paper comprehensively considers factors including the simulation results, magnesium production for one retort, magnesium production rate, and the service life of retort steel. It provides the optimal range of values for each factor: briquette diameter of 20–30 mm, briquette stacking layer thickness of 100–140 mm, and small fluctuations of approximately 1473 K for the retort wall temperature. (3) The results of four classic machine learning models for predicting the reduction rate of magnesium for retorts show that eXtreme gradient boosting (XGBoost) has the highest prediction accuracy (R² = 0.997) and smallest prediction deviation for new design cases. The random forest model had a large prediction error (R² = 0.8654) due to the low discrete values of features such as the briquette stacking layer thickness in the dataset. The SHapley additive exPlanations analysis of the XGBoost model shows that the thickness of the briquette stacking layer had the greatest impact on the reduction rate, followed by the temperature of the retort wall and diameter of the briquette, while the diameter of the inner tube had the smallest impact. The application of the XGBoost model is a reliable prediction tool for the optimization of the magnesium-production retort structure and process parameters, while also compensating for the long prediction time of traditional simulation models.