Robot integrated joints are extensively employed in the fields of medical and collaborative robots, and their friction characteristics are the crucial factors that affect robot performance. To improve the accuracy of the friction model, a hybrid friction modeling method integrating the mechanistic model and ensemble learning is proposed herein. First, the mechanistic model is developed based on the comprehensive analysis of various factors influencing joint friction and its periodic fluctuation. Assuming that the viscous friction and flow rate of lubricants are linear, the Stribeck model is used to describe the relationship between friction torque and motor velocity. Meanwhile, the Fourier series is used to model the periodic fluctuation of joint friction with the motor angle. The main harmonic components of the fluctuating friction torque are concentrated in the low frequency range. Considering the high frequency components will increase the complexity of the model, while the accuracy improvement of the parameterized model is limited. Therefore, the high-frequency components of the fluctuating friction torque are neglected. Moreover, considering the power function rule between external load and Coulomb friction, a simplified parametric mechanistic model is developed to describe the changes in joint friction under external load. Second, for the nonlinear residual error of the mechanistic model caused by linear hypothesis and neglecting higher-order terms, a residual compensation model based on eXtreme Gradient Boosting (XGBoost) is proposed. By adopting the boosting ensemble learning strategy, the generalization ability of the residual compensation model is enhanced. The input of the XGBoost model includes the independent variables of the parametric mechanistic model, namely the motor angle, motor velocity, and external load. The output is the difference between actual friction torque and its predicted value by the mechanistic model. Further, the Bayesian optimization method is employed for the optimization of the hyperparameters of the XGBoost model to improve model accuracy and training efficiency. Finally, a series of experiments are performed based on a joint-friction measurement platform and industrial robots to verify the effectiveness of the proposed method. Compared with the parametric mechanistic model, the prediction error of the proposed hybrid model is mainly maintained within ±0.005 N·m, and the peak error, mean absolute error (MAE), and root mean square error (RMSE) are reduced by more than 60%. Comparison experiments with backpropagation neural networks, support vector machines, and long short-term memory neural networks reveal that the nonlinear fluctuation of friction residuals can be well predicted by the proposed method and the MAE and RMSE are reduced by more than 40%. Thus, the proposed XGBoost-based residual compensation model exhibits stronger feature extraction ability and can effectively improve the accuracy of the hybrid friction modeling method.