XGBoost is considered a state-of-the-art model for regression, classification, and learning-to-rank problems on tabular data. Unfortunately, tree-based ensemble models are notoriously difficult to explain, limiting their application in critical fields. Techniques like SHapley Additive exPlanations (SHAP) and Explainable Boosting Machine (EBM) have become common methods for assessing how much each feature contributes to the model prediction. This talk will introduce SHAP and EBM, explaining the theory behind their mechanisms in an accessible way and discussing the pros and cons of both techniques. We will also comment on Python snippets where SHAP and EBM are used to explain a gradient boosting model. Attendees will walk away with an understanding of how SHAP and EBM work, the limitations and merits of both techniques, and a tutorial on how to use these methods in Python, courtesy of the "shap" and "interpret-ml" packages.