The rise of machine learning around the globe in fields like medicine, education, employment, credit lending, and criminal sentencing has the potential to reflect and reinforce societal biases at large scale through the models deployed. While fairness concerns are multifaceted, technical evaluations and improvements of models are a critical aspect of a developer’s role. And, for these considerations to truly scale, they must integrate into existing processes. In particular, we focus on seamlessly integrating known technical methods with existing libraries used for the training, evaluation, and deployment of models. To showcase the suite of tools built in TensorFlow, we present an interactive case study demo in conjunction with Conversation AI, an ML research initiative to make online conversations more inclusive.