Train & Debug ML Models for Responsible AI – Join the #AISkillsChallenge!

Welcome to the eleventh post in my This Week In AI News series. Want to keep up with my weekly posts? Now there’s a tag you can follow:

The 3 Resources To Know This Week

| Register Here: April 10, 2024

Earlier in this series, I shared this post which focused on three things with respect to Responsible AI:

• The 6 guiding principles for Responsible AI
• The Responsible AI Developer Hub for self-guided learning.
• 3 Responsible AI Workshops for hands-on skilling.

Today, I want to dive deeper into one specific workshop and tell you about an ongoing AI Skills Challenge where we will be running a live training session (online, free) where you can get hands-on experience with the Responsible AI Dashboard guided by us! But first, let’s talk about the AI Skills Challenge and why it matters.

1 | The Microsoft Learn #AISkillsChallenge

The Microsoft Learn #AISkillsChallenge is a 9-part series of livestreamed training sessions that cover the gamut of topics from generative AI to LLM Ops. Visit this link for registration pages and details for each episode – or view the tweet below for a quick video tour.

2 | Register to Learn Live

The Responsible AI episode is scheduled for April 10 – just click the link below to go directly to the registration page for this event.

| Register Here: April 10, 2024

Then join us on the livestream for a hands-on walkthrough of this Microsoft Learn module. All you need is your laptop and a modern browser – the module comes with an Azure sandbox so we can dive straight into learning by doing, with no other setup overheads.

The session will be 90-minutes long, with time for live Q&A. While we do recommend following along with us, you should be able to continue working on the training even after the livestream is done. Read on for more details on what you will learn, and the process involved.
 

3 | What We’ll Be Doing

We’ll learn how to train a machine learning model on Azure, then debug it to ensure that it applies responsible AI principles in practice.

We’ll build our model based on the UCI Diabetes dataset which has a decade of data from 130 US hospitals, with 50+ features recorded including attributes like patient number, race, gender, age, admission type, time in hospital, number of medications, number of outpatient, inpatient, and emergency visits in the year before hospitalization, etc.

Our model can then predict the likelihood of a patient needing to be readmitted to hospital within 30 days – allowing a physician or hospital to them make decisions related to patient care.

4 | Why Debug Models?

The reality is that model predictions can have real-world implications when used for decision-making. Models are rarely 100% accurate – so humans (doctors, admins) need to make their decisions based on metrics like accuracy and confidence. Poor prediction can result in sick people being denied critical care, or healthy ones being billed for unnecessary hospitalizations.

With the Responsible AI Dashboard, we can learn to debug models interactively using different components (error analysis, model performance, data biases, model interpretability) that can be analyzed across different cohorts (subsets of data matching a common set of attributes) to ensure that predictions are made fairly, correctly, and consistently for everyone.

5 | How We Debug Models

The training session consists of two main sections, each with code and documentation in the form of Jupyter Notebooks for convenience.

1. In the first section, we’ll learn to setup an Azure Machine Learning Studio workspace, upload our test and training data, setup a training job, and register our trained model on Azure to expose a prediction endpoint for applications.

2. In the next section, we’ll create a Responsible AI Dashboard for the deployed model on Azure, selecting the required subset of components from our Responsible AI toolbox. The result is an interactive dashboard that allows us to create cohorts based on various criteria (e.g., by error rates) and use them to analyze the model for fairness, performance and other responsible AI practices.

6 | What is the Responsible AI Toolbox?

Some of you may have heard of The Responsible AI Toolbox repository maintained by Microsoft Research, and wondered how that relates to the Responsible AI Dashboard.

The toolbox refers to the suite of components for operationalizing Responsible AI end-to-end, many of which are open-source projects from Microsoft or the community. See the figure below to understand the components involved.

The dashboard is a single pane of glass (unified interface across tools) that helps you easily flow through different stages of model debugging (identification, diagnosis, and mitigation) and decision-making. See the figure below to get a sense of how the toolbox components are organized into dashboard views for model debugging (left) and decision making (right).

There is a lot more to learn about Responsible AI in the context of end-to-end development workflows for predictive AI and generative AI applications. By joining us in this training, you should be able to jumpstart your learning journey with a hands-on exercise that can help translate abstract concepts into real-world practice.

| Register Here to join us: April 10, 2024

7 | Related Resources

Here are the three main resources for this post:

And don’t forget to follow the Microsoft Azure publication or the individual author pages, for more updates.