Watch live: March 8, 2023 | 12:40 – 1:10 pm ET

Measuring MLOps ROI & Team Productivity

How do we measure MLOps objectives and metrics to capture the effectiveness of an MLOps department?

For DevOps, there’s some standard set of metrics, e.g. DORA metrics.
What the common metrics that should be used for MLOps?

For most teams and companies, there isn’t a common standard. Or it’s dependent on how “platform-y” vs “product-y” the MLOps team is in question.

Should serving should be measured similar to DORA? Are dev and training different games?

How about data enrichment pipelines? Should you measure how many POCs a team can get out of the door, and how fast those POCs can be turned into MVPs?

In this talk we’ll try to answer some questions:

– Why is it so hard to measure MLOps efficacy?
– What are some common metrics that teams are currently being measure on in the industry?
– What are some factors that might impact how metrics are created?
– When should you NOT use metrics?
– How do we empower MLOps teams when performance is going south?