NotesByLex

Reliance on Metrics is a Fundamental Challenge for AI

Notes from paper Reliance on Metrics is a Fundamental Challenge for AI by Rachel Thomas and David Uminsky

Lex Toumbourou · · revised

ML Ethics
  • Data-driven companies heavily rely on metrics for decision making
  • Metrics also play a central role in ML, therefore, the risks are maximised there.
  • Examples of instances where metrics have caused problems:
    • Recommender algorithms radicalising viewers
    • Good teachers fired due to algorithmic mistakes
    • Essay grading software rewarding "sophisticated garbage"

  • Goodhart's Law, generalised by Marilyn Strathern in 1997, describes the fundamental problem with metrics:

    "When a measure becomes a target, it ceases to be a good measure"

  • Paper finds supporting principles:

    • Proxy Metrics what you really care about
    • Metrics can and will be gamed: aka Metric Hacking
    • Metrics tend to overemphasis short-term concerns
    • Many online metrics are gathered in highly addictive environment
  • Paper proposes framework for healthier use of metrics:
    • Use a wide range of metrics for evaluation
    • Combine metrics with qualitative accounts
    • Involve range of stakeholders, especially those most impacted