Deriving Loss Functions for Regression and Classification from Humans
Hansol Ryu, University of Calgary, Canada; Manoj Srinivasan, The Ohio State University, United States
Session:
Posters 2 Poster
Location:
Pacific Ballroom H-O
Presentation Time:
Fri, 26 Aug, 19:30 - 21:30 Pacific Time (UTC -8)
Abstract:
Understanding how humans perceive patterns in visually presented data is useful for understanding data-based decision making or control. Here, we examined how humans perform the simplest machine learning or statistical estimation tasks, namely linear regression (fitting) and classification, and then used simple inverse optimization to derive the loss function humans optimize for when they perform these tasks. We designed tests such that different loss functions result in different optimal solutions for the given data, so that we can compare that with what humans do. For the regression task, minimizing error squared loss best described human fitting for sparse data, whereas for less sparse data with more points, loss functions with lower exponents, which would reject outliers more effectively, were better descriptors. For the classification task, minimizing error power 1.4 on misclassified data or exponential loss on all data were good descriptors of human choices. People changed their strategies as data density increased, such that loss functions with lower exponents described human data better. Future work may examine other loss function families and other tasks.