Data-Driven Modeling of Face Evalution

People make intuitive judgments of other people’s traits (e.g., intelligence, kindness) from their appearances with ease, irrespective of the accuracy of the judgments. Many of these judgments affect important social outcomes, like voting, and hiring (Todorov et al. Ann Rev Psychol 2015).

Data-driven modeling of social impressions can identify the visual information responsible for these impressions (e.g., an impression of competence) with little prior assumptions as to what facial features matter. Some features that matter reflect stereotypes prevalent in society. For example, masculinity is one of the main ingredients of impressions of competence (Oh, Buck, & Todorov, Psych Sci 2019). The gender stereotype was not apparent because competence impressions are strongly correlated with facial attractiveness, which tends to be negatively correlated with masculinity. Taking advantage of the computational nature of the models of impressions, we built a model of competence that can manipulate perceived competence while controlling for attractiveness. This model removes the strong halo effect on competence impressions and shows that competent-looking faces are more masculine. This line of research not only identifies the visual information that affects social impressions but also uncovers hidden stereotype biases in these impressions.

Controlling for facial attractiveness (a natural confound of competence impressions) more competent faces look more masculine.
x = the manipulation level of the competence impression models; y = to what extent people categorized faces as male; arrays of faces = two models used to manipulate faces on their perceived competence).

The multidimensional nature of a data-driven computatoinal model allows a separate analysis of the effects the facial shape and reflectance on various social judgments (e.g., attractiveness), two main sources of facial social judgments.

Using mixed-effects models, we found that (1) the effect of face shape and face reflectance information on social judgments is largely linear and additive, (2) which kind of information (shape vs. reflectance) is weighted more heavily in judgments depends on the judged dimension (i.e., shape is more important for trustworthienss, dominance, and extroversion judgments; reflectance for competence; both for attractiveness), and (3) the relative amount of contribution of shape and reflectance is stable irrespective of the amount of visual information available (Oh, Dotsch, & Todorov, Vis Res, 2019).

Oh D, Buck EA, Todorov A (2019) Revealing hidden gender biases in competence impressions from faces. Psychological Science.

Oh D, Dotsch R, Todorov A (2019) Contribution of shape and reflectance information to social judgments from faces. Vision Research

Todorov A, Olivola CY, Dotsch R, Mende-Siedlecki P (2015) Social attributions from faces: Determinants, consequences, accuracy, and functional significance. Annual Review of Psychology, 66, 519–545.