Measuring algorithms in online marketplaces
Permanent URL:
http://hdl.handle.net/2047/D20251583
Mislove, Alan (Committee member)
Zhao, Ben (Committee member)
Riedl, Christoph (Committee member)
Algorithms are powerful tools that have the potential to improve the efficiency of markets, but come at a cost of possible harms. On one hand, algorithms can be beneficial: RTB maps billions of Internet users to customized advertisements based on their interests in real time, and thus increases the effectiveness for both advertisers and publishers in terms of advertising inventory sold. Similarly, ride sharing has redefined the transportation market by matching millions of drivers and riders around the globe in real-time. On the other hand, evidence shows that problems may be caused by algorithms, such as racial discrimination on Googles advertisement services, or unpredictable prices shown to users in online marketplaces.
Unfortunately, we currently lack measurement tools or methodologies to audit the behavior of these algorithms. As a result, we are unable to measure the impact of these algorithms on people. For example, the general public is unable to access the details about how credit scores and prices are calculated in online marketplaces, since algorithms are typically proprietary trade secrets. Similarly, for Machine Learning algorithms, the data for training the predicative models may also be unavailable. The lack of transparency surrounding algorithms and the data that powers them has led to concerns about whether they are being manipulated by companies to increase their own profit, and whether they are fair and unbiased to users.
The goal of my work is to develop methodologies and build measurement tools to audit and understand the impact of algorithms in online marketplaces. I focus on three types of marketplaces: the ride sharing marketplace Uber, the e-commerce marketplace Amazon, and human labor market- places Indeed, Monster, and CareerBuilder. Algorithms play crucial roles on all these platforms, and potential fairness and manipulation issues caused by the algorithms may be present in these systems.
First, I examine Ubers surge pricing algorithm to answer questions such as whether ride prices are true reflections of supply and demand dynamics, and whether surge prices can be manipulated by the company or passengers. I gather four weeks of data from Uber by emulating 43 copies of the Uber smartphone app and distributing them throughout downtown San Francisco (SF) and midtown Manhattan. Using my dataset, I am able to characterize the dynamics of Uber in SF and Manhattan, as well as identify key implementation details of Ubers surge price algorithm. My observations about Ubers surge price algorithm raise important questions about the fairness and transparency of this system.
Next, I investigate two major and correlated algorithmic components on Amazon Market- place that determine the product prices paid by consumers: the Buy Box and dynamic pricing by sellers in the market. In this study, I first conduct an in-depth investigation on the features and weights that drive the Buy Box algorithm. Then I develop a methodology for detecting dynamic pricing by sellers, and use it to empirically analyze their prevalence and behavior on Amazon Marketplace. I gather four months of data covering all merchants selling any of 1,641 best-seller products. Using this dataset, I am able to uncover the algorithmic pricing strategies adopted by over 500 sellers. I then explore the characteristics of these sellers and characterize the impact of these strategies on the dynamics of the marketplace.
Finally, I study the ranking algorithms that power resume search engines on hiring websites, and investigate gender-based inequalities in their search results. I collect search results from Indeed, Monster, and CareerBuilder based on 35 job title queries in 20 American cities, resulting in data on over 855K job candidates. Using statistical tests and regression analysis, I find statistically significant evidence of two types of inequality on all three websites (ranking bias and unfairness), almost always to the detriment of female candidates. Motivated by these findings, I propose two alternative ranking methods that encode different definitions of fairness, and examine the inherent tradeoffs posed by trying to achieve gender-fairness in hiring markets.
Altogether, my work presents techniques to measure the impact of algorithms in online marketplaces. My methods can be extended to other platforms and services, in order to increase transparency and provide insights into how these systems affect people. Ultimately, I hope that my research helps people to find and mitigate issues present in opaque algorithmic systems.
measurement
transparency
web-based services
Copyright restrictions may apply.