Business & Economics

The Paradox of Choice Meets the Information Age

Maximum data privacy may not be the best option for consumers

The paradox of choice. Image Credit: MassanPH/Getty Images

Barry Schwartz’s seminal work, “The Paradox of Choice,” has received substantial attention since its publication nearly 20 years ago. Schwartz argued that, faced with an ever-increasing plethora of products to choose from, consumers can often feel overwhelmed and seek to limit the amount of choices they have to make.

In today’s online digital economy, a possible response to this problem is for digital platforms to use consumer data to present consumers with a manageable array of choices and thereby simplify their product selection. Appropriate curation of product choice options may substantially benefit consumer welfare, if government regulators stay out of the way.

Three Consumer Data Regimes

In a new paper in the American Economic Review, Mark Armstrong and Jidong Zhou, of Oxford and Yale Universities respectively, develop a theoretical framework to understand how companies compete using consumer data. They find that there is, in fact, an effect on consumer, producer and total welfare when different privacy regimes are enacted to change the amount of information a company can use to personalize recommendations.

Personalized Product

In theory, the optimal solution to the paradox of choice would be to maximize total welfare under a personalized-product regime. In this scenario, a platform is able to gather information on consumers to such a degree that buyers and sellers are perfectly matched, leading to consumers buying their first-best option.

A personalized-product regime is akin to unlimited data gathering, where platforms such as Amazon are able to use all the available information to perfectly suggest products based on revealed data. For instance, when a consumer is looking to buy a new computer, Amazon can gather data on what websites were visited and what keywords were searched (such as “4k display,” “bezel-less screen” and “portable size”) to recommend the best product based on the consumer’s search preferences. This means the consumer doesn’t need to spend any time on research, since Amazon has already analyzed computer specifications before offering the product for sale.

This type of system can result in slightly higher prices because it decreases competition among producers: Platforms will accentuate the differences between products, showing a single product as the “best” choice, and consumers will not want to select a different, “worse” product. As a result, the personalized-product regime leads to higher profits for producers, but search and mismatch costs are minimized by the platform, leading to a high level of welfare for consumers also. In other words, this regime maximizes total welfare.

Data Privacy

The highest level of consumer welfare (rather than total welfare) comes with a data-privacy regime, under which there is minimal product differentiation, leading to a high number of substitutes and low prices. Since any differences in offered products are downplayed, producers will tend to reduce prices and increase quality, making consumers better off.

Under this regime, when consumers want to buy a new computer, a platform such as Amazon cannot gather or use personal data to recommend products. It can still highlight specific products for consumers, but they are not personalized because the platform can use only general information such as sales trends, new products or basic product specifications. A graphic designer who needs a high-definition display and a powerful processor may be suggested a suite of Chromebooks, since they are currently top sellers, but these products are more suitable to on-the-go students.

Thus, the data-privacy scenario comes with some level of mismatch. Since consumers are not matched with tailored recommendations, search costs are high and introduce some error. Some consumers may not purchase the product that best meets their needs, but they do not feel the negative effects of the mismatch because they paid a lower price. However, consumers who want the best product match for their needs will likely take a long time individually researching products. This search time increases the likelihood of a correct match, but it leads to a real-life cost—cost of the product plus the time it took to find the right product. Therefore, consumer welfare is maximized for some consumers but not all—and producer welfare is significantly lower than in the personalized-product regime.

Curated List

Finally, a nearly optimal compromise approach is known as a curated-list regime. Here, the platform suggests a “top two-best” scheme, where consumers are shown their top two best options without explicit ranking. This nearly maximizes total welfare, since consumers are shown the best options for them, and even if the best match isn’t chosen, the second-best match is close in terms of welfare.

In a curated-list regime, platforms gather information in the same way they would under the personalized-product scenario, but there would be clearly defined rules surrounding the display of that information. Amazon is still able to gather as much information as it desires to make a personalized recommendation, but it then displays an array of products that represent the first two (or three or four) best-choice options. These options are displayed without ranking the products, allowing the consumer to chose from a curated list instead of a single product.

In the computer-buying example, Amazon could show the graphic designer laptops with similar specifications from brands such as Apple, Dell, HP and Lenovo; allow the designer to compare and contrast the offerings; and ultimately help the designer find the best choice without leaving Amazon to search for the “right” computer.

The curated-list regime has two distinct effects on the market:

  1. It will tend to decrease prices through increased competition. Since producers can know only which consumers to target, but not which ones will actually choose the product, they have to effectively compete with closely related products.
  2. It will likely spur innovation among existing producers (who seek to differentiate their products from their competitors’) and increase competition from new producers.

In this regime, a new product has a relatively good chance of being included on the consumer’s curated list, because all it has to do is fit a consumer profile well enough to beat out another product. In contrast, a new product in a personalized-product regime has a low chance of being picked as “best” since it must be a better match than all other existing products. Similarly, under a data-privacy regime, there are so many products to choose from that the chance of choosing any one new product is small.

Consumer Data Regimes in Practice

Armstrong and Zhou’s paper provides a theoretical framework for these three consumer data regimes, but the true test is how they work in practice. The U.S. and Europe are both attempting to implement rules restricting data usage and bans on “self-preferencing”—that is, when a company leverages its advantages to offer products for a lower price through an in-house brand. Examples of in-house brands include Amazon’s AmazonBasics, Target’s up & up and Costco’s Kirkland Signature. But are these rules the best way to increase economic welfare for all?

Europe’s General Data Protection Regulation

The EU’s General Data Protection Regulation (GDPR) is among the most ambitious and all-encompassing data privacy regimes to date, and it has significant negative ramifications for economic welfare. The GDPR seriously diminishes competition because of its restrictions on data collection and sharing; these restrictions reduce the competitive pressure platforms face.

For platforms to gain a complete profile of the consumer for personalization, they cannot rely only on data collected from their platform. To ensure a level of personalization that effectively reduces search costs for consumers, these platforms must be able to acquire data from a range of sources and combine that data to create a complete consumer profile. Restrictions on such collection and combination lead to diminished competition online.

The GDPR grants consumers the right to choose both how their data is collected and how it is distributed. This creates a high regulatory burden for both the platform and the data seller, and reduces the incentive to transfer data between firms. Further, the data seller can be held liable for actions taken by the platform, which significantly increases the risk a data seller faces. Therefore, the data seller will request a higher price to transfer the data, which will reduce the demand for outside data.

This regulatory burden decreases the quality of personalization and tilts the scales toward larger platforms with more robust data collection practices and the ability to absorb high regulatory enforcement costs. Additionally, those platforms that are already entrenched and have large user bases are better able to manage the regulatory burden of the GDPR. Many U.S. companies with more than 500 workers planned to spend between $1 million and $10 million in up-front costs to prepare for compliance, a number that will likely pale in comparison to the long-term compliance costs. New and emerging competitors will find it hard to compete in a market that is stacked against them.

Additionally, consumers benefit from platforms that are able to accurately recommend products. Large platforms with vast amounts of accumulated, first-party data will be consumers’ destinations of choice, since they will be able to create the most accurate profile and offer the best recommendations. Smaller firms will be less able to compete, simply because they do not have access to the same scale of data as the large platforms when data cannot be easily transferred between parties.


Claims of digital platforms’ anti-competitive behavior are abundant, and they often focus on the concept of self-preferencing. Tech companies are able to leverage data and personalization to drive web traffic toward their own products, which many commentators and politicians decry as an “unfair advantage.” It is far from clear, however, that this practice decreases consumer welfare. Numerous commentaries have circulated since anti-preferencing bills were released in the U.S. House and Senate, rejecting the notion that self-preferencing is anti-competitive or anti-consumer.

In fact, there are good reasons to believe that self-preferencing promotes both competition and consumer welfare. If a company manufactures (or contracts out manufacturing for) its own in-house products, it can generally offer those products at a lower price for the same relative quality. This decrease in price raises consumer welfare. The in-house brand’s entrance into the market also represents a potent competitive threat to existing producers, causing them to lower their own prices, raise the quality of their goods, or both to maintain their consumer base. This in turn creates even more consumer welfare since all consumers, not just the ones purchasing the in-house goods, benefit from the entrance of an in-house brand.

Under a personalized-product regime, platforms such as Amazon may appear to have an incentive to self-preference to the detriment of consumers. If consumers trust the platform to show them their personalized “best” product, the platform may use this consumer trust to its advantage and suggest its own, potentially inferior product. But an unexpectedly low level of house-brand product quality will diminish the platform’s reputation, eroding consumer trust and forcing the platform to enhance the quality of its inferior in-house offering, or perhaps even refrain from offering an in-house brand at all.

Still, a curated-list regime is unequivocally better for consumers, even with the existence of in-house brands. Since consumers are shown several more options than under a personalized-product regime, they are able to actively compare the offerings from different companies to determine the correct product for their individual use. Self-preferencing does not harm consumers in this case because they are able to make value judgments between the in-house brand and the alternatives, weighing their relative price and quality. The only instance where the in-house brand has a strong chance of success is when the price is lower than, and the quality is greater than or equal to, competing products. This will tend to increase consumer welfare, both because the in-house product is a good option and because competing producers will improve their quality and pricing so they can remain competitive.

Don’t Impose More Restrictions on Data

Recently published economic research has developed theoretical scenarios that demonstrate how digital platform curation of consumer data may facilitate welfare-enhancing consumer purchase decisions. This research should give pause to proponents of major new restrictions of platform data usage.

Furthermore, actual and proposed regulatory restrictions demonstrate the serious harm that can result from government meddling in digital platform data usage. The first four years of GDPR show that it has caused significant negative unintended consequences: Competition has decreased, regulatory barriers to entry have increased and consumers are marginally worse off. Since companies are less willing and able to leverage data in their operations and service offerings—due in large part to the risk of hefty fines—they are less able to curate and personalize services to consumers.

Additionally, anti-preferencing bills in the U.S. Congress threaten to suppress the proper functioning of platform markets and reduce consumer welfare by restricting the use of data in product market decisions. More research is necessary to determine the overall effect of such preferencing on platforms, but early results indicate that consumers are better off when an in-house brand enters the market and increases competition.

By contrast, current U.S. government policy that generally allows platforms to use consumer data freely is good for consumer welfare. Indeed, the consumer welfare benefits generated by digital platforms, which depend critically on large volumes of data, are enormous. Thus, governments should avoid imposing additional regulations on digital platform consumer data use. Such meddling would ultimately harm the very consumers the government is trying to help.

Submit a Letter to the Editor
Submit your letter
Subscribe to our newsletter