Warehouse Club Study Flawed in Conclusion

As noted in a post in this blog last week, a study by two business professors, Michael Norton of Harvard Business School and Leonard Lee of Columbia Business School drew a lot of media attention for its assertion that warehouse clubs lead people to spend more than they intend, however, their study's conclusion, which leads to the assertion about overspending, seems flawed.

The conclusion â€" and their reason people spend more than intended -- is that warehouse club fees make people behave irrationally. Norton and Lee contend that consumers attribute low prices to the perceived savings offered by retailers that charge dues, then try to capitalize on these seeming savings by buying more than they otherwise would. The professors assembled a test group, told them that they were going to participate in a one-hour session of experiments. The experiments actually were a decoy for the real study. After the bogus experiments, the participants were informed that they could shop from a variety of discount priced-products including a 25-cent candy bar, a $1 university logo pen and a $2 Beanie Baby. Participants, on a random basis, were assigned to pay a fee for purchasing or not. Those required to pay the fee were told they had to give up 50 cents to purchase from the store while those not required could buy without condition.

The professors put a twist on the experiment in that the participants got to see the products and the prices charged before deciding whether or not to pay the fee. The idea was to avoid a sunk cost motivation, where consumers justify payment of a fee by increasing consumption so that savings offset the charge.

The study participants made their choices and, the researchers stated:

The results revealed that there was no significant difference in purchase likelihood between our two stores: 58 percent of participants in the fee condition chose to pay the fee and buy at least one item, while 55 percent of participants in the no fee condition did so. However, as predicted, participants in the fee condition spent significantly more than those in the no fee condition.

The researchers did other tests, asking people about the prices of bulk products at Costco and Wal-Mart -- on average, participants in that study thought Costco's prices 4.5 percent less expensive than Wal-Mart's --- then surveying people at the student center of a Northeastern university about their perception of prices at two stores, one that charged a $100 annual membership dues and one that didn't require a fee. In that case, 88 percent thought the $100 fee store offered better prices.

Now, besides the notion that university students may not be the best proxy for typical warehouse club shoppers, the report contains certain elements that undermine its own conclusion. As noted in an earlier blog post, the report contained a field study demonstrating that Costco prices did beat Wal-Mart's in a unit-price level comparison. Not only that but, in the report's conclusion, the authors finish by saying, "consumers may on average come out ahead despite their overgeneralized perception of the link between fees and savings."

Why overgeneralized? The report suggests that warehouse club pricing makes comparisons with what's charged at more traditional retailers difficult, but that's an arguable point. Consumers might not continually do the math required to break down prices to basic units and make direct comparisons. Still, they probably do on frequently purchased items and likely have a rough idea of the relative value of other purchases.

In analyzing the study's key experiment, the one with the fee/no fee test store, the report doesn't consider an important point: Participants may have made or had access to unit price comparisons between warehouse clubs and other retailers, and, based on what they had learned previously, concluded quite rationally that clubs can offer superior value. If that's the case, their assumption that the store charging dues would deliver a bargain isn't really irrational.