Riplaki Be Fun and Happy

yahooresearch:

By Farshad Kooti, Kristina Lerman, Mihajlo Grbovic, Vladan Radosavljevic, Nemanja Djuric, and Luca Maria Aiello

Consumer spending is an essential component of economic activity. In 2015, it accounted for 68% of the US Gross Domestic Product (GDP), a measure often used to quantify economic output and the general prosperity of a country. Consumers are increasingly researching their products online with 72% of people under 35 in recent years researching their options online before going to stores. Congruently, online shopping is becoming more popular. Consumers spent 349 billion dollars online in 2015, which was a 14% increase from 2014. With this increase in popularity, we wanted to find out the answers to some important questions, including who spends the most online, when people shop most online, and how people spend their money online.

In beginning our investigation, we found that previous online shopping behavior studies have been limited to analyzing data from surveys or only one particular shopping website. We decided to take a different approach. Most online purchases result in a confirmation email sent to the shopper by the merchant. These emails provide a rich source of evidence to study online consumer behavior. In our research paper entitled “Portrait of an Online Shopper: Understanding and Predicting Consumer Behavior,” published in the proceedings of the the 9th ACM International Conference on Web Search and Data Mining (WSDM 2016), we analyzed anonymized shopping receipts from 20.1M Yahoo Mail users, amounting to 121M purchases worth 5.1B dollars, along with user demographics such as age, gender, and zip code. Our analysis yielded some interesting findings to our questions.

We started the study by looking at the effects of age and gender in online purchases. We found that women are more likely to make online purchases (Figure 1a), while men are typically responsible for purchasing more expensive items (Figure 2a), such as TVs and refrigerators.

To find the role of income in online shopping, we use a database of zip codes and median income in that area to estimate the income of the users. We determined that people from more affluent areas make more purchases (Figure 2a) and purchase more expensive items (Figure 2b), and as a result spend much more money online (Figure 2c).

Next, we studied people’s cyclic behavior in online shopping and we found strong weekly and daily patterns. People make significantly more purchases on Mondays and Tuesdays (Figure 3a) and also people are more likely to make purchases in the early afternoons and mornings (Figure 3b).

A dataset of the magnitude we worked with gave us a chance to test several interesting hypotheses with regard to users’ budget-aware and socially-driven behavior. One of the first hypotheses we tested was that an individual’s purchasing decisions are not independent of, but constrained by their finances. To test this hypothesis, we examined the relationship between the purchase price of an item and the time period since a person’s last purchase. If our hypothesis was correct, then there should have been a refractory period after a given purchase, with users waiting longer to make larger purchases. To test that our analysis did not have any bias in the way the users were grouped, we performed a shuffle test by randomly swapping the prices of products purchased by users. In the end, we clearly observed a positive relationship between (normalized) purchase price and the time since a last purchase (Figure 4), but not in the shuffled data, which produced a horizontal line.

We then hypothesized that an individual’s shopping behavior is correlated with that of his or her friends due to homophily, influence, or an external event. To investigate whether social correlations existed, we compared the purchasing behavior of users that had emailed each other, with users that were not linked to someone else with an email. We represented each user’s purchase history as a vector and calculated the similarity of users using cosine similarity. In Table 1 we show the average cosine similarity of purchase vectors for different groups of connected people compared to random pairs of users. It confirms that socially connected users are significantly more similar with regard to online shopping behavior.

Finally, we used the data to test how accurately the time or price of a next purchase could be predicted. Accurate predictions could help e-commerce sites improve one’s shopping experience via personalization and timed recommendations. We used a variety of features including demographics, purchase price, time history, and items purchased by contacts, and compared our results against three baselines: random prediction, previous purchase delay/price, and the most popular price or purchase delay. We experimented with different classification algorithms and Bayesian Network Classification yielded the highest accuracy. To estimate the conditional probability distributions we used direct estimates of the conditional probability with α = 0.5. The classifier was trained on the first six months of purchase data and evaluated on the last two. We were able to achieve 31% accuracy for predicting the timing and the price of the next item. The details of the prediction results can be found in Table 2.

In studying our large-scale online shopping dataset extracted from email confirmations sent to Yahoo Mail users, we found that women are more likely shop online, though men spend more when they do. Additionally, we confirmed that income plays an important role in people’s spending behavior online. Moreover, we constructed a classifier to predict the time and price of the next item a person is going to purchase, and we achieved higher accuracy in doing so as compared to competitive baselines. Going forward, we plan on using our findings and classifier to make better shopping recommendations for users.