Which grocers does the data come from?

Coverage

Prices are gathered daily and cover the North York area of Toronto, except for Save on Foods (that one is a location in Calgary). Prices are gathered as a "pick up only" customer would see them.

First date prices are available for: February 28, 2024

Update frequency

Data is scraped daily, but may not be available on this website daily.

Quirks

• The product_id unique product identifiers are not stable across the dataset if you download copies of it on different days - they're only unique "today". Essentially, identifier 5 might point to product A today but be reassigned to product B tomorrow.
• "Sale" indicators are scraped from what's visually displayed by the merchant on the website - not from the underlying API. When you are seeing an "old price" or "sale" announced, this is not calculated by me - this is what's claimed by the merchant. It may be intentionally misleading.
• There may be times when the scrape failed to set the geographic location of a store to the proper value. You might see this in a dramatic drop/increase of stock availability of certain things.

Data quality / noisiness

• Sometimes same products show up on "different pages" over and over
• Advertised/promoted products may show up in different category URLs
• A scrape that runs across the 12am day bounday may appear in 2 dates, but it is actually the same scrape event... so you may see 2 scrapes on the same date
• There are situations where the system tries to scrape all merchants, but one fails and is retried later. So you'll see "morning data" from certain merchants but "evening data" on the same day from the failed one.
• To resolve data mysteries, there is a detailed event log available. Reach out to us if you'd like to review key events around a certain date.
• The data that's available for download had been post-processed. Raw data is available upon vetting.

Rules for using this data

• When using this data for nonprofit/academic use, please mention Project Hammer
• If using this data for profit: do not mention this project

Others who are using this data

• At UofT: https://tellingstorieswithdata.com

How this data compares to other data sources

• Statcan collects "Point-of-Sale" / "Scanner" data from retailers as part of their inflation calculation. I believe that academics can access it in a Research Data Centre (RDC) under record ID 2301 "Consumer Price Index" , under component "Monthly average retail prices for selected products" table (18-10-0245-01)
  • Challenges Associated with Using Scanner Data for the Consumer Price Index (issues that may be relevant to Project Hammer data)
  • Potential Uses of Scanner Data - A Case Study Using Coffee Data
• Statcan has a great CPI inflation tool. Unfortunately, it groups products into generic categories (eg. baked goods)
• Here is a Statcan dataset with dollar prices for generic products  / online table (eg. 675g white bread)
• Flyers are another data source, but they only expose deals
• Project Hammer data shows daily prices per-retailer and per-brand / SKU.

Assumptions / Limitations

• For the same grocer, different locations do have different prices for the same product. Similarly, different locations have certain products out of stock (this might cause a particular product to disappear from the scrape).
• The data will not show fast price changes within the same day - because there is just 1 scrape per day.
• Store e-ink prices could change within a single day, probably drift out of alignment with website. This is "Algorithmic Pricing".