Which grocers does the data come from?

Coverage

Prices are gathered daily and cover the North York area of Toronto, except for Save on Foods (that one is a location in Calgary). Prices are gathered as a "pick up only" customer would see them.

Prices are available starting on: February 28, 2024

Update frequency

Data is scraped daily, but may not be available on this website daily.

Quirks

• "Sale" indicators are scraped from what's visually displayed by the merchant on the website - not from the underlying API. When you are seeing an "old price" or "sale" announced, this is not calculated by me - this is what's claimed by the merchant. It may be intentionally misleading.
• There may be times when the scrape failed to set the geographic location of a store to the proper value. You might see this in a dramatic drop/increase of stock availability of certain things.

Known data issues

• Sometimes the same products show up on "different category pages" over and over. So you'll get multiples of the same price for the same product on the same day.
• Advertised/promoted products may show up in different category URLs
• A scrape that runs across the 12am day bounday may appear in 2 dates, but it is actually the same scrape event... so you may see 2 scrapes on the same date
• There are situations where the system tries to scrape all merchants, but one fails and is retried later. So you'll see "morning data" from certain merchants but "evening data" on the same day from the failed one.
• Sometimes a price will appear as a multi-buy: "2/$7.00" - if you are casting prices to numbers, then these prices might be silently dropped. Missing those data points understates how deep a Sale discount is / if a discount is present.
• Walmart:
  • On some days shows prices as $2.98, sometimes as $298. This is a problem with the underlying data and is incorrect. Watch out for 100x price increases.
• To resolve data mysteries, there is a detailed event log available. Reach out to us if you'd like to review key events around a certain date.
• The data that's available for download had been post-processed. Raw data is available upon vetting.

Citations when using this data

You can use this data for whatever purposes you want.

• When using this data for nonprofit/academic use, please mention Project Hammer
• If using this data for profit: do not mention this project

Others who are using this data

• At UofT: https://tellingstorieswithdata.com

How this data compares to other data sources

• Statcan collects "Point-of-Sale" / "Scanner" data from retailers as part of their inflation calculation. I believe that academics can access it in a Research Data Centre (RDC) under record ID 2301 "Consumer Price Index" , under component "Monthly average retail prices for selected products" table (18-10-0245-01)
  • Challenges Associated with Using Scanner Data for the Consumer Price Index (issues that may be relevant to Project Hammer data)
  • Potential Uses of Scanner Data - A Case Study Using Coffee Data
• Statcan has a great CPI inflation tool. Unfortunately, it groups products into generic categories (eg. baked goods)
• Here is a Statcan dataset with dollar prices for generic products  / online table (eg. 675g white bread)
• Flyers are another data source, but they only expose deals
• Project Hammer data shows daily prices per-retailer and per-brand / SKU.

Assumptions / Limitations

• For the same grocer, different locations do have different prices for the same product. Similarly, different locations have certain products out of stock (this might cause a particular product to disappear from the scrape).
• The data will not show fast price changes within the same day - because there is just 1 scrape per day.
• In-store prices shown on e-ink "Electronic Shelf Labels" could change multiple within a single day, and probably drift out of alignment with website. This is "Algorithmic Pricing".