Table of Contents
Deals Scraper is a Canadian tool to find good deals on websites like Facebook Marketplace, Kijiji, Ebay, Amazon and Lespacs
- Zooming fast
- Specify a price range (Min & Max) per scraper
- Blacklist keywords from the scraping
- Strict mode so only ads containing your keywords are picked
- Schedule the project to run on a recurring basis
- Easily add your own website if you know what you're doing
Clone the repo
git clone https://github.com/JustSxm/Deals-Scraper.git
Install dependencies
pip install -r requirement.txt
Open the config file (config.ini) and find the section named DEFAULT
[DEFAULT]
Keywords = airpods pro
Exclusions = case
StrictMode = False
Interval = 1
- Keywords: This config sets the keywords that the software will search for.
- Exclusions: This config sets the words or phrases that the software will ignore.
- StrictMode: This config determines whether the software will strictly match the keywords and exclusions or be more flexible in its search.
- Interval: This config sets the time interval for how often the software will search for the keywords in minutes.
Open the config file (config.ini) and find the section named FACEBOOK
[FACEBOOK]
Enabled = True
CityId =
MinPrice = 0
MaxPrice = 1000
SortBy = distance_ascend
; best_match, price_ascend, price_descend, distance_ascend, creation_time_descend
- Enabled: This config determines whether the Facebook module is enabled (True) or disabled (False).
- CityId: This config sets the ID for the desired city or location for the Facebook search.
- MinPrice: This config sets the minimum price range for the Facebook search.
- MaxPrice: This config sets the maximum price range for the Facebook search.
- SortBy: This config sets the sorting method for the Facebook search results. Available options include "best_match" (sorted by Facebook's relevance algorithm), "price_ascend" (sorted by price in ascending order), "price_descend" (sorted by price in descending order), "distance_ascend" (sorted by distance in ascending order), and "creation_time_descend" (sorted by time in descending order).
To find your CityId:
- go to facebook and search for your city (Ex: https://www.facebook.com/search/top?q=toronto)
- Click on "Places"
- Click on your city
- Copy your city's id
After make sure to login to Facebook, this scraper uses your browser to scrape, it doesn't connect automatically.
Open the config file (config.ini) and find the section named Kijiji
[KIJIJI]
Enabled = True
CityUrl =
Identifier =
MinPrice = 20
MaxPrice = 100
Type = ownr
; ownr, delr, all
- Enabled: This config determines whether the Kijiji module is enabled (True) or disabled (False).
- CityUrl: This config sets the URL for the Kijiji website for the desired city or location.
- Identifier: The identifier with the city url.
- MinPrice: This config sets the minimum price range for the Kijiji search.
- MaxPrice: This config sets the maximum price range for the Kijiji search.
- Type: This config sets the type of Kijiji ads to search for, which can be "ownr" (owner-sold ads), "delr" (dealer-sold ads), or "all" (both types).
To find your CityId and Identifier:
Open the config file (config.ini) and find the section named Ebay
[EBAY]
Enabled = True
MinPrice = 20
MaxPrice = 100
- Enabled: This config determines whether the eBay module is enabled (True) or disabled (False).
- MinPrice: This config sets the minimum price range for the eBay search.
- MaxPrice: This config sets the maximum price range for the eBay search.
The LesPACs configuration provided in this repository is not currently implemented due to the website either undergoing a rewrite or implementing security measures to prevent web scraping. Unfortunately, without more information from LesPACs themselves, it is not possible to provide an ETA for when the configuration will be functional again.
If you are looking to use the Amazon configuration in this repository, please note that it is not included in the current version and is not planned to be included in the future. This is due to the implementation of anti-scraping measures on the Amazon website, such as CAPTCHAs, which make it difficult or impossible to retrieve data using a web scraper. As such, the Amazon configuration provided in previous versions of the repository may no longer be functional.
Run the python script
python main.py
Facebook uses a city to look around as it is not international, you can find the id by looking for your city on facebook and copy the id of their page. (usually facebook.com/..../place/id)
Facebook is a good website for scraping.
Kijiji is a good website for scraping
Ebay is an okay website for scraping
Since Amazon is a vast website, it is way harder to find new ads and to precise what we want, therefore you will most likely get garbage from it than what you're actually looking. It could be fine if you're looking for the cheapest price for a "popular" item
Amazon is a bad website for scraping
Lespacs is just like kijiji except it is more Quebec centered than Canada, therefore it can be a bad site for scraping if you are not from Quebec, otherwise it is a pretty good one
Outside of Quebec: Lespacs is a bad website for scraping
Inside of Quebec: Lespacs is a good website for scraping
This project was made with the help of scrapy
README - ChatGPT