Skip to content

This is a web scraping project using open source tools: Beautiful soup and Selenium. This solution should work on every website that doesn't block bots or web crawlers. It works by selecting html tags and getting the data from there. This is the most verbose solution.

Notifications You must be signed in to change notification settings

IR-System-TTU/poc-beatifulsoup-selenium

Repository files navigation

Lazy Shop

Installing dependencies

  pip install -r requirements.txt

This command will install all the dependencies of the project that are specified in the configuration file "requirements.txt"

To check if the dependencies are correctly installed in your system use the following command:

pip list

If everything has been installed correctly, you should see all the dependencies in the "requirements.txt" listed on you terminal after running pip list command.

Installing ChromeDriver

ChromeDriver is a standalone server (executable file) that is used by Selenium WebDriver to control Chrome. This is key to perform correct web scraping of the different online stores. Follow this tutorial for setup: https://www.browserstack.com/guide/run-selenium-tests-using-selenium-chromedriver

About

This is a web scraping project using open source tools: Beautiful soup and Selenium. This solution should work on every website that doesn't block bots or web crawlers. It works by selecting html tags and getting the data from there. This is the most verbose solution.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published