Skip to content

A spring boot service that has various APIs that can help crawl Amazon(.com) and extract the SKU details

Notifications You must be signed in to change notification settings

Prashant-Chaurasia/priceFinder

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

priceCrawler

This is a spring boot service. The service has various APIs that can help crawl Amazon(.com) and extract the SKU details. It uses PostgreSQL as database for storing data. The code uses Strategy Pattern inorder to support adding algorithms for new retailer like Walmart, Instacart etc.

How to run?

  • Database used is PostgreSql. Install it in your machine.
  • Create a database commerce with user postgres and password postgres
  • Alternatively, you can change the configuration in application.properties
  • In the terminal run mvn spring-boot:run.
  • The server will be up and running.

APIs

GetHtml

/products/html - GET

Query Params -

  • sku = string
  • url = string
{
   "timeTaken": "81 ms",
   "data": {html}
}

Example - http://localhost:7007/products/html?sku=B07Q7L6SRG


GetProductDetails

/products - GET

Query Params -

  • sku = string
  • url = string
	Response
{
   "timeTaken": "81 ms",
   "data": {
       "id": "cd8ef413-6203-4dd7-b537-8bbcd0eeed7f",
       "sku": "B07Q7L6SRG",
       "title": "Sunfeast Mom's Magic Cashew and Almond, 600g",
       "description": "Mom's…..",
       "offerPrice": 72.0,
       "createdAt": "2021-05-09T22:22:33.803+00:00",
       "ratings": {
           "3star": "9%",
           "2star": "2%",
           "4star": "25%",
           "overallCount": "12,945",
           "5star": "60%",
           "1star": "3%"
       }
   }
}

Example - http://localhost:7007/products?sku=B07Q7L6SRG


GetProductDetailsHistory

/products - GET

Query Params -

  • sku = string
  • url = string
  • date=dd/MM/yyyy Note - you need to pass date
	Response -
	{
   "id": "a5d62e3b-9c77-41bb-9cef-149ed816043f",
   "sku": "B07XMLWH7J",
   "title": "Kellogg's Corn Flakes Original, High in Iron, High in B Group Vitamins, Breakfast Cereals, 1.2 kg Pack",
   "description": "Don't compromise on morning nourishment when you have a balanced, great-tasting breakfast that is high in iron, vitamin b1, b2, b3, b6 and vitamin c, it is quick to eat and gives you energy even after 3 hours.",
   "offerPrice": 298.0,
   "createdAt": "2021-05-09T15:56:26.758+00:00",
   "ratings": {
           "3star": "9%",
           "2star": "2%",
           "4star": "25%",
           "overallCount": "12,945",
           "5star": "60%",
           "1star": "3%"
       }
}

Example - http://localhost:7007/products?sku=B07XMLWH7J&date=10/05/2021


List All crawled Products and details till now.

/products - GET

Response
{
   "timeTaken": "275 ms",
   "data": [
       {
           "id": "cd8ef413-6203-4dd7-b537-8bbcd0eeed7f",
           "sku": "B07Q7L6SRG",
           "title": "Sunfeast Mom's Magic Cashew and Almond, 600g",
           "description": "Mom's....",
           "offerPrice": 72.0,
           "createdAt": "2021-05-09T22:22:33.803+00:00",
           "ratings": {
               "3star": "9%",
               "2star": "2%",
               "4star": "25%",
               "overallCount": "12,945",
               "5star": "60%",
               "1star": "3%"
           }
       },
       {
           "id": "0b7d8f2f-d0b9-4573-a84d-1322b8a281de",
           "sku": "B07Q7L6SRG",
           "title": "Sunfeast Mom's Magic Cashew and Almond, 600g",
           "description": "Mom's....",
           "offerPrice": 72.0,
           "createdAt": "2021-05-09T22:57:30.968+00:00",
           "ratings": {
               "3star": "9%",
               "2star": "2%",
               "4star": "25%",
               "overallCount": "12,945",
               "5star": "60%",
               "1star": "3%"
           }
       }
   ]
}

Example - http://localhost:7007/products


PriceTrend

/products/trends

Response 
{
   "timeTaken": "85 ms",
   "data": [
       {
           "sku": "B07Q7L6SRG",
           "timestampPrices": [
               {
                   "timestamp": "2021-05-09T22:57:30.968+00:00",
                   "price": 72.0
               },
               {
                   "timestamp": "2021-05-09T22:22:33.803+00:00",
                   "price": 72.0
               }
           ]
       }
   ]
}

Example - http://localhost:7007/products/trends

About

A spring boot service that has various APIs that can help crawl Amazon(.com) and extract the SKU details

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages