- Introduction
- Authentication
- Amazon
- Wayfair
- Other Sites
- Targets
- Parameters
- Response Codes
- Postman Collection
- License
With our eCommerce Scraping API, you can gather most important data from eCommerce websites. Most notably and in depth, Amazon, but also Wayfair.
Once you have an active eCommerce subscription, you can try sending a request right from the dashboard eCommerce > Authentication method tab simply by entering your username, password, and clicking on Generate. You will also see an example of curl request generated right below your entered user:pass.
Note that this is only an example with preset values to get you on the right track for forming your own request, meaning you will not be able to change the request values in the dashboard itself – that will have to be done in your code.
Amazon can be targeted using direct URL, by using the Amazon product ID, or Amazon search bar.
You can also retrieve Amazon reviews, as well as questions and answers about a product.
API Link: https://scraper-api.smartproxy.com/v2/scrape
POST /scrape
Required parameters: url
Parameter | Type | Description |
---|---|---|
url |
url |
Amazon product page URL |
target |
string |
Scraping target |
First 2 top & bottom ads and product listing info
{
"results": [
{
"content": {
"ads": [
{
"pos": 1,
"asin": "B07TC8J6HK",
"type": "sponsored_highly_rated",
"price": 28.99,
"title": "Jeecoo USB Pro Gaming Headset for PC - 7.1 Surround Sound Headphones with Noise Cancelling Microphone- Memory Foam Ear Pads RGB Lights for Laptops",
"images": [
"https://m.media-amazon.com/images/I/416Q4VpWM+L._AC_SR160,160_.jpg"
],
"rating": 4.5,
"location": "carousel",
"price_upper": 28.99,
"reviews_count": 20122,
"is_prime_eligible": true
},
{
"pos": 2,
"asin": "B09MJB8QZN",
"type": "sponsored_highly_rated",
"price": 19.99,
"title": "Gaming Headset with Microphone, Gaming Headphones Stereo 7.1 Surround Noise Cancelling Xbox Headset PS4 Headset for PS5 Xbox one PC Switch Playstation Laptop",
"images": [
"https://m.media-amazon.com/images/I/41VM7Iz2MbL._AC_SR160,160_.jpg"
],
"rating": 4,
"location": "carousel",
"price_upper": 19.99,
"reviews_count": 97,
"is_prime_eligible": true
},
{
"pos": 3,
"asin": "B012DFI02O",
"type": "sponsored_highly_rated",
"price": 18.69,
"title": "VersionTECH. G2000 Gaming Headset for PS5 PS4 PC Xbox One, Surround Sound Over Ear Headphones with Mic, LED Light for Mac Laptop Switch Playstation Xbox Series X/S -Blue",
"images": [
"https://m.media-amazon.com/images/I/517CVbs+2xS._AC_SR160,160_.jpg"
],
"rating": 4.5,
"location": "carousel",
"price_upper": 18.69,
"reviews_count": 13110,
"is_prime_eligible": true
},
{
"pos": 1,
"asin": "B0B2JSYWL3",
"type": "sponsored_products_bottom",
"price": 49.99,
"title": "Gaming Headset with Microphone, LTXHorde Over Ear Gaming Headphone with Noise Cancelling, RGB Wired Gamer Headset, Hi-Fi Stereo Surround Sound, LED Lights, for PC, Laptop, PS4, PS5, Xbox, iPad, Switch",
"images": [
"https://m.media-amazon.com/images/I/41ZcjPDzzXL._AC_SR160,160_.jpg"
],
"rating": 0,
"location": "carousel",
"price_upper": 49.99,
"reviews_count": 0,
"is_prime_eligible": true
},
{
"pos": 2,
"asin": "B09LQP7V3S",
"type": "sponsored_products_bottom",
"price": 59.99,
"title": "SOMIC GS510 Gaming Headset, Cat Ear Headset PC Gaming Headphones with Retractable Mic Noise Cancelling, Stereo Sound, DIY Face Covers for PC, PS4, PS5,Xbox One(Only White LED Light)",
"images": [
"https://m.media-amazon.com/images/I/41D6Qku154L._AC_SR160,160_.jpg"
],
"rating": 4,
"location": "carousel",
"price_upper": 59.99,
"reviews_count": 551,
"is_prime_eligible": true
},
{
"pos": 1,
"asin": "B0B71N6XG5",
"type": "organic_also_viewed",
"price": 27.99,
"title": "",
"images": [
"https://m.media-amazon.com/images/I/415P1K4iYCL._SS150_.jpg"
],
"rating": 0,
"location": "carousel",
"price_upper": 27.99,
"reviews_count": 0,
"is_prime_eligible": false
}
],
"url": "https://www.amazon.com/dp/B09H74FXNW",
"asin": "B09H74FXNW",
"page": 1,
"price": 20.98,
"stock": "In Stock.",
"title": "Gaming Headset with Microphone, Gaming Headphones Stereo 7.1 Surround Sound PS4 Headset 50mm Drivers, 3.5mm Audio Jack Over Ear Headphones Wired for PC Switch Playstation Xbox PS5 Laptop",
"buybox": {
"used": 17.7,
"buy_new": 20.98
},
"coupon": "",
"images": [
"https://m.media-amazon.com/images/I/71+jTaVbHxL._AC_SL1450_.jpg",
"https://m.media-amazon.com/images/I/715fsq+rZaL._AC_SL1450_.jpg",
"https://m.media-amazon.com/images/I/713OLdYw+iL._AC_SL1280_.jpg",
"https://m.media-amazon.com/images/I/71ZEBbRz2eL._AC_SL1450_.jpg",
"https://m.media-amazon.com/images/I/713OLdYw+iL._AC_SL1280_.jpg",
"https://m.media-amazon.com/images/I/71ILJdNwbjL._AC_SL1450_.jpg",
"https://m.media-amazon.com/images/I/81fhbj7zQNL._AC_SL1450_.jpg",
"https://m.media-amazon.com/images/I/71662ua-CpL._AC_SL1450_.jpg"
],
"rating": 4.4,
"category": [
{
"ladder": [
{
"url": "/computer-video-games-hardware-accessories/b/ref=dp_bc_aui_C_1/131-2911047-2733206?ie=UTF8&node=468642",
"name": "Video Games"
},
{
"url": "/Mac-Games/b/ref=dp_bc_aui_C_2/131-2911047-2733206?ie=UTF8&node=229647",
"name": "Mac"
},
{
"url": "/Mac-Accessories/b/ref=dp_bc_aui_C_3/131-2911047-2733206?ie=UTF8&node=318812011",
"name": "Accessories"
},
{
"url": "/Mac-Game-Headsets/b/ref=dp_bc_aui_C_4/131-2911047-2733206?ie=UTF8&node=405593011",
"name": "Headsets"
}
]
}
],
"currency": "USD",
"delivery": [
{
"date": {
"by": "Wednesday, September 7",
"from": null
},
"type": "FREE delivery"
},
{
"date": {
"by": "Saturday, September 3",
"from": null
},
"type": "Or fastest delivery"
}
],
"_warnings": [
"Could not parse lightning deal."
],
"deal_type": "",
"page_type": "Product",
"price_sns": 0,
"variation": [
{
"asin": "B09H74FXNW",
"selected": true,
"dimensions": {
"Color": "A Camo Gray"
},
"tooltip_image": "https://m.media-amazon.com/images/I/414I6MAMwfL._SS36_.jpg"
},
{
"asin": "B09K476WW1",
"selected": false,
"dimensions": {
"Color": "Black Gray"
},
"tooltip_image": "https://m.media-amazon.com/images/I/414I6MAMwfL._SS36_.jpg"
},
{
"asin": "B09XJVXDGL",
"selected": false,
"dimensions": {
"Color": "Camo LED"
},
"tooltip_image": "https://m.media-amazon.com/images/I/414I6MAMwfL._SS36_.jpg"
},
{
"asin": "B0B5CD5QL3",
"selected": false,
"dimensions": {
"Color": "Blue"
},
"tooltip_image": "https://m.media-amazon.com/images/I/414I6MAMwfL._SS36_.jpg"
}
],
"has_videos": true,
"sales_rank": [
{
"rank": 101,
"ladder": [
{
"url": "/gp/bestsellers/videogames/ref=pd_zg_ts_videogames",
"name": "Video Games "
}
]
},
{
"rank": 2,
"ladder": [
{
"url": "/gp/bestsellers/videogames/405593011/ref=pd_zg_hrsr_videogames",
"name": "Mac Game Headsets"
}
]
}
],
"top_review": "For the price of this I was worried how well it would really work. To my surprise it’s like a top of the line product. Works better than I expected it to. The cord is longer than some but that’s what I needed for my set up. They also give a cord to hook it up to your pc if you are a pc gamer rather than a console gamer so it’s a win win for everyone. The ear pieces are bigger than I expected but I like this as it actually slows the noise cancellation to actually work. You can’t hear someone screaming at you for a long period of time. The sound quality is amazing considering this isn’t one of the best known brands out there. Usually you have to pay more money to get good quality. This however is not the case. Been extremely satisfied since receiving this and has been used on a daily basis for about a month now with no issues. The mouthpiece to talk into you can adjust it more than you think it’s made of this bendy rubber material that allows you to bend it inwards or outwards to make your voice clearer on the receiving end. Most models do not offer this so this is a real welcomed bonus! I highly recommend this product. You can wear the headset for an extended period of time like a few hours and it not give you a headache or earache. Look no further this is the best headset out of at least a dozen that I’ve bought and used.\n \nRead more",
"asin_in_url": "B09H74FXNW",
"description": "1. Extremely Comfortable Wearing: the gaming headphones with microphone uses protein memory foam padded headband, cozy to long time wear. Flexible extendable headband, fits different people's demand. Fully-wrapped earmuff will not press your ear.",
"price_upper": 20.98,
"pricing_str": "",
"pricing_url": "https://www.amazon.com/gp/offer-listing/B09H74FXNW?startIndex=0",
"discount_end": "",
"manufacturer": "BettergamesBetterlife",
"max_quantity": 5,
"price_buybox": 20.98,
"product_name": "Gaming Headset with Microphone, Gaming Headphones Stereo 7.1 Surround Sound PS4 Headset 50mm Drivers, 3.5mm Audio Jack Over Ear Headphones Wired for PC Switch Playstation Xbox PS5 Laptop",
"bullet_points": "Make sure this fits by entering your model number.\nImmersive Stereo - Gaming headset with microphone is 7.1 Surround Sound from our professional gaming headsets, outfitted with 50mm neodymium magnet driver, gives vivid and clear sound field, deep bass. Accurate positional audio across any game environment, our gaming headphones with microphone help you get better performance in game, .ps4 games, ps5 games, xbox one games, wii games, etc.\nAll-day Comfort - Extremely light design for all-day wearing, gaming headset are padded with breathable and durable protein soft memory foam cushions, preventing overheating and pressure build-up and pc gaming headset is built with adjustable headband. A must-have PC headsets, xbox headset, ps4 headset, every gamer will enjoy with this wired headphones.\n360° Noise Cancelling Mic - 160° retractable, 360° omnidirectionally rotatable sensitive microphone, premium noise cancellation, sound localisation, this gaming headset pc helps pick up sound quickly, eliminate unwanted noise from environment, deliver messages clearly during gameplay. Adjustable microphone can be swivelled to the right position at any time.\nIn-line Audio Controls - This PC game headsets have an volume control wheel and mic mute switch on the gaming headset ps4 with microphone for quick and convenient adjustments. High tensile strength, anti-entanglement braided USB cable prevents the ps4 headset with mic cable from chaos, and help easily control the volume or one-key microphone mute.\nUniversally Compatible - xbox headphones wired Support PlayStation 4, New Xbox One, PC, Nintendo 3DS, Laptop, PSP, Tablet, iPad, Computer, Mobile Phone. Please note you need an extra Microsoft Adapter (Not Included) when this xbox one headset connect with an old version Xbox One controller, Sega Genesis.",
"is_addon_item": false,
"price_initial": 0,
"pricing_count": 0,
"reviews_count": 2239,
"sns_discounts": [],
"developer_info": [],
"lightning_deal": null,
"price_shipping": 0,
"is_prime_pantry": false,
"product_details": {
"asin": "B09H74FXNW",
"item_weight": "1.17 pounds",
"manufacturer": "FEIYING",
"customer_reviews": "4.4 out of 5 stars 2,239 ratings 4.4 out of 5 stars",
"best_sellers_rank": "#101 in Video Games (See Top 100 in Video Games) #2 in Mac Game Headsets",
"item_model_number": "K5MC",
"package_dimensions": "9.49 x 7.36 x 4.61 inches",
"date_first_available": "August 25, 2021"
},
"featured_merchant": {
"link": "/gp/help/seller/at-a-glance.html/ref=dp_merchant_link?ie=UTF8&seller=A99HKVS6N0RPL&asin=B09H74FXNW&ref_=dp_merchant_link&isAmazonFulfilled=1",
"name": "Fandraw US",
"seller_id": "A99HKVS6N0RPL",
"shipped_from": "Amazon",
"is_amazon_fulfilled": true
},
"is_prime_eligible": false,
"parse_status_code": 12005,
"answered_questions_count": 39,
"rating_stars_distribution": [
{
"rating": 5,
"percentage": 72
},
{
"rating": 4,
"percentage": 13
},
{
"rating": 3,
"percentage": 6
},
{
"rating": 2,
"percentage": 3
},
{
"rating": 1,
"percentage": 7
}
]
},
"status_code": 200,
"url": "https://www.amazon.com/dp/B09H74FXNW",
"task_id": "6971060034289555457",
"created_at": "2022-09-01 11:03:48",
"updated_at": "2022-09-01 11:04:13"
}
]
}
Programming Language | Example location | Download |
---|---|---|
Python | python/amazon.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/amazon.py > amazon.py |
PHP | php/amazon.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/amazon.php > amazon.php |
Node.js | nodejs/amazon.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/amazon.js > amazon.js |
Required parameters: query
Target by supplying Amazon product ID (ASIN) using the query parameter.
Parameter | Type | Description |
---|---|---|
query |
string |
Amazon product ID (ASIN) |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/amazonproduct.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/amazonproduct.py > amazonproduct.py |
PHP | php/amazonproduct.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/amazonproduct.php > amazonproduct.php |
Node.js | nodejs/amazonproduct.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/amazonproduct.js > amazonproduct.js |
Required parameters: query
Retrieve Amazon pricing results by supplying Amazon product ID (ASIN).
Parameter | Type | Description |
---|---|---|
query |
string |
Amazon product ID (ASIN) |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/amazonpricing.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/amazonpricing.py > amazonpricing.py |
PHP | php/amazonpricing.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/amazonpricing.php > amazonpricing.php |
Node.js | nodejs/amazonpricing.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/amazonpricing.js > amazonpricing.js |
Required parameters: query
Retrieve Amazon reviews by supplying Amazon product ID (ASIN).
Parameter | Type | Description |
---|---|---|
query |
string |
Amazon product ID (ASIN) |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/amazonreviews.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/amazonreviews.py > amazonreviews.py |
PHP | php/amazonreviews.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/amazonreviews.php > amazonreviews.php |
Node.js | nodejs/amazonreviews.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/amazonreviews.js > amazonreviews.js |
Required parameters: query
Retrieve Amazon questions and answers by supplying Amazon product ID (ASIN).
Parameter | Type | Description |
---|---|---|
query |
string |
Amazon product ID (ASIN) |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/amazonquestions.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/amazonquestions.py > amazonquestions.py |
PHP | php/amazonquestions.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/amazonquestions.php > amazonquestions.php |
Node.js | nodejs/amazonquestions.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/amazonquestions.js > amazonquestions.js |
Required parameters: query
Retrieve Amazon listings by supplying your query to the Amazon search bar.
Parameter | Type | Description |
---|---|---|
query |
string |
Amazon search query |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/amazonsearch.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/amazonsearch.py > amazonsearch.py |
PHP | php/amazonsearch.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/amazonsearch.php > amazonsearch.php |
Node.js | nodejs/amazonsearch.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/amazonsearch.js > amazonsearch.js |
Required parameters: query
Retrieve Amazon listings by supplying your query to the Amazon search bar.
Parameter | Type | Description |
---|---|---|
query |
string |
Bestseller category ID |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/amazonbestsellers.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/amazonbestsellers.py > amazonbestsellers.py |
PHP | php/amazonbestsellers.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/amazonbestsellers.php > amazonbestsellers.php |
Node.js | nodejs/amazonbestsellers.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/amazonbestsellers.js > amazonbestsellers.js |
Wayfair can be targeted using direct URL, by providing the query parameter to the Wayfair search bar, or by providing the Wayfair product ID.
Required parameters: url
Target by supplying direct URL link.
Parameter | Type | Description |
---|---|---|
url |
url |
Wayfair product URL |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/wayfair.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/wayfair.py > wayfair.py |
PHP | php/wayfair.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/wayfair.php > wayfair.php |
Node.js | nodejs/wayfair.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/wayfair.js > wayfair.js |
Required parameters: query
Target by supplying query parameter to the Wayfair search bar.
Parameter | Type | Description |
---|---|---|
query |
string |
Wayfair search query |
target |
string |
Scraping target |
Programming Language | Example location | Download |
---|---|---|
Python | python/wayfairsearch.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/wayfairsearch.py > wayfairsearch.py |
PHP | php/wayfairsearch.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/wayfairsearch.php > wayfairsearch.php |
Node.js | nodejs/wayfairsearch.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/wayfairsearch.js > wayfairsearch.js |
Retrieve results from any eCommerce website by supplying a full URL of the website to the url parameter. In this case, additional parameters can be specified in the URL itself.
Required parameters: url
Parameter | Type | Description |
---|---|---|
url |
string |
Ecommerce site url |
target |
string |
Scraping target |
Note that to get a result with the AI parser, you will need to set the parse
parameter as 'true'
and parser_type
parameter as ecommerce_product
.
Programming Language | Example location | Download |
---|---|---|
Python | python/ecommerce.py | curl hhttps://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/python/ecommerce.py > ecommerce.py |
PHP | php/ecommerce.php | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/php/ecommerce.php > ecommerce.php |
Node.js | nodejs/ecommerce.js | curl https://github.com/raw/Smartproxy/eCommerce-Scraping-API/main/nodejs/ecommerce.js > ecommerce.js |
amazon
amazon_product
amazon_pricing
amazon_reviews
amazon_questions
amazon_search
amazon_sellers
amazon_bestsellers
wayfair
wayfair_search
ecommerce
Parameter | Type | Description |
---|---|---|
target |
string |
Data source. Available targets are listed here. |
url |
url |
Direct URL (link) |
locale |
string |
This will change the web interface language. Example: – en-US – en-GB |
geo |
string |
The geographical location that the result depends on. City location names, state names, country names, coordinates and radius, Google’s Canonical |
device_type |
string |
Device type and browser. Supported: desktop , desktop_chrome , desktop_firefox , mobile , mobile_android , mobile_ios . |
headless |
string |
Enable JavaScript rendering. Supported: html , png |
parser_type |
string |
Enable product page parsing for ecommerce target |
Response | Description | Solution |
---|---|---|
200 - Success | Server has replied and given requested response. | Celebrate! |
204 - No content | Job not completed yet. | Wait a few seconds before trying again. |
400 - Multiple error messages | Bad structure of the request. | Re-check your request to make sure it is in the correct format. |
401 - Invalid / not provided authorization header (client not found) | Incorrect login credentials or missing authorization. | Re-check your provided credentials for authorization. |
403 - Forbidden | Your account does not have access to this resource. | Make sure the target is supported by us |
404 - Not found | Your target was not found. | Re-check your targeted URL. |
429 - Too many requests | Exceeded rate limit for your subscription. | Make sure you still have at least one request left. Wait a couple minutes and try again. If you are encountering the error often – chat with us to see if your rate limit can be increased. |
500 - Internal error | Service unavailable, possibly due to some issues we are encountering. | Wait a couple minutes and send another request. Contact us for more information. |
524 - Timeout | Service unavailable, possibly due to some issues we are encountering. | Wait a couple minutes and send another request. Contact us for more information. |
Response | Description |
---|---|
12000 - Success | Server has replied and given the requested response. |
12002 - Error | Parsing has failed completely. |
12003 - Not supported | Targeted website parsing is not supported. |
12004 - Response not full | Some fields were not parsed and are missing. |
12005 - Response not fully parsed | Some fields might not have been parsed and are returned unparsed. |
12006 - Error | Unexpected error. Let us know the task ID and we will check what went wrong. |
12007 - Unknown | We could not determine whether the data was parsed correctly. |
12008 - Error | Failed to parse all the data. |
12009 - Error | Target not found. Make sure the parameters you passed are correct and supported. |
All code is released under MIT License