Discussion: What are you wanting to do with mws ?

[ericblade]

Hi all! Just noticed that there's been a few clones of this repo, and thought I'd ask what other people are wanting to do with this?

Personally, I'm a part time Amazon merchant, and I have wanted to build a system that can locally store a list of all of my products, and keep tabs on orders and shipments automatically, without having to go and hit Seller Central .. and something that will help make sense out of the huge quantity of reports that are available.

Lately, when I've had time to work on this, I've been playing with the report downloading functions, as that seems to be where the bulk of everything is. The real time APIs are interesting, but don't seem to be geared towards the sort of information that I'm looking to get in touch with.

A far out there goal, is that I'd like to build something that can replace the Seller Central FBA shipment workflow, and play with ways to make that much easier for me.

[tmvanetten]

@ericblade I have similar needs. In my day job my team and I are solving the challenge of getting all the Seller Central data and reports pulled into our tools where we can use the data as we like. Putting a nice GUI around all the scrapes, api requests, feeds, etc is something I am working on.

[ericblade]

Yeah, I've spent entirely too long pulling reports on Seller Central, and trying to cross-reference all the data across different CSV reports inside OpenOffice. Keeping track of costs, fees, and sales, inside a spreadsheet works, but doesn't give me lots of warm fuzzies.

I think my first goal is to go over all the data in the Inventory reports, import it into a database, and be able to associate local data to the Amazon data.

[ericblade]

I'm hoping to get back to writing some more support for this over the next few weeks.. things have been a bit on the rough side, had a long trip for my day job, which ended up with my personal laptop that i usually use for development stolen. ugh! And now life is getting in the way of writing more code outside of work :-)

I'm hoping to put together some real unit testing, now that i've validated that at least the basics of this actually work, as well as do some decent research into how to pick apart the data that comes back from reporting, while some way or another making the data fetch routines also understand throttling, and hopefully automating throttle handling to a better degree than simply using a sleep() utility with a constant argument.

[aflansburg]

@ericblade @tmvanetten I'm reading through some of these comments and it sounds like I'm on a similar path. I want to be able to do some custom analytics against things from our Amazon Seller Account. I'm interested in both using and contributing to this - so if you have any to-dos let me know.

[ericblade]

While life and work have been in the way of my doing much more work on this recently, I did have some inspiration last weekend that allowed me to get some nice stuff in. My primary goal when I set out was to make a library that uses modern tech, documents it's own things, without expecting you to already understand the entire MWS API, and hopefully returns values in a way that make sense in a Javascript environment versus the typical XML-Like responses that others give out.

That said, I didn't really have any design, I just started writing code. So I don't have a whole big todo list, other than what I mark with // TODO when I think of something.

Recently, inspired by work I've been doing with a python project called "troposphere", which does pre-validation before calling Amazon's CloudFormation APIs, I decided to add validation of parameters, which also allows transformation as well, so it makes it easy to do things like accept Javascript Date objects and send them out to the underlying API as ISO strings. Have a look at the last few commits in here, and you can probably see what I was aiming at. That's something that will take a while to complete, poring over the not-always-so-good MWS docs, and translating them into a JSON structure for each category. I had initially mulled over the idea of actually fetching the docs, and parsing them to turn them into a JSON structure .. but the format is so widely variable between pages, that it looked like a bad idea.

[ericblade]

Hey all :) Couldn't sleep this morning, so I decided to sit down and do a little more work on the parameter validation.. which led to me realizing that it was getting a little cramped in some of these files and the base directory.. so i just pulled a major re-organization to shove all the internal operations stuff a directory or two down from the higher level stuff. I'm doing a little background thinking on how to best breakup what remains in index.js, as well as how i might handle the higher level stuff better (along with return value validation)

[ericblade]

... and I ended up burning way more time than I expected yesterday on this project :) added automatic eslinting, Mocha/Chai testing, code coverage reports with Istanbul/nyc/codebeat . . all of which found a lot of bugs in my refactoring / not yet implemented in testing code.. plenty of value was achieved.

[aflansburg]

Wasn't sure if this would be useful or not - but I did run the tests - trying to keep up - been a busy holiday season, especially at work. I admit, I'm a bit new to proper testing, but am familiar enough with Mocha (not overly comfortable). What's the best place to get started?

sanity-test-results-aflansburg.pdf

Results weren't consistent each time. I've run a second time and only the listInventorySupply unit had errors:

AssertionError: expected undefined to be an array at Context.it (test\test-sanity.js:377:45) at <anonymous> at process._tickCallback (internal/process/next_tick.js:188:7)

[ericblade]

@aflansburg i found myself with a lot more time during the holidays than i thought -- since you inspired me to bump this up to the next level :) was hoping you hadn't disappeared on me hehe

I'm extremely new to mocha/chai, so seeing that nice professional looking PDF is interesting. And there's probably better ways that I can do validation that I'm not aware of yet.

As far as getting most of your tests to run there, increasing the timeout should work. "npm test" should run it with --timeout=5000 or you can run it manually with a larger timeout value.

A problem that I have with the API tests - since there's no sandbox environment, that has a relaxed or non-existant throttle.. I'm afraid that following the normal pattern of performing all the steps necessary for each test IN each test, could cause the throttle limits on many API calls to be reached long before testing is complete. Some API calls only regenerate a few times an hour! SO.. the GetOrder test relies on the results of listOrders, because GetOrder needs an order number to work. I'm not sure if there's a way to skip a test that cannot be performed? Something I can probably Google real quick though.

The other thing about working on live data, is that your account, and my account, are probably quite different, and I'm not entirely certain how to adjust for them. On the bright side, that will help us find data errors :)

I think increasing the timeout will fix all your problems there except listInventorySupply -- are you able to comment if you have FBA inventory on hand? Without being able to see the real data, I'm guessing that FulfillmentInventory/ListInventorySupply is either returning an empty result, and I'm handling it wrong, or it's returning a vastly different result than what I got when I tried it with a few different queries. If you don't mind pasting the results of a callEndpoint() to that, I can have a look, or you can dive right in there in lib/list-inventory-supply.js :-)

const mws = require('mws-advanced');
const keys = require('./keys.json');

mws.init(keys);

async function main() {
    try {
        const startDate = new Date();
        startDate.setDate(startDate.getDate() - 7);
        const result = await mws.callEndpoint('ListInventorySupply', { QueryStartDateTime: startDate });
        console.warn(JSON.stringify(result, null, 4));
    } catch (err) {
        console.warn('* error', err);
    }
}

main();

looks like it should do the trick. Or stick a console.log in list-inventory-supply.js after the callEndpoint .. or hit it with a debugger breakpoint right there.

I'm going to go make an Issue for that specific item, remove that log in getReport test, and see if there is a way to maybe mark a test as skipped if it's not possible to run it, due to not having data.

[ericblade]

fwiw, there were problems with Feeds related to mws-simple, and a total lack of support in mws-advanced. mws-advanced now supports providing a feedContent parameter to callEndpoint that will be straight passed through to mws-simple, and mws-simple=2.2.3 has feed submission fixed. SO, feed submission all works out. I haven't yet worked out how I want to make it easier to build/submit feeds, but I'm sure there's plenty of stuff that can be done on that. I'll submit a separate issue to discuss that if anyone wants to.

[ericblade]

Someone asked a question about getting a list of all inventory.. if you've got any merchant fulfilled inventory, it seems that you'll need to pull reports to get that, and you can do that for FBA as well.. for grabbing all inventory, I parse these reports:

        '_GET_FBA_MYI_UNSUPPRESSED_INVENTORY_DATA_',
        '_GET_MERCHANT_LISTINGS_DATA_',
        '_GET_MERCHANT_LISTINGS_DATA_LITE_',
        '_GET_AFN_INVENTORY_DATA_',

You can use, I think, listInventorySupply, to grab everything currently onhand FBA, if you give it a start date that is prior to any of your current onhand inventory being shipped in.

[alexis-de-la-torre]

It was me that asked, then i read the amazon documentation and figured out is a 3 step proccess.

Step 1:
Request a GET_FLAT_FILE_OPEN_LISTINGS_DATA report

Step 2:
Use the ReportRequestId that you got on the first step and use it on the GetReportRequestList endpoint

Step 3:
use the GeneratedReportID from step 2 on the getReport endpoint

Tahs a lot of steps to do something really basic.

[ericblade]

Yeah, that might also work, I'll need to see what all data is contained in that report, might be that I needed more info for my purposes than that report alone produced, or maybe i just never looked at that one.

If you're using this lib, although it's not entirely bulletproof, you can simply do await mws.requestAndDownloadReport('_GET_FLAT_FILE_OPEN_LISTINGS_DATA_', './output.json');

[alexis-de-la-torre]

yeah, the data returned is just inventory, price and asin of the products.
What i will do is cross reference that info with the GetMatchingProductForId endpoint.

You have a really good library here btw, for the moment i am using your fork of the mws-simple because iam learning the mws api, i think when i understand how mws works and all the endpoints i will graduate to mws-advanced