Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Satellite] Cannot set property statusCode of Error which has only a getter #2755

Closed
humphd opened this issue Jan 27, 2022 · 10 comments
Closed
Assignees
Labels
area: microservices area: satellite Issues related to the Satellite microservice project dependencies Pull requests that update a dependency file type: bug Something isn't working
Milestone

Comments

@humphd
Copy link
Contributor

humphd commented Jan 27, 2022

While I was testing #2753, I ran into a crash in our error handling code:

TypeError: Cannot set property statusCode of Error which has only a getter
    at createError (/app/node_modules/http-errors/index.js:101:33)
    at /app/src/routes/query.js:13:10
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
{
  error: ResponseError: index_not_found_exception
      at onBody (/app/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
      at IncomingMessage.onEnd (/app/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
      at IncomingMessage.emit (node:events:402:35)
      at endReadableNT (node:internal/streams/readable:1343:12)
      at processTicksAndRejections (node:internal/process/task_queues:83:21) {
    meta: {
      body: [Object],
      statusCode: 404,
      headers: [Object],
      meta: [Object]
    }
  }
} query error
/app/node_modules/http-errors/index.js:101
    err.status = err.statusCode = status

This is being caused by the following code in the search servce:

// search.js

const search = async (
  textToSearch,
  filter = 'post',
  page = 0,
  perPage = ELASTIC_MAX_RESULTS_PER_PAGE
) => {
  ...
  const {
    body: { hits },
  } = await client.search({
    from: calculateFrom(page, perPage),
    size: perPage,
    _source: ['id'],
    index,
    type,
    body: query,
  });

  return {
    results: hits.total.value,
    values: hits.hits.map(({ _id }) => ({ id: _id, url: `${POSTS_URL}/${_id}` })),
  };
};

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

// query.js
router.get('/', validateQuery, async (req, res, next) => {
  try {
    const { text, filter, page, perPage } = req.query;
    res.send(await search(text, filter, page, perPage));
  } catch (error) {
    console.log({ error }, 'query error');
    next(createError(503, error));   // <-- crash is here
  }
});

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

// http-errors/index.js

  if (!HttpError || !(err instanceof HttpError) || err.status !== status) {
    // add properties to generic error
    err.expose = status < 500
    err.status = err.statusCode = status
  }

So what's happening here is that we do a call to Elasticsearch, and because there is no index built yet (I hadn't started the parser), it fails (Promise rejects). This takes us to the catch block in the router, and we try to createError() using 503 and the err object. This calls into this code (not ours): https://github.com/jshttp/http-errors/blob/master/index.js#L42-L105. It is trying to decorate the error with a status code property, but it fails:

TypeError: Cannot set property statusCode of Error which has only a getter
    at createError (/app/node_modules/http-errors/index.js:101:33)
    at /app/src/routes/query.js:13:10
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
{
  error: ResponseError: index_not_found_exception
      at onBody (/app/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
      at IncomingMessage.onEnd (/app/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
      at IncomingMessage.emit (node:events:402:35)
      at endReadableNT (node:internal/streams/readable:1343:12)
      at processTicksAndRejections (node:internal/process/task_queues:83:21) {
    meta: {
      body: [Object],
      statusCode: 404,
      headers: [Object],
      meta: [Object]
    }
  }
}

This causes the server to crash. This same problem will affect all of our microservices.

We should add a test to Satellite here that tries to pass an Error object vs. a string as the second arg, and figure out whose bug this is: ours, elasticsearch's, or http-errors.

@humphd humphd added type: bug Something isn't working dependencies Pull requests that update a dependency file area: microservices area: satellite Issues related to the Satellite microservice project labels Jan 27, 2022
@Kevan-Y Kevan-Y self-assigned this Feb 3, 2022
@RC-Lee
Copy link
Contributor

RC-Lee commented Feb 12, 2022

After doing tests for Search, I think its us.
We might need a Try-catch statement around the client.search(), like I've mentioned in #2892

@AmasiaNalbandian
Copy link
Contributor

While working on my branch for the search bar with the latest changes from master, I ran into the same issue. I fixed the cause on my end, but will need to confirm it again after I hook up everything to work from 8080 with the front end.

@humphd
Copy link
Contributor Author

humphd commented Feb 17, 2022

Would love a fix for this, thanks @AmasiaNalbandian!

@AmasiaNalbandian
Copy link
Contributor

After exploring this, I found the reason we keep getting this error. The issue is when we try to call elastic port which is http://127.0.0.1, it doesn't work. The work around for development right now is to change env.development file

ELASTIC_URL=http://127.0.0.1

to use ELASTIC_URL=http://localhost

http://127.0.0.1 is returning a 404 page not found error.

Unfortunately, this is completely blocking search - however I'm using the workaround to work on both right now.

@RC-Lee
Copy link
Contributor

RC-Lee commented Feb 18, 2022

I've updated the tests.

I'm not sure if I should post this in Satellite or here. But, the ES Error change was in 7.16, it's written here in the documentation. No direct link, but its right under the first code block

@humphd
Copy link
Contributor Author

humphd commented Feb 18, 2022

See my comment in your PR about normalizing this automatically in createError

@AmasiaNalbandian
Copy link
Contributor

@humphd can you confirm this happens locally only?

@humphd
Copy link
Contributor Author

humphd commented Feb 23, 2022

I've only seen it happen locally, but I wonder if this is what the feed-discovery service is doing when it crashes.

@RC-Lee
Copy link
Contributor

RC-Lee commented Feb 26, 2022

Wondering if any similar crashes are still happening after the fix in Satellite

@humphd
Copy link
Contributor Author

humphd commented Feb 26, 2022

Not sure. We can close this for now and re-open if we see it again.

@humphd humphd closed this as completed Feb 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: microservices area: satellite Issues related to the Satellite microservice project dependencies Pull requests that update a dependency file type: bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants