Move Satellite into Telescope (#3136)

* Initial work * Docs and bodyParser * fix: correct typo and add missing quotation mark * Move router onto Satellite instance * Use built-in body parsing with Express * Add tests for body parsers * Add GitHub CI + README badge * Fix package-lock.json sync with package.json * Fix README build badge URL * Add release workflow * 1.0.1 * Use standard tag format for npm with v prefix * 1.0.2 * Release yaml fix * 1.0.3 * Explicitly set package public for npm publish * 1.0.4 * Remove private field completely in order to publish to npm * 1.0.5 * Switch org name to @senecacdot * 1.0.6 * Update lock file * 1.0.7 * Expose Router from package * 1.1.0 * Switch from new Router() to Router() * 1.1.1 * Use env variables to start apm monitoring sooner * 1.2.0 * Switch to ELASTIC_APM_SERVER_URL, better 404 reporting, refactor Router() * 1.3.0 * Add beforeParsers and beforeRouter options with tests * 1.4.0 * Add pino-colada for debug logging * Improve logging, use ELASTIC_APM_SERVICE_NAME env var, add router option to ctor * 1.5.0 * Remove pino-tiny dep * Fix logger picking logic on startup * 1.5.1 * Document healthCheck and add more tests * 1.5.2 * Add default favicon support * 1.6.0 * Update README install instructions, deps * 1.6.1 * Add JWT validation, tests, and update docs * 1.7.0 * Ported Hash to Satellite * Removed Redundant Code, Added Comment Block, Fixed Import * Re-add crypto * Add test for req.user * Refactor into src/, breakup middleware authenticate vs. authorize, remove favicon * 1.8.0 * Init Prettier Commit * Adds the createError module for use in Telescope (#5) * Fixed CreateError module, use http-errors * removed merge conflict errors * Added Docs in README Co-authored-by: David Humphrey <david.andrew.humphrey@gmail.com> * Finish prettier integration * Fix workflows * Prettier for jest.config.js * Update deps, fix prettier-check on windows * 1.8.5 * Specify main entry point in package.json * 1.8.6 * Add .husky directory and pre-commit hook * Fix #1930 * Support credentials for HTTPS vs. HTTP server * 1.8.7 * Don't install Husky on postinstall * 1.8.8 * Add support for generating a service token * 1.9.0 * Updated redis and added ping test * Updated redis export * Fixed Redis test case * 1.10.0 * isAuthorized() always takes a function with req, user params * 1.11.0 * 1.12.0 * Initial Elastic client code * Updated elastic contructor, add initial tests * Fixed elastic search client, mock elastic connection * Updated README.md with Elastic() info * 1.13.0 * Add shutDown() to allow killing connections * 1.14.0 * Add automatic, graceful shutdown for Redis and Elastic clients * Update deps for 1.14.0 * 1.15.0 * Initial exported Fetch() function to Satellite * Updated exports to require node-fetch instead of it being in a separate file * Updated spelling to 'fetch' and updated tests to use nock * Removed done() from tests and moved node-fetch to be a dependency vs dev-dependency * Update lock file * 1.16.0 * chore: include nodejs 16 in the CI build matrix * feat: add auto-opt-out of FLoC * 1.17.0 * Add eslint to satellite * adding and configuring eslint * Fixing linting errors * configuring anti-trojan-source plugin * adding lint to pre-commit hook * Removing ts from eslint run and removing comment * removing unused function validateAuthorizationOptions * Adding no-unused-vars override to test.js and removing the override from config * Adding ESLint to CI runs * Integrating eslint with the release workflow * Fix Dependencies: * Remove elastic-apm-node * Remove @elastic/ecs-pino-format * Update Jest (To fix deprecated dependencies) * * Update pino * Switch from express-pino-logger to pino-http * Standardize Dependencies * Configure Renovate (#23) * Configure renovate bot Co-authored-by: Renovate Bot <bot@renovateapp.com> Co-authored-by: Duke Manh <manhducdkcb@gmail.com> * chore(deps): update dependency pretty-quick to v3.1.3 * fix(deps): update dependency express to v4.17.2 * fix(deps): update dependency node-fetch to v2.6.7 * fix(deps): update dependency @elastic/elasticsearch-mock to v0.3.1 * fix(deps): update dependency http-errors to v1.8.1 * chore(deps): update dependency eslint to v8.7.0 * chore(deps): update dependency eslint-plugin-anti-trojan-source to v1.1.0 * chore(deps): update dependency eslint-plugin-jest to v25.7.0 * chore(deps): update dependency husky to v5.2.0 * Switch npm to pnpm * Release v1.18.0 * 1.20.0 * 1.21.0 * Use --no-git-checks with pnpm publish to avoid failure on CI * 1.22.0 * remove pre-commit * Release v1.23.0 * fix(deps): update dependency pino-pretty to v7.5.1 * fix(deps): update dependency pino to v7.6.5 * chore(deps): update dependency eslint to v8.8.0 * chore(deps): update dependency nock to v13.2.2 * bump prettier to v2.5.1 and run prettier on entire tree * fix(deps): update dependency @elastic/elasticsearch to v7.16.0 * fix(deps): update dependency @godaddy/terminus to v4.10.2 (#48) Co-authored-by: Renovate Bot <bot@renovateapp.com> * fix(deps): update dependency express-jwt to v6.1.0 * fix(deps): update dependency ioredis to v4.28.3 (#50) Co-authored-by: Renovate Bot <bot@renovateapp.com> * fix(deps): update dependency ioredis-mock to v5.9.1 * fix-renovate-bot * Refactoring elastic.js so mock is exported for tests - Adding tests for mock Elastic() - Added mock Elastic() description in README.md * Release v1.24.0 * chore(deps): update dependency nock to v13.2.4 * fix(deps): update dependency ioredis to v4.28.4 (#55) Co-authored-by: Renovate Bot <bot@renovateapp.com> * chore(deps): update dependency jest to v27.5.0 (#56) Co-authored-by: Renovate Bot <bot@renovateapp.com> * fix(deps): update dependency ioredis to v4.28.5 * fix(deps): update dependency @elastic/elasticsearch to v7.17.0 * chore(deps): update dependency jest to v27.5.1 (#59) Co-authored-by: Renovate Bot <bot@renovateapp.com> * chore(deps): update dependency eslint to v8.9.0 (#60) Co-authored-by: Renovate Bot <bot@renovateapp.com> * changed all uses of SECRET -> JWT_SECRET * Release v1.25.0 * Adding more tests for createError * fix(deps): update dependency express to v4.17.3 * fix(deps): update dependency pino to v7.8.0 (#66) Co-authored-by: Renovate Bot <bot@renovateapp.com> * adding ES error cases to createError * Release v.1.26.0 * fix(deps): update dependency express-jwt to v6.1.1 * chore(deps): update dependency eslint to v8.10.0 * fix(deps): update dependency ioredis-mock to v7 * fix(deps): update dependency ioredis-mock to v7.1.0 * fix(deps): update dependency pino-pretty to v7.5.3 * chore(deps): update dependency husky to v7 * chore(deps): update dependency eslint-plugin-jest to v26 * fix(deps): update dependency helmet to v5 * set default values for status and argToSend so they're not undefined * Delete unneded config files from Satellite repo Co-authored-by: David Humphrey <david.andrew.humphrey@gmail.com> Co-authored-by: Josue <josue.quilon-barrios@senecacollege.ca> Co-authored-by: Metropass <moho472@gmail.com> Co-authored-by: Mo <58116522+Metropass@users.noreply.github.com> Co-authored-by: Abdulbasid Guled <guled.basid@gmail.com> Co-authored-by: Josue <manekenpix@fastmail.com> Co-authored-by: dhillonks <kunwarvir@hotmail.com> Co-authored-by: Kevan-Y <58233223+Kevan-Y@users.noreply.github.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Renovate Bot <bot@renovateapp.com> Co-authored-by: Duke Manh <manhducdkcb@gmail.com> Co-authored-by: Cindy Le <cindyledev@gmail.com> Co-authored-by: AmasiaNalbandian <amasia.nalbandian@mitel.com> Co-authored-by: Amasia <77639637+AmasiaNalbandian@users.noreply.github.com> Co-authored-by: rclee91 <32626950+rclee91@users.noreply.github.com> Co-authored-by: Jia Hua Zou <jiahua.zou1@gmail.com> Co-authored-by: Joel Azwar <joel_azwar@yahoo.com> Co-authored-by: Anatoliy Serputoff <65831678+aserputov@users.noreply.github.com> Co-authored-by: tpmai <thienphuoc.0108@gmail.com>
Seneca-CDOT · Mar 8, 2022 · 93299b4 · 93299b4
1 parent 8360d5d
commit 93299b4
Show file tree

Hide file tree

Showing 15 changed files with 1,964 additions and 0 deletions.
diff --git a/src/satellite/.eslintrc.js b/src/satellite/.eslintrc.js
@@ -0,0 +1,20 @@
+module.exports = {
+  env: {
+    node: true,
+    commonjs: true,
+    es2021: true,
+    jest: true,
+  },
+  extends: 'eslint:recommended',
+  parserOptions: {
+    ecmaVersion: 13,
+  },
+  plugins: ['anti-trojan-source', 'jest'],
+  rules: {
+    /**
+     * Halt if a trojan source attack is found
+     * https://github.com/lirantal/eslint-plugin-anti-trojan-source
+     */
+    'anti-trojan-source/no-bidi': 'error',
+  },
+};
diff --git a/src/satellite/README.md b/src/satellite/README.md
@@ -0,0 +1,337 @@
+# Satellite
+
+![node-js-ci](https://github.com/Seneca-CDOT/satellite/workflows/node-js-ci/badge.svg)
+
+A Microservice Framework for [Telescope](https://github.com/Seneca-CDOT/telescope).
+Because Ray said we should try microservices!
+
+Satellite creates an [Express.js](http://expressjs.com/) based server with
+various common pieces already set up. Bring your own router and let us do the rest.
+
+## Install
+
+```
+npm install --save @senecacdot/satellite
+```
+
+## Configure
+
+The following JWT verification values are required:
+
+- `JWT_SECRET`: the secret used for JWT token verification
+- `JWT_AUDIENCE`: the audience (aud) claim expected in JWT token verification
+- `JWT_ISSUER`: the issuer (iss) claim expected in JWT token verification
+
+## Usage
+
+In its most basic form, a Satellite-based microservice looks like this:
+
+```js
+const {
+  Satellite, // the Satellite constructor
+  logger, // pre-configured logger
+} = require('@senecacdot/satellite');
+
+// Define your microservice
+const service = new Satellite();
+
+// Add your routes to the service's router
+service.router.get('/my-route', (req, res) => {
+  res.json({ message: 'hello world' });
+});
+
+// Start the service on the specified port
+service.start(8888, () => {
+  logger.info('Satellite Microservice running on port 8888');
+});
+```
+
+### `Satellite(options)`
+
+- `healthCheck`: an optional `function` returning a `Promise`, used to determine if the service is healthy. If no function is defined, a default one will be provided. The `healthCheck` function is what runs at the `/healthcheck` route by default.
+
+```js
+const service = new Satellite({
+  healthCheck: async () => {
+    // Connect to db and return a promise
+    const ok = await doSomeAsyncTask();
+    return ok;
+  },
+});
+```
+
+- `shutDown`: an optional `function` returning a `Promise`, used to run any shutdown logic (sync or async) before the server is shut down. If no function is defined, a default one will be provided.
+
+```js
+// Open connections to Redis and Elasticsearch
+const redis = Redis();
+const elastic = Elastic();
+
+const service = new Satellite({
+  // On shut down, close the open connection to Redis and Elasticsearch
+  shutDown: () => Promise.all([redis.quit(), elastic.close()]),
+});
+```
+
+- `cors`: the options to pass to the [cors](https://www.npmjs.com/package/cors) middleware. By default all options are turned on. Use `cors: false` to disable cors.
+
+- `helmet`: the options to pass to the [helmet](https://www.npmjs.com/package/helmet) middleware. By default all options are turned on. Use `helmet: false` to disable helmet.
+
+- `optOutFloc`: Enable adding the appropriate headers to Satellite to opt out of [Google's Floc](https://www.wired.com/story/google-floc-privacy-ad-tracking-explainer/). Disabled by default.
+
+```js
+const service = new Satellite({
+  options.optOutFloc: true,
+});
+```
+
+- `beforeParsers`: an optional hook function that allows access to the `app` during creation prior to adding the body parsers
+
+```js
+const service = new Satellite({
+  beforeParsers(app) {
+    // Optionally add some middleware before the parser are attached
+    app.use(myMiddlewareFunction());
+  },
+});
+```
+
+- `beforeRouter`: an optional hook function that allows access to the `app` during creation prior to adding the router.
+
+```js
+const service = new Satellite({
+  beforeRouter(app) {
+    // Optionally add some middleware before the router is attached
+    app.use(myMiddlewareFunction());
+  },
+});
+```
+
+- `router`: an optional router to use in place of the default one created automatically.
+
+```js
+const myRouter = require('./router);
+
+const service = new Satellite({
+  router: myRouter
+});
+```
+
+- `credentials`: an optional Object containing a `key` and `cert`, to be used in the creation of a secure HTTPS server. If `credentials` is not provided, an HTTP server is created instead (the default).
+
+```js
+const service = new Satellite({
+  credentials: {
+    key: fs.readFileSync('/path/to/privkey.pem'),
+    cert: fs.readFileSync('/path/to/fullchain.pem'),
+  },
+});
+```
+
+There are also a number of optional objects and functions available to further
+customize your service.
+
+### Router()
+
+Some services are easier to write using more than one router (e.g., defining
+complex routes in their own files). This is easily done with the `Router`:
+
+```js
+// custom-router.js
+const { Router } = require("@senecacdot/satellite");
+
+const router = Router();
+
+router.get('/custom-route', (req, res) => {...});
+router.post('/custom-route', (req, res) => {...});
+router.put('/custom-route', (req, res) => {...});
+router.delete('/custom-route', (req, res) => {...});
+
+module.exports = router;
+
+
+// index.js
+const { Satellite } = require("@senecacdot/satellite");
+const router = require('./custom-router");
+
+const service = new Satellite({
+  // Use our custom router instead of the default router
+  router
+});
+```
+
+### Middleware
+
+A number of middleware functions are available to help with your routes.
+
+- `isAuthenticated()` - used to make sure that a request includes a valid JWT and the user has previously been authenticated.
+
+```js
+router.get('/private-route', isAuthenticated(), (req, res) => {...});
+```
+
+- `isAuthorized()` - used to check if an authenticated user is authorized. NOTE: `isAuthorized()` must be used in conjunction with `isAuthenticated()`:
+
+Here are some examples:
+
+```js
+const { isAuthenticated, isAuthorized } = require("@senecacdot/satellite");
+
+// Authorize based on arbitrary user claims
+router.post(
+  '/:user',
+  isAuthenticated(),
+  isAuthorized(
+    // `user` is the decoded payload of the user's token.  Here we use it
+    // to make sure that the user param matches the user's `sub` claim,
+    // or that the user is an admin.
+    (req, user) => {
+      // Check if the user making the request is the same one for the route
+      if(user.sub === req.params.user) {
+        return true;
+      }
+
+      // If not, check if they are an admin
+      return user.roles.includes('admin');
+    }
+  ),
+  (req, res) => {...}
+);
+```
+
+### Logger
+
+The `logger` object is a pre-configured logger based on [Pino](https://getpino.io/#/).
+
+```js
+const { logger } = require('@senecacdot/satellite');
+
+logger.info('Hello World!');
+```
+
+### Hash
+
+The `hash()` function is a convenience hashing function, which returns a 10 character hash:
+
+```js
+const { hash } = require('@senecacdot/satellite');
+
+const id = hash('http://someurl.com');
+```
+
+### Create Service Token
+
+Services authorize requests using the `isAuthenticated()` and `isAuthorized()` middleware discussed above.
+For the most part, this is meant to be used for the case of user-to-service requests: an authenticated
+user passes a JWT token (acquired via the `auth` service), and uses it to request authorization to some
+protected route.
+
+However, in cases where you need to do a service-to-service request, you can use the `createServiceToken()`
+function in order to get a short-lived access token that will include the `"service"` role:
+
+```js
+const { createServiceToken } = require('@senecacdot/satellite');
+...
+const res = await fetch(`some/protected/route`, {
+  headers: {
+    Authorization: `bearer ${createServiceToken()}`,
+  },
+});
+```
+
+The receiving service can then opt-into allowing this service to be authorized by using
+the `isAuthenticated()` and `isAuthorized()` middleware like so:
+
+```js
+const { isAuthenticated, isAuthorized } = require("@senecacdot/satellite");
+
+// Allow requests with a token bearing the 'service' role to proceed
+router.get('/admin-or-service', isAuthenticated(), isAuthorized({ roles: ["service"] }), (req, res) => {...});
+```
+
+### Create Error
+
+The `createError()` function creates a unique HTTP Error Object which is based on [http-errors](https://www.npmjs.com/package/http-errors).
+
+```js
+const { createError } = require('@senecacdot/satellite');
+
+const e = createError(404, 'This is a message that describes your Error object');
+
+console.log(e.status); // of type: Number
+
+console.log(e.message); // of type: String
+```
+
+### Elastic
+
+The `Elastic()` function creates an instance of ElasticSearch connected to the elasticsearch url and port number specified in your config environment.
+An optional object can be passed to this function which will contain client settings that can be used with ElasticSearch.
+
+```js
+const { Elastic } = require('@senecacdot/satellite');
+
+const client = Elastic();
+
+const indexPost = async ({ text, id, title, published, author }) => {
+  try {
+    await client.index({
+      index,
+      type,
+      id,
+      body: {
+        text,
+        title,
+        published,
+        author,
+      },
+    });
+  } catch (error) {
+    logger.error({ error }, `There was an error indexing a post for id ${id}`);
+  }
+};
+```
+
+If `MOCK_ELASTIC=1` is set in the environment variables, the `Elastic()` function creates an instance of [ElasticSearch-mock](https://www.npmjs.com/package/@elastic/elasticsearch-mock) with a `mock` object for testing.
+
+```js
+const client = Elastic();
+const mock = client.mock;
+
+beforeEach(() => mock.clearAll());
+
+afterAll(() => {
+  mock.clearAll();
+});
+
+test('Should mock an API', async () => {
+  mock.add(
+    {
+      method: 'GET',
+      path: '/_cat/indices',
+    },
+    () => {
+      return { status: 'ok' };
+    }
+  );
+
+  const response = await client.cat.indices();
+  expect(response.body).toStrictEqual({ status: 'ok' });
+  expect(response.statusCode).toBe(200);
+});
+```
+
+### fetch
+
+The `fetch(url, options)` function will initiate an http request to an endpoint url specified by `url` with any options also specified by the `options` parameter.
+This function uses `node-fetch` on the side to make the http requests. Returns a promise.
+
+```js
+const { fetch } = require('@senecacdot/satellite');
+
+const response = await fetch('https://jsonplaceholder.typicode.com/posts');
+
+const data = await response.json();
+
+// Do whatever with the data you received
+```
diff --git a/src/satellite/jest.config.js b/src/satellite/jest.config.js
@@ -0,0 +1,19 @@
+// Define the env variables our code expects
+process.env = Object.assign(process.env, {
+  JWT_SECRET: 'your-super-secret-jwt-token-with-at-least-32-characters-long',
+  JWT_AUDIENCE: 'http://localhost',
+  JWT_ISSUER: 'http://localhost',
+  JWT_EXPIRES_IN: '1h',
+  MOCK_REDIS: '1',
+  MOCK_ELASTIC: '1',
+});
+
+module.exports = {
+  testEnvironment: 'node',
+  bail: 1,
+  verbose: true,
+  testPathIgnorePatterns: ['/node_modules/'],
+  coverageDirectory: '<rootDir>/coverage',
+  collectCoverageFrom: ['<rootDir>/src/**/*.js'],
+  testTimeout: 8000,
+};