Skip to content

Nestjs project handling big size files with node stream and mongodb cursor. Exposing data through elasticsearch

License

Notifications You must be signed in to change notification settings

tupizz/nestjs-mongo-elasticsearch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Nest Logo

A progressive Node.js framework for building efficient and scalable server-side applications, heavily inspired by Angular.

NPM Version Package License Travis

📚 Description

This boilerplate is made to quickly prototype backend applications. It comes with database, logging, security, and authentication features out of the box.


🛠️ Prerequisites

Non Docker

  • Please make sure to either have MongoDB Community installed locally or a subscription to Mongo on the cloud by configuration a cluster in atlas.

Docker 🐳

  • Please make sure to have docker desktop setup on any preferred operating system to quickly compose the required dependencies. Then follow the docker procedure outlined below.

Note: Docker Desktop comes free on both Mac and Windows, but it only works with Windows 10 Pro. A workaround is to get Docker Toolbox which will bypass the Windows 10 Pro prerequisite by executing in a VM.


🚀 Deployment

Manual Deployment without Docker

  • Create a .env file using the cp .env.example .env command and replace the existing env variables with personal settings (MongoDB URL either srv or localhost)

    • Modify the connection string by modifying the following line.
  • Download dependencies using npm i or yarn

  • Start the app in pre-production mode using npm run start or npm run start:dev for development (the app will be exposed on the port 9000; not to conflict with React, Angular, or Vue)

Deploying with Docker 🐳

  • Execute the following command in-app directory:
# creates and loads the docker container with required configuration
$ docker-compose up -d

🔒 Environment Configuration

By default, the application comes with a config module that can read in every environment variable from the .env file.

APP_ENV - the application environment to execute as, either in development or production. Determines the type of logging options to utilize. Options: dev or prod.

APP_URL - the base URL for the application. Made mainly to showcase the power of ConfigService and can be removed as it doesn't serve any other purpose

WEBTOKEN_SECRET_KEY - the secret key to encrypt/decrypt web tokens with. Make sure to generate a random alphanumeric string for this.

WEBTOKEN_EXPIRATION_TIME - the time in seconds indicating when the web token will expire; by default, it's 2400 seconds which is 40 mins.

DB_URL - the URL to the MongoDB collection


🏗 Choosing a Web Framework

This boilerplate comes with Fastify out of the box as it offers performance benefits over Express. But this can be changed to use Express framework instead of Fastify.

For interchangeability:

  • Replace the following lines of code in the main.ts file with the ones detailed below.

Fastify:

// for fastify:
import {
  FastifyAdapter,
  NestFastifyApplication,
} from '@nestjs/platform-fastify';
import headers from 'fastify-helmet';
import fastifyRateLimiter from 'fastify-rate-limit';
const app = await NestFactory.create<NestFastifyApplication>(
  AppModule,
  new FastifyAdapter({ logger: console }),
);
app.register(headers);
app.register(fastifyRateLimiter, {
  max: 100,
  timeWindow: '1 minute',
});

Express:

// for express:
import headers from 'helmet';
import rateLimiter from 'express-rate-limit';
const app = await NestFactory.create(AppModule, {
  logger: console,
});
app.use(headers());
app.use(
  rateLimiter({
    windowMs: 60, // 1 minutes
    max: 100, // limit each IP to 100 requests per windowMs
  }),
);

Note: The boilerplate comes with production dependencies for both Express and Fastify to support moving between two. But this is going to leave it bloated especially when only one web framework is used at a time. Thus, it is recommended that when deploying to production, unused dependencies are purged.

If you choose to use Fastify, this command will purge all of the Express dependencies:

# removing Express dependencies
$ npm rm @nestjs/platform-express express-rate-limit helmet swagger-ui-express @types/express --save

If you choose to use Express, this command will purge all of the Fastify dependencies:

# removing Fastify dependencies
$ npm rm @nestjs/platform-fastify fastify-helmet fastify-rate-limit fastify-swagger --save

✅ Testing

Docker 🐳

# unit tests
$ docker exec -it nest yarn test

# e2e tests
$ docker exec -it nest yarn test:e2e

# test coverage
$ docker exec -it nest yarn test:cov

Non-Docker

# unit tests
$ npm run test

# e2e tests
$ npm run test:e2e

# test coverage
$ npm run test:cov

💡 TypeDocs

The documentation for this boilerplate can be found on Github pages.

The docs can be generated on-demand, simply, by typing npm run typedocs. This will produce a docs folder with the required front-end files and start hosting on localhost.

# generate docs for code
$ npm run typedocs

📝 Open API

Out of the box, the web app comes with Swagger; an open api specification, that is used to describe RESTful APIs. Nest provides a dedicated module to work with it.

The configuration for Swagger can be found at this location.


✨ Mongoose

Mongoose provides a straight-forward, schema-based solution to model your application data. It includes built-in type casting, validation, query building, business logic hooks and more, out of the box. Please view the documentation for further details.

The configuration for Mongoose can be found in the app module.


🔊 Logs

This boilerplate comes with an integrated Winston module for logging, the configurations for Winston can be found in the app module.


👥 Support

Nest has been able to grown because of sponsors and support from backers. If you'd like to join them, please read more here.


License

Nest is MIT licensed.

Author

About

Nestjs project handling big size files with node stream and mongodb cursor. Exposing data through elasticsearch

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages