Skip to content
/ web-crawler-worker-node-js Public

Worker part of the Web Crawler application. The Web Crawler app takes an input from the user such as a link maximum number of pages and depth. At the end it shows in real time a tree of all the links and pages that the crawler found in the provided URL.

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

almoggutin/web-crawler-worker-node-js

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
kubernetes
kubernetes
 
 
src
src
 
 
.dockerignore
.dockerignore
 
 
.eslintrc.json
.eslintrc.json
 
 
.gitignore
.gitignore
 
 
.prettierignore
.prettierignore
 
 
.prettierrc
.prettierrc
 
 
Dockerfile
Dockerfile
 
 
README.md
README.md
 
 
package-lock.json
package-lock.json
 
 
package.json
package.json
 
 

Repository files navigation

Web Crawler Worker Side

Worker side of the Web Crawler Application

< Technologies /> 💻

Frontend Technologies:

  1. HTML5
  2. CSS3
  3. Javascript
  4. Handlebars.js

Backend Technologies:

  1. Node.js
  2. Express
  3. Cors
  4. AWS
  5. Redis
  6. Jest
  7. Node html parser

Scaling Services:

  1. Dcoker
  2. Kubernetes

< Repositories />

  1. Web Crawler Full App
  2. Client Side
  3. Server Side

About

Worker part of the Web Crawler application. The Web Crawler app takes an input from the user such as a link maximum number of pages and depth. At the end it shows in real time a tree of all the links and pages that the crawler found in the provided URL.

Topics

nodejs javascript docker kubernetes redis aws parser express jest rest-api web-crawler web-worker web-scraper backend-api

Resources

Readme
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Languages