Skip to content

Commit

Permalink
Change more docs
Browse files Browse the repository at this point in the history
  • Loading branch information
TueeNguyen committed Apr 7, 2022
1 parent 89bfae1 commit 32b2ee3
Show file tree
Hide file tree
Showing 6 changed files with 7 additions and 31 deletions.
8 changes: 2 additions & 6 deletions src/api/parser/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Parser Service: (To be updated when Parser service is dockerized and live)
# Parser Service:

The Parser service parses posts from user's feeds to populate Redis
Read more on Parser at `src/web/docusaurus/docs/api-services/parser.md`

## Install

Expand All @@ -23,7 +23,3 @@ pnpm dev
```

By default the server is running on <http://localhost:10000/>.

### Examples

## Docker
2 changes: 1 addition & 1 deletion src/web/docusaurus/docs/api-services/parser.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
### Parser Service

Telescope's back-end began as a single, monolithic node.js app. During the 2.0 release, much of the back-end was split into separate microservices. Parts of the legacy back-end code were still in use, see `src/backend/*` which were removed in 3.0 release, telescope now consists of only microservices/workspaces.
Telescope's back-end began as a single, monolithic node.js app. During the 2.0 release, much of the back-end was split into separate microservices. Parts of the legacy back-end code were still in use but removed in 3.0 release, Telescope now consists of only microservices/workspaces.

The current system uses the parser service in order to run the feed parser and feed queue, see [`src/api/parser/data/feed.js`](https://github.com/Seneca-CDOT/telescope/blob/master/src/api/parser/src/data/feed.js). The processed feeds and posts are then stored in `Redis` (cache) and `Elasticsearch` (indexing) databases, and various microservices use these in order to get their data.

Expand Down
21 changes: 2 additions & 19 deletions src/web/docusaurus/docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,32 +18,15 @@ It shows RSS/Atom blog feeds (on the left) being downloaded (1) and then queued

The following gives an overview of the current (i.e. 2.4.0) design of Telescope. There are a number of major systems that make up the app:

- legacy monolithic back-end (1.0)
- back-end microservices (2.0)
- front-end web app
- SAML2-based authentication and authorization
- autodeployment server
- test, build, and other automation tooling

### Legacy Monolithic Back-end (1.0)

Telescope's back-end began as a single, monolithic node.js app. During the 2.0 release, much of the back-end was split into separate microservices (see below). However, parts of the legacy back-end code are still in use, see `src/backend/*`.

The current system uses the legacy backend in order to run the feed parser and feed queue, see `src/backend/feed/*`. The processed feeds and posts are then stored in Redis (cache) and Elasticsearch (indexing) databases, and various microservices use these in order to get their data.

Telescope's data model is built on Feeds and Posts. A feed represents an RSS/Atom feed, and includes metadata about a particular blog (e.g., URL, author, etc) as well as URLs to individual Posts. A Post includes metadata about a particular blog post (e.g., URL, date created, date updated, etc).

The legacy back-end is started using `pnpm start` in the root of the Telescope monorepo, and it (currently) must be run alongside the microservices. When it runs, the logs show information about feeds being parsed in real-time, which continues forever.

The parser downloads the [CDOT Feed List](https://wiki.cdot.senecacollege.ca/wiki/Planet_CDOT_Feed_List#Feeds), parses it, creates `Feed` objects and puts them into a queue managed by [Bull](https://github.com/OptimalBits/bull) and backed by Redis. These are then processed in `src/backend/feed/processor.js` in order to download the individual Posts, which are also cached in Redis.

There is code duplication between the current back-end and the Parser microservice (see `src/api/parser`), and anyone changing the back-end will also need to update the Parser service at the same time (for now). One of the 3.0 goals is to [remove the back-end and move all of this logic to the Parser service](https://github.com/Seneca-CDOT/telescope/issues?q=is%3Aissue+is%3Aopen+parser+service).

In production, the legacy back-end is deployed as a container named `telescope` (see `docker/production.yml`), and its Dockerfile lives in the root at `./Dockerfile`.

### Back-end Microservices (2.0)

The legacy back-end has been split into a series of microservices. Each microservice follows a similar architectural pattern and has a single focuses. The current microservices include:
Telescope consists of a series of microservices. Each microservice follows a similar architectural pattern and has a single focuses. The current microservices include:

- SSO Auth Service (`src/api/sso`) - authentication and authorization with Seneca's SSO service
- Feed-Discovery Service (`src/api/feed-discovery`) - blog feed discovery and validation
Expand All @@ -52,7 +35,7 @@ The legacy back-end has been split into a series of microservices. Each microser
- Posts Service (`src/api/posts`) - API for accessing Post and Feed data in Redis (probably not well named at this point)
- Search Service (`src/api/search`) - API for doing searches against Elasticsearch
- Status Service (`src/api/status`) - API for accessing Telescope status information, as well as providing the Dashboards
- Parser Service (`src/api/parser`) - feed and post parsing. Currently disabled, see <https://github.com/Seneca-CDOT/telescope/issues/2111>
- Parser Service (`src/api/parser`) - feed and post parsing.

All microservices are built on a common foundation, the [Satellite module](https://github.com/Seneca-CDOT/telescope/tree/master/src/satellite). Satellite provides a common set of features for building Express-based microservices, with proper logging, health checks, headers, authorization middleware, as well as connections to Redis and Elasticsearch. It saves us having to manage the same set of dependencies a dozen times, and repeat the same boilerplate code.

Expand Down
1 change: 0 additions & 1 deletion src/web/docusaurus/docs/contributing/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ Now that we know how to launch the server. We will look at different launching o

![VS Launch Options Screenshot](../../static/img/VS-Launch-Options.png)

1. Launch Telescope -> Launches index.js and tries to run the backend.
1. Launch Auto Deployment -> Launches autodeployment found in tools.
1. Launch All Tests -> Runs all the tests in the tests folder.
1. Launch Opened Test File -> Will run a test that is currently opened in a VS code tab.
Expand Down
2 changes: 0 additions & 2 deletions src/web/docusaurus/docs/getting-started/environment-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,8 +260,6 @@ This is the default setting, you do not need to copy or modify any `env` file.

```bash
pnpm services:start

pnpm start
```

Then visit `localhost:8000` in a web browser to see Telescope running locally. `localhost:3000/posts` will show you the list of posts in JSON
Expand Down
4 changes: 2 additions & 2 deletions src/web/docusaurus/docs/tools-and-technologies/pino.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ sidebar_position: 5

# Logging Support Using Pino

This project uses [Pino](http://getpino.io/#/) to provide support for logging in Production as well as development environments. The [logger.js](https://github.com/Seneca-CDOT/telescope/blob/master/src/backend/utils/logger.js) module exports a logger instance that can be used in other modules to implement logging for important events.
This project uses [Pino](http://getpino.io/#/) to provide support for logging in Production as well as development environments.`Satellite` exports a `logger` instance (https://github.com/Seneca-CDOT/telescope/blob/master/src/satellite/src/logger.js) that can be used in modules to implement logging for important events.

## How to use the logger

```javascript
const { logger } = require('../src/backend/utils/logger');
const { logger } = require('@senecacdot/satellite');

logger.info('Important information...');
logger.trace('Information About Trace');
Expand Down

0 comments on commit 32b2ee3

Please sign in to comment.