Skip to content

Architecture

Jump to bottom
David Humphrey edited this page Jan 23, 2023 · 3 revisions

The following are high-level overviews of the architecture of Starchart and its various systems.

DNS

Create/Update Domain for User

An authenticated user can create or update a domain name via the Remix web app. This is an asynchronous operation that can take more than a minute to complete, which is too long to block the HTTP response. Instead we use the Asynchronous Request-Reply Pattern with a queue and background worker using BullMQ and Redis.

Consider the case that a user wants to create or update an A Record for foo.{username}.starchart.com to point to 192.25.31.24:

sequenceDiagram
    Front-End->>Back-End: 1. Create/Update foo.*
    Back-End->>Queue: 2. Add to DB + Job Queue
    Back-End->>Front-End: 3. HTTP 202 + Location URL
    Queue-->>Worker: 6. Worker gets job
    Worker->>Route53: 7. AWS SDK call UPSERT
    Route53->>Worker: 8. Returns Change ID
    Worker->>Route53: 9. AWS SDK calls GetChange w/ Change ID
    Route53->>Worker: 10. PENDING | INSYNC
    Note right of Route53: Repeat until INSYNC (~60s)
    Worker-->>Back-End: 11. Update DB record with READY | ERROR
    Front-End->>Back-End: 4. GET Location URL
    Back-End->>Front-End: 5. 200 with PENDING | READY | ERROR
    Note left of Front-End: Poll until !PENDING
Loading

Front-End and Back-End

  1. The Remix front-end web app presents the user with an HTML form. The user enters the domain information and clicks Create or Update (the steps are similar). A POST request is sent to the back-end.

  2. The back-end Remix Action receives the POST request with form data. It validates it, returning a 400 if there are errors; otherwise, it adds/updates a record in the DB for the domain and adds a new job to the *queue for processing the request.

  3. An HTTP 202 (Accepted) is immediately returned to the front-end with a Location header indicating the URL to access the domain once created/updated. For example: /domains/foo

  4. The front-end begins polling the URL received in the Location header over and over. The data returned will include information about the state of the domain, which could be one of PENDING, READY, or ERROR

{
  "name": "foo",
  "state": "PENDING",
  ...
}
  1. The front-end will update the UI in response to the state, and decide whether or not to continue polling for more updates.

Worker Queue

Meanwhile, an asynchronous operation happens separate to the front-end and back-end interaction.

  1. A background worker process picks up the job from the queue and begins the process of updating AWS Route53

  2. A ChangeResourceRecordSetsCommand is created using the UPSERT change operation and domain info, then sent to AWS.

  3. AWS sends back ChangeInfo, which can be used to track the status of the change.

  4. A GetChangeCommand is created using the change info obtained in 8. and sent to AWS.

  5. AWS sends back the status of the change, which will be one of PENDING while still happening, or INSYNC when complete. This process (9. and 10.) is repeated as many times as necessary until INSYNC is received. NOTE: this can take up to a minute.

  6. Once the DNS change is INSYNC, or if there was an error above, the record in the DB for the domain is updated to indicate that the state is now READY (or in ERROR if there was an error). Doing so will mean that the next time the front-end polls the back-end for the status of the domain (4. and 5.) the new status will be returned.

List Domains for User

An authenticated user can view a list of their existing domain names via the Remix web app.

sequenceDiagram
    Front-End->>Back-End: 1. GET /domains
    Back-End->>Database: 2. Prisma queries MySQL
    Database->>Back-End: 3. MySQL results to Prisma
    Back-End->>Front-End: 4. HTTP 200 with JSON
Loading

  1. The Remix front-end web app requests the authenticated user's domain via a GET request to the back-end.

  2. The back-end Remix Loader receives the GET request. Prisma is used to run a findMany() query against the domains model and database table.

  3. Domains data is stored for the user in MySQL. The domain records include the username, so it's easy to query for all records matching a particular user. The data returned to the back-end from MySQL is made available via a Prisma Model, so that it can be used in TypeScript.

  4. The back-end Remix Loader returns the json necessary to populate the front-end web components with the user's domains. Each of the domains is decorated such that they can be modifed, deleted, etc.

Delete Domain for User

An authenticated user can delete an existing domain name via the Remix web app. Similar to create/update, this is an asynchronous operation

Consider the case that a user wants to delete an existing A Record for foo.{username}.starchart.com that points to 192.25.31.24:

sequenceDiagram
    Front-End->>Back-End: 1. Delete foo.*
    Back-End->>Queue: 2. Update DB + Job Queue
    Back-End->>Front-End: 3. HTTP 202 + Location URL
    Queue-->>Worker: 6. Worker gets job
    Worker->>Route53: 7. AWS SDK call DELETE
    Route53->>Worker: 8. Returns Change ID
    Worker->>Route53: 9. AWS SDK calls GetChange w/ Change ID
    Route53->>Worker: 10. PENDING | INSYNC
    Note right of Route53: Repeat until INSYNC (~60s)
    Worker-->>Back-End: 11. Delete DB record or update to ERROR
    Front-End->>Back-End: 4. GET Location URL
    Back-End->>Front-End: 5. 200 with PENDING | READY | ERROR or 404
    Note left of Front-End: Poll until !PENDING
Loading

Front-End and Back-End

  1. The Remix front-end web app presents the user with their existing domains. The user clicks Delete. A POST request is sent to the back-end.

  2. The back-end Remix Action receives the POST request. It validates it, returning a 400 if there are errors; otherwise, it updates the record in the DB for the domain and adds a new job to the *queue for processing the request.

  3. An HTTP 202 (Accepted) is immediately returned to the front-end with a Location header indicating the URL to access the domain. For example: /domains/foo

  4. The front-end begins polling the URL received in the Location header over and over. The data returned will include information about the state of the domain, which could be one of PENDING, READY, or ERROR. When the domain is removed, a 404 will be returned instead of a 200.

{
  "name": "foo",
  "state": "PENDING",
  ...
}
  1. The front-end will update the UI in response to the state, and decide whether or not to continue polling for more updates.

Worker Queue

Meanwhile, an asynchronous operation happens separate to the front-end and back-end interaction.

  1. A background worker process picks up the job from the queue and begins the process of updating AWS Route53

  2. A ChangeResourceRecordSetsCommand is created using the DELETE change operation and domain info, then sent to AWS.

  3. AWS sends back ChangeInfo, which can be used to track the status of the change.

  4. A GetChangeCommand is created using the change info obtained in 8. and sent to AWS.

  5. AWS sends back the status of the change, which will be one of PENDING while still happening, or INSYNC when complete. This process (9. and 10.) is repeated as many times as necessary until INSYNC is received. NOTE: this can take up to a minute.

  6. Once the DNS change is INSYNC, or if there was an error above, the record in the DB for the domain is either deleted (Route53 has deleted it) or updated (there was an error) to indicate that the state is now ERROR. Doing so will mean that the next time the front-end polls the back-end for the status of the domain (4. and 5.) the new status will be returned.

Home

Winter 2023

Telescope

Planning

Technologies

Developer documentation

Don't like this sidebar? Let me know what to improve on!

Clone this wiki locally