Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud Backend test solution #26

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
a7a9456
chore: adding lint tools
himabindugit Aug 12, 2022
9d6485d
chore: add prettier config file
himabindugit Aug 12, 2022
c58e2ea
chore: add husky with linting pre-commit pre-push commit-msg hooks
himabindugit Aug 12, 2022
2b7c7e2
chore: add deploy supported scripts
himabindugit Aug 12, 2022
ced73c9
chore: added lambda handler
himabindugit Aug 12, 2022
72687e7
chore: package files for dependencies
himabindugit Aug 12, 2022
c6e5e5f
chore: added sample.env for reference
himabindugit Aug 12, 2022
8533093
chore: add documentation folder with screenshots
himabindugit Aug 12, 2022
c3a7fba
chore: typecript complier config files
himabindugit Aug 12, 2022
aa3ff3b
chore: graphql schema file
himabindugit Aug 12, 2022
9781b06
chore: gitignore file
himabindugit Aug 12, 2022
3b85cbf
chore: mocha configuration file
himabindugit Aug 12, 2022
cee1771
chore: implementation around env files and global configs
himabindugit Aug 12, 2022
23d4a92
chore: tests utilities
himabindugit Aug 12, 2022
f2f5d74
chore: unit tests
himabindugit Aug 12, 2022
2fd5f1b
chore: integration tests
himabindugit Aug 12, 2022
cfc64d1
chore: playing around express context and tracing
himabindugit Aug 12, 2022
6579e80
chore: file to calculate coordinates using node-geocoder
himabindugit Aug 12, 2022
0072c8c
chore: starting apollo server file
himabindugit Aug 12, 2022
6acfa44
chore: logging plugin for logs for more info
himabindugit Aug 12, 2022
dec91e8
chore: coordinate resolvers
himabindugit Aug 12, 2022
081e625
chore: starting apollo server file
himabindugit Aug 12, 2022
ed291c4
chore: typedefs
himabindugit Aug 12, 2022
1c9bb93
chore: logging,types, handler tests
himabindugit Aug 13, 2022
8241fe5
chore: review comments> documentation, more edge cases andtests
himabindugit Aug 16, 2022
99b99da
chore: more documentation changes
himabindugit Aug 16, 2022
a610e3d
chore: more tests, used hive plugin, better error handling, more tests
himabindugit Aug 16, 2022
1f0315a
chore: better error handler, tests fixes
himabindugit Aug 16, 2022
ce31a0f
chore: consuming Google Geocode API once using data sources and consu…
himabindugit Aug 16, 2022
babec8d
chore: shuffling prd and dev dependencies
himabindugit Aug 16, 2022
be911c9
chore: tests fix, better jwt unauthorized error handling, unskip tests
himabindugit Aug 17, 2022
ce0d627
chore: cleanup and added more tests
himabindugit Aug 17, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
node_modules/*

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @himabindugit! I'm looking forward to hearing your thoughts following my review :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @rleonr-sf thanks for the detailed review, i have tried my best to answer your questions, i can take an another look tomorrow just to make i haven't missed anything.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @himabindugit, I appreciate the time invested in this submission. We'll get in touch with you with an update soon

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @rleonr-sf Just FYI - I have pushed more changes. I have just included screenshots (success and error cases) using my GraphQL API, am happy to upload interactive demo with using GraphQL API.

25 changes: 25 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
module.exports = {
parser: '@typescript-eslint/parser',
plugins: ['@typescript-eslint', 'prettier'],
extends: [
'plugin:@typescript-eslint/eslint-recommended',
'plugin:@typescript-eslint/recommended',
'prettier',
],
parserOptions: {
ecmaVersion: 'latest',
sourceType: 'module',
},
rules: {
'require-await': 2,
'@typescript-eslint/camelcase': 0,
'prettier/prettier': [
'error',
{
singleQuote: true,
trailingComma: 'es5',
arrowParens: 'always',
},
],
},
};
20 changes: 20 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
node_modules
.env
dist/

coverage/*
.nyc_output/*

# CDK asset staging directory
.cdk.staging
cdk.out
cdk.context.json


# Parcel build directories
.cache
.build
nodejs
nodejs.zip
.DS_Store
conf/.env
4 changes: 4 additions & 0 deletions .husky/.husky/commit-msg
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

npm run commit-msg
4 changes: 4 additions & 0 deletions .husky/.husky/pre-commit
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

npm run pre-commit
4 changes: 4 additions & 0 deletions .husky/.husky/pre-push
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

npm run pre-push
13 changes: 13 additions & 0 deletions .mocharc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"diff": true,
"extension": ["ts"],
"opts": false,
"sort": true,
"color": true,
"slow": 75,
"timeout": 99999,
"ui": "bdd",
"spec": "src/test/**/*.test.ts",
"require": "ts-node/register",
"reporter": "spec"
}
3 changes: 3 additions & 0 deletions .prettierrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"singleQuote": true
}
6 changes: 6 additions & 0 deletions .sample.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
SERVICE_PORT=
API_KEY=<YOUR_API_KEY>
JWT_TOKENS_AUTHORITY=
DEPLOY_REGION=
ENV_SHORT=
STACK_NAME=
37 changes: 37 additions & 0 deletions bin/graphql-cdk.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/usr/bin/env node

import * as cdk from '@aws-cdk/core';

import { GraphqlLambdaStack } from '../deploy/graphql-stack';
import conf from '../src/conf';
import { Environment } from '../src/conf/config';

const account = conf.deployAccount;
const region = conf.deployRegion;
const hostedZone = process.env.DEPLOY_ZONE ?? 'us-east-1';

const authority = conf.jwtTokens.authority;
const environment = conf.env as Environment;
const serviceName = 'graphql';
const apiKey = conf.apiKey as string;

class GraphQLCDKApp extends cdk.App {
constructor() {
super();

new GraphqlLambdaStack(this, `${conf.stackName}`, {
environment,
env: {
account,
region,
},
hostedZone: hostedZone,
authority,
serviceName,
apiKey,
});
}
}

const app = new GraphQLCDKApp();
app.synth();
9 changes: 9 additions & 0 deletions cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"app": "npx ts-node bin/graphql-cdk.ts",
"requireApproval": "never",
"context": {
"@aws-cdk/core:enableStackNameDuplicates": "true",
"aws-cdk:enableDiffNoFail": "true"
}
}

41 changes: 41 additions & 0 deletions deploy/deploy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/bin/bash

set -e

# Installing only prod dependencies because of 250MB limitation
# https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
echo "## Installing prd dependencies..."
npm install --only=prod --loglevel=warn
npm run build -- --project tsconfig.prod.json

AWS_CREDS_FILE=~/.aws/credentials
if [[ ! -e ~/.aws/credentials ]]; then
mkdir ~/.aws
touch $AWS_CREDS_FILE
chmod 600 $AWS_CREDS_FILE
fi

grep -qF -- "[graphql-deploy]" ~/.aws/credentials || cat << EndOfFile >> $AWS_CREDS_FILE
[graphql-deploy]
aws_access_key_id=${AWS_ACCESS_KEY_ID}
aws_secret_access_key=${AWS_SECRET_ACCESS_KEY}
EndOfFile

echo "## Check for existing nodejs directory..."
if [[ ! -e nodejs ]]; then
echo "## Deleting nodejs zip"
rm -rf nodejs.zip
echo "## Creating nodejs directory..."
mkdir nodejs
fi
cd nodejs
cp -r ../node_modules node_modules
cp ../package.json .
cp ../schema.graphql ../dist
cd .. || exit
npm run zip:nodejs

npm run cdk -- deploy --profile graphql-deploy -o cdk.out

rm -r nodejs
rm -r nodejs.zip
77 changes: 77 additions & 0 deletions deploy/graphql-stack.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as apiGateway from '@aws-cdk/aws-apigateway';
import { Construct } from '@aws-cdk/core';
import { Duration } from '@aws-cdk/core';
import * as gql from './graphql';
import _ from 'lodash';
import * as ec2 from '@aws-cdk/aws-ec2';

const createTagger =
(tags: { [key: string]: string }) => (taggable: Construct) =>
Object.keys(tags).map((tag) => cdk.Tags.of(taggable).add(tag, tags[tag]));

export class GraphqlLambdaStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props: gql.StackProps) {
super(scope, id, props);
const tagIt = createTagger({
env: props.environment,
'provisioned-by': 'cdk',
service: 'graphql',
Name: `${props.environment}-${props.env?.region}-${props.serviceName}`,
});
// The code that defines your stack goes here
const layer = new lambda.LayerVersion(this, 'NpmModulesLayer', {
code: new lambda.AssetCode('nodejs.zip'),
compatibleRuntimes: [lambda.Runtime.NODEJS_12_X],
description: 'A layer to package modules',
});

// If you want, you can configure own VPC, subnets, SGs when creating
// Lambda function.

// const vpc = ec2.Vpc.fromLookup(this, 'Vpc', {
// vpcId: props.vpcId,
// });
// const securityGroup = ec2.SecurityGroup.fromSecurityGroupId(
// this,
// 'GraphQLSG',
// 'sg-0e8485c6cad5d3be7',
// {
// mutable: false,
// }
// );
const graphqlLambda = new lambda.Function(this, 'MessageHandler', {
functionName: 'GraphQLLambda',
description: 'Process GraphQL requests',
// Where our lambda function is located
code: new lambda.AssetCode('dist'),
// What should be executed once the lambda is invoked
// - in that case, the `handler` function exported by `handler.ts`
handler: 'handler.handler',
runtime: lambda.Runtime.NODEJS_12_X,
timeout: Duration.seconds(300),
environment: {
AUTHORITY: props.authority,
API_KEY: props.apiKey,
},
layers: [layer],
// vpc: vpc,
// vpcSubnets: {
// subnets: _.filter(vpc.privateSubnets, (subnet) => {
// return props.subnetIds.includes(subnet.subnetId);
// }),
// },
// securityGroups: [securityGroup]
});
tagIt(graphqlLambda);
const apiGatewayLambdaRestApi = new apiGateway.LambdaRestApi(
this,
'graphqlEndpoint',
{
handler: graphqlLambda,
}
);
tagIt(apiGatewayLambdaRestApi);
}
}
17 changes: 17 additions & 0 deletions deploy/graphql.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import * as cdk from '@aws-cdk/core';

export enum Environment {
Production = 'prd',
Staging = 'stg',
QA = 'qa',
Development = 'dev',
}

export interface StackProps extends cdk.StackProps {
apiKey: string;
environment: Environment;
imageTag?: string;
hostedZone: string;
authority: string;
serviceName: string;
}
Binary file added documentation/1_Architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
66 changes: 66 additions & 0 deletions documentation/2_HowToDeploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
## Deployment guide

### Welcome to CDK TypeScript project

You should explore the contents of this project. It demonstrates a CDK app with an instance of a stack (`DevGraphQLStack`)
which contains a Lambda Function and API Gateway.

The `cdk.json` file tells the CDK Toolkit how to execute your app.<p>&nbsp;</p>

### What prerequisites do I need to deploy the service?

1. Install node and awscli, follow this [link](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) to confugure aws.

2. Install cdk, `npm install aws-cdk`
This same depedency is mentioned in `package.json` as well, but installing ahead for cdk bootstrap purposes

3. Run `cdk bootstrap aws://<YOUR_AWS_ACCOUNT>/<YOUR_AWS_REGION>` which will set up a deployment environment on AWS you can use with your default AWS credentials. For example, `cdk bootstrap aws://1234567/us-east-1`
rleonr-sf marked this conversation as resolved.
Show resolved Hide resolved

Note: Keep in mind not to install extra packages other than prd dependencies in `package.json`, because of lambda 250MB limitation, for more [details](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html)


### <u>How to deploy CDK stack</u>

deploy.sh script to deploy cdk stacks to Dev, QA, STG, PRD environments.

`deploy.sh` script is to deploy CDK stacks to AWS
- pass these as command line arguments for `deploy.sh`
AWS_ACCESS_KEY_ID - <YOUR_AWS_ACCESS_KEY_ID>
AWS_SECRET_ACCESS_KEY - <YOUR_AWS_SECRET_ACCESS_KEY>

eg:, from the root dir:
```bash
AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID> AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY> bash deploy/deploy.sh
```
Grab GraphQL endpoint from cdk output.

Example output
```bash
Outputs:
DevGraphQLStack.graphqlEndpoint711CBBBD = https://6holtoh5t4.execute-api.us-east-1.amazonaws.com/prod/
```
<p>&nbsp;</p>

### Troubleshooting

1.verify cloudformation stack with right resources like lambda, api gaeway deployed in AWS
2. Clod watch logs are helpful to debug any issues

### Useful commands

- `npm run build` compile typescript to js
- `npm run watch` watch for changes and compile
- `npm run test` perform the jest unit tests
rleonr-sf marked this conversation as resolved.
Show resolved Hide resolved
- `cdk deploy` deploy this stack to your default AWS account/region
- `cdk diff` compare deployed stack with current state
- `cdk synth` emits the synthesized CloudFormation template

NOTE: To deploy, you will need to run `cdk bootstrap` which will set up a deployment environment on AWS you can use with your default AWS credentials.

- cdk deploy deploy this stack to your default AWS account/region, in graphql case, stack is going to be Lambda Function, Lambda Layer, IAM Role, Policy, API Gateway etc..
- cdk diff compare deployed stack with current state
- cdk synth emits the synthesized CloudFormation template to ./cdk.out

[Layers](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html#layers): using lambda layers to provision code to Lambda function which are node modules.
rleonr-sf marked this conversation as resolved.
Show resolved Hide resolved
Check [here](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html#bundling-asset-code) for more details.<p>&nbsp;</p>

10 changes: 10 additions & 0 deletions documentation/3_HowToRun.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
### How to run
Once Stack is deployed, access API Gateway in Postman. First update Lambda configuration environment variables with the right API key. By providing right jwt token, you can access GraphQL API by appending `/graphql` at the end.

#### How to generate API_KEY in Google Cloud Console?

> Follow this [link](https://developers.google.com/maps/documentation/geocoding/get-api-key) to create API key and restrict to use it for Geocoding API

### GraphQL Endpoint

For example, you can hit `https://6holtoh5t4.execute-api.us-east-1.amazonaws.com/prod/graphql` and make queries with the right JWT token.
rleonr-sf marked this conversation as resolved.
Show resolved Hide resolved
Loading