HTTP REST services for infra/containers testing: echo, leaks, CPU-intensive ops, backend usage, etc.
See full Swagger definition and sample CURL commands at http://yourhost:8080/api/v2/swagger.yaml See Swagger-UI at http://yourhost:8080/docs/ (mind the final '/').
Return back message.
Sample requests (both GET and POST)
curl -i -H 'Accept: application/json' http://192.168.59.103:8080/api/v2/echo/hello
curl -X POST -H "Accept: Application/json" -H "Content-Type: application/json" http://localhost:8080/api/v2/echo -d '{"message":"hello"}'
Return back message after {delayms} milliseconds
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/echo/hello/1000
Display REST server environment.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/env/vars
Return the server ENV value for variable {name}.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/env/vars/HOME
Return the server hostname.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/env/hostname
Return the REST server process ID (PID).
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/env/pid
Health check service.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/health
Return a 'up' message {percentage}% time and an HTTP error 503 otherwise. The {percentage} is a float from 0 (0%) to 1 (100%).
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/health/0.5
Cause the process to exit, simulating a crash.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/kill
Leak {size} bytes of memory.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/leak/1024
Free all memory leaked by calls to /api/v2/leak/{size}.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/leak/free
Sum {n} to {id} counter in and return the new value. The data is stored in the instance memory by default (statefull) and can be stored to Cassandra or Redis to emulate a stateless 12-factor behavior.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/calc/sum/counter-0/1
Compute the n-th term of fibonacci", notes = "Compute the n-th term of fibonacci which is CPU intensive, expecially if {n} > 50.
Sample request:
curl -i -H 'Accept: application/json' http://localhost:8080/api/v2/calc/fibo-nth/42
The application runs either as a NodeJS application or as a Docker container.
Get from GIT and run:
git clone https://github.com/legdba/servicebox-nodejs.git && cd servicebox-nodejs && npm install && npm test && npm start
Swagger API available at http://localhost:8080/swagger; use it to explore exposed services and how to use them.
Display help for more CLI details:
npm start -- --help
Latest version is always available at Quai.io or Docker Hub and can be used as a docker application:
docker run -ti -p :8080:8080 --rm=true quay.io/legdba/servicebox-nodejs:latest
Help available the usual way:
docker run -ti -p :8080:8080 --rm=true quay.io/legdba/servicebox-nodejs:latest --help
Swagger API documentation available on http://your_container_ip:8080/swagger
Note that in the docker registry each image is tagged with the branch name. Initially the image tag contained the GIT commit ID, but this is a non-standard DockerHub/Quay.io feature and was available only through customer builds, which are a pain to manage and maintain.
Servicebox can be deployed an AWS Lambda function by setting the AWS credential in your environment (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY):
export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=xxx
And then running Serverless Framework:
npm install
./node_modules/.bin/serverless deploy
This will deploy the application as an AWS lambda function bound to an API Gateway endpoint as per the serverless.yml
configuration. Serverless will display the HTTP endpoints once deployed. Those are servicebox REST endpoints; example:
curl -v https://xxxxxxxxxx.execute-api.us-west-2.amazonaws.com/dev/api/v2/echo/hello
NOTE: the default configuration exposes a public REST service that anybody could call without control.
The following environment variable can be set to configure servicebox while it runs as a lambda function:
LOG_LEVEL="debug|info|warn|error"
BACKEND_TYPE="memory|cassandra|redis-cluster|redis-sentinel|dynamodb"
BACKEND_OPTIONS='same value as --beopts, single-quote enclosed'
Note that as of Feb-2017 AWS Lambda does not allow commas (',') in variable values. As a workaround servicebox replaces any =comma=
into the BACKEND_OPTIONS value with a comma.
- AWS Lambda runs on NodeJS 4.3.2 which lack some welcome features of more modern NodeJS versions :(
- Calling
serverless deploy
with nodejs 4 cause the deployed function to be broken for some reason. Deployemnt is successfully tested with nodejs 7.5.0 only. - There is no CI/CD integration test for lambda servicebox-nodejs yet.
Serverless Framework can run the lambda function locally for testing purpose:
./node_modules/.bin/serverless offline
Once it runs the servicebox lambda function is accessible via regular REST enpoints.
There is one significant difference with a real Lambda environment: internal state is lost between each call (this affects the calc/sum service). This should not be much a concern as Lambda functions shall be stateless and disposable; in that regard this serverless behavior is sane as it enforces the worse case scenario.
Servicebox needs a backend for the /api/v2/calc/sum
service which increments a counter into a backend. This service is intended to test 12-factor type of interaction with a backend used for transient data such as a session cache. All backends implement an atomic increment that is totally stateless and share-nothing; the same counters can be increment by any number of servicebox instances.
This is the default. No configuration needed.
To use cassandra as a backend add the following options:
--betype=cassandra --beopts='{"contactPoints":["46.101.16.49","178.62.87.192"]}'
Plain-text credentials can be set this way (no other credentials supported so far):
--betype cassandra --beopts '{"contactPoints":["52.88.93.64","52.89.85.132","52.89.133.153"], "authProvider":{"type":"PlainTextAuthProvider", "username":"username", "password":"p@ssword"}}'
Set load balancing policies:
--betype cassandra --beopts '{"contactPoints":["52.88.93.64","52.89.85.132","52.89.133.153"], "loadBalancingPolicy":{"type":"DCAwareRoundRobinPolicy","localDC":"DC_name_"}}'
When running as a lambda function, the same value as the --beopts
can be set in SB_BEOPTIONS
environment variable.
To use a redis cluster as a backend add the following options:
--betype=redis-cluster --beopts='{"drivercfg":[{"host":"localhost","port":"6379"}]}'
When running as a lambda function, the same value as the --beopts
can be set in SB_BEOPTIONS
environment variable.
The "drivercfg" JSON param is the IORedis client JSON connection object. Any IORedis config is supported. This is basically a list of of the cluster nodes to connect to in order to discover the cluster topology. A single node can be configured and this enough, but to improve reliability it is advised to configure three (3) nodes.
Note that the redis cluster client will try to connect to the cluster until it succeeds.
To use a redis with sentinels as a backend add the following options:
--betype=redis-sentinel --beopts='{"drivercfg":{ "sentinels": [ {"host": "104.131.130.202", "port":26379}, {"host": "104.236.144.145", "port":26379}, {"host": "104.236.145.222", "port":26379} ], "name": "mymaster" }}'
When running as a lambda function, the same value as the --beopts
can be set in SB_BEOPTIONS
environment variable.
The "drivercfg" JSON param is the IORedis client JSON connection object. Any IORedis config is supported. This is basically a list of the sentinels to connect to in order to discover the whole list of sentinels, masters and slaves. While a single sentinel can be configured it is advised to configure 3 sentinels to improve reliability.
Note that the redis client will try to connect to the groups until it succeeds.
To use DynamoDB as a backend add the following options (make sure to replace value with your key, your secret and your region):
--betype=dynamodb --beopts='{"accessKeyId": "yourkey", "secretAccessKey": "yoursecret","region": "us-west-2", "apiVersion": "2012-08-10"}'
When running as a lambda function, the same value as the --beopts
can be set in SB_BEOPTIONS
environment variable.
The DynamoDB shall have a servicebox
table with a string
id
attribute as the primary key.
A local DynamoDB can be used for testing with this sample set of options:
--betype=dynamodb --beopts='{"region":"us-west-2","apiVersion":"2012-08-10","endpoint":{"protocol":"http:","host":"localhost:8000","port":8000,"hostname":"localhost","pathname":"/","path":"/","href":"http://localhost:8000/"}}'
A local DynamoDB can be run this way:
wget http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.tar.gz
mkdir dynamodb
cd dynamodb
tar xvzf ../dynamodb_local_latest.tar.gz
java -Djava.library.path=./DynamoDBLocal_lib/ -jar DynamoDBLocal.jar -inMemory
The servicebox table can be created with this code snippet (run with nodejs):
var aws = require('aws-sdk');
var opts = {"region":"localhost","apiVersion":"2012-08-10","endpoint":{"protocol":"http:","host":"localhost:8000","port":8000,"hostname":"localhost","pathname":"/","path":"/","href":"http://localhost:8000/"}};
aws.config.update(opts);
var dynamodb = new aws.DynamoDB(opts);
var createdb_params = {
TableName: 'servicebox',
AttributeDefinitions: [
{ AttributeName: 'id', AttributeType: 'S' }
],
KeySchema:[
{ AttributeName: 'id', KeyType: 'HASH' }
],
ProvisionedThroughput: {ReadCapacityUnits: 5, WriteCapacityUnits: 5},
};
dynamodb.createTable(createdb_params, function(err, data) {
if (err)
console.log(err, err.stack); // an error occurred
else {
console.log('created servicebox table');
console.log(data);
}
});
Get Swagger definition at http://yourhost:8080/api/v2/swagger (this will return JSON or YAML based your Accept header).
Debug log level will log backend configuration strings, including credentials. Make sure NOT to use debug where this is a security issue.
- Add Redis-Sentinel integration tests
- Add Redis-Cluster integration tests
- Add txn logs
- Add support for Cassandra cluster options (same as in servicebod-jaxrs)
This software is under Apache 2.0 license.
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.