Skip to content

Commit

Permalink
Document the test setup for single and sharded testing
Browse files Browse the repository at this point in the history
  • Loading branch information
lutter committed Mar 25, 2021
1 parent b2e02e9 commit d94f1a2
Show file tree
Hide file tree
Showing 4 changed files with 128 additions and 3 deletions.
57 changes: 57 additions & 0 deletions store/test-store/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Setting up for running tests

This document describes what needs to be done to run tests against a
Postgres database.

## Initial setup

Before starting, you will need to set up a database user `graph` and give
it the password `graph` by running `createuser -W graph`.

You also need to arrange for `psql` and similar tools to connect to
Postgres automatically as the `postgres` user.

For that, add this line as the first non-comment line to
`$PGDATA/pg_hba.conf`:
```
local all postgres peer map=admin
```

and add these lines to `data/pg_ident.conf`:
```
admin postgres postgres
admin <your username> postgres
```

You might also want to set `log_statement = 'all'` to
`$PGDATA/postgresql.conf`. That will cause Postgres to log all SQL
statements that the tests send it which can sometimes be very helpful when
tests fail.

After making all these changes, restart Postgres to make sure it actually
uses these settings.

## Resetting the test databases

The script `db-reset` will (re)create databases in the test cluster. This
script can be used to remove old databases and/or broken databases schemas
and cause `cargo test` to create the schema from scratch on the next test
run.

Note that `db-reset` will completely delete the databases `graph-test` and
`graph-sgd` in the databases cluster that a plain invocation of `psql`
connects to.

## Running tests

Before running tests set the environment variable `GRAPH_NODE_TEST_CONFIG`
to point either at `config.simple.toml` or `config.sharded.toml`.

Run tests with `cargo test --workspace --exclude graph-tests`; this will
run all unit and integration tests. The `graph-tests` crate contains more
extensive integration tests that depend on running docker containers; that
setup is outside the scope of the document, and we therefore exclude them
from running for basic usage.

When you switch from one of the test configurations to another, you will
need to clean out the test databases by running `db-reset`.
35 changes: 35 additions & 0 deletions store/test-store/config.sharded.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# A sharded setup that uses two databases
[store]
[store.primary]
connection = "postgresql://graph:graph@127.0.0.1:5432/graph-test"
pool_size = [ { size = 10 } ]
[store.sgd]
connection = "postgresql://graph:graph@127.0.0.1:5432/graph-sgd"
pool_size = [ { size = 10 } ]

[deployment]
[[deployment.rule]]
match = { name = "abi.*|grafted" }
shard="primary"
indexers = [ "default" ]

[[deployment.rule]]
shard = "sgd"
indexers = [ "default" ]

[chains]
ingestor = "default"

# The tests do not talk to ethereum clients; we use valid free client
# endpoints as an example
[chains.fake_network]
shard = "sgd"
provider = [
{ label = "penguin", url="https://main-light.eth.linkpool.io/", features = [] }
]

[chains.fake_network_shared]
shard = "sgd"
provider = [
{ label = "owl", url="https://rinkeby-light.eth.linkpool.io/", features = [] }
]
21 changes: 18 additions & 3 deletions store/test-store/config.simple.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,27 @@

[store]
[store.primary]
connection = "<postgres URL>"
connection = "postgresql://graph:graph@127.0.0.1:5432/graph-test"
pool_size = 10

[deployment]
[[deployment.rule]]
store = "primary"
indexers = [ "default" ]

[ingestor]
node = "default"
[chains]
ingestor = "default"

# The tests do not talk to ethereum clients; we use valid free client
# endpoints as an example
[chains.fake_network]
shard = "primary"
provider = [
{ label = "penguin", url="https://main-light.eth.linkpool.io/", features = [] }
]

[chains.fake_network_shared]
shard = "primary"
provider = [
{ label = "owl", url="https://rinkeby-light.eth.linkpool.io/", features = [] }
]
18 changes: 18 additions & 0 deletions store/test-store/db-reset
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#! /bin/bash

resetdb() {
host=$(hostname -s)
db=$1
echo "Reset $db"
dropdb $db && createdb -O graph $db
psql -q -X $db <<EOF
create extension pg_trgm;
create extension pg_stat_statements;
create extension btree_gist;
create extension postgres_fdw;
grant usage on foreign data wrapper postgres_fdw to graph;
EOF
}

resetdb graph-test
resetdb graph-sgd

0 comments on commit d94f1a2

Please sign in to comment.