This library is published in the NPM registry and can be installed using any compatible package manager.
npm install firebolt-sdk --save
# For Yarn, use the command below.
yarn add firebolt-sdk
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
const connection = await firebolt.connect({
auth: {
username: process.env.FIREBOLT_USERNAME,
password: process.env.FIREBOLT_PASSWORD,
},
database: process.env.FIREBOLT_DATABASE,
engineName: process.env.FIREBOLT_ENGINE_NAME
});
const statement = await connection.execute("SELECT 1");
// fetch statement result
const { data, meta } = await statement.fetchResult();
// or stream result
const { data } = await statement.streamResult();
data.on("metadata", metadata => {
console.log(metadata);
});
data.on("error", error => {
console.log(error);
});
const rows = []
for await (const row of data) {
rows.push(row);
}
console.log(rows)
- About
- Documentation
- Usage
- Resource Manager
- Recipes
The Firebolt client for Node.js. firebolt-sdk provides common methods for quering Firebolt databases, fetching and streaming results, and engine management.
firebolt-sdk supports Node.js > v14
.
const connection = await firebolt.connect(connectionOptions);
type UsernamePasswordAuth = {
username: string;
password: string;
};
type AccessTokenAuth = {
accessToken: string;
};
type ServiceAccountAuth = {
client_id: string;
client_secret: string;
};
type ConnectionOptions = {
auth: UsernamePasswordAuth | AccessTokenAuth | ServiceAccountAuth;
database: string;
engineName?: string;
engineEndpoint?: string;
account?: string;
};
You can pass engineName: "system"
to use system engine, which is always on and execute AQL queries on it.
If engineName
or engineEndpoint
was not passed sdk will fallback to default engine endpoint for database.
Instead of passing username/password directly, you can also manage authentication outside of node sdk and pass accessToken when creating the connection
const connection = await firebolt.connect({
auth: {
accessToken: "access_token",
},
engineName: 'engine_name',
account: 'account_name',
database: 'database',
});
Instead of passing username/password, you can also use service account
const connection = await firebolt.connect({
auth: {
client_id: 'b1c4918c-e07e-4ab2-868b-9ae84f208d26';
client_secret: 'secret';
},
engineName: 'engine_name',
account: 'account_name',
database: 'database',
});
TODO: write motivation connection can be tested using:
const firebolt = Firebolt();
await firebolt.testConnection(connectionOptions)
which will perform authentication and simple select 1
query
Firebolt engine URLs use the following format:
<engine-name>.<account-name>.<region>.app.firebolt.io
For example: your-engine.your-account.us-east-1.app.firebolt.io
. You can find and copy your engine endpoint name in the Firebolt web UI.
const statement = await connection.execute(query, executeQueryOptions);
const statement = await connection.execute(query, {
settings: { query_id: 'hello' }
});
export type ExecuteQueryOptions = {
parameters:? unknown[];
settings?: QuerySettings;
response?: ResponseSettings;
};
parameters
field is used to specify replacements for ?
symbol in the query.
For example:
const statement = await connection.execute("select ?, ?", {
parameters: ["foo", 1]
});
will produce select 'foo', 1
query
Format Tuple
:
import { Tuple } from 'firebolt-sdk'
const statement = await connection.execute("select ? where bar in ?", {
parameters: [
1,
new Tuple(['foo'])
]
});
namedParameters
field is used to specify replacements for :name
tokens in the query.
For example:
const statement = await connection.execute("select :foo, :bar", {
namedParameters: { foo: "foo", bar: 123 }
});
will produce select 'foo', 123
query
Parameter | Required | Default | Description |
---|---|---|---|
output_format | JSON_COMPACT | Specifies format of selected data |
You can also use QuerySettings to specify set flags.
For example: { query_id: 'hello' }
Parameter | Required | Default | Description |
---|---|---|---|
normalizeData | false | Maps each row in response from array format to object | |
bigNumberAsString | false | Hydrate BigNumber as String |
const { data, meta, statistics } = await statement.fetchResult();
The Promise API is not recommended for SELECT
queries with large result sets (greater than 10,000 rows). This is because it parses results synchronously, so will block the JS thread/event loop and may lead to memory leaks due to peak GC loads.
It is recommended to use LIMIT
in your queries when using the Promise API.
const { data } = await statement.streamResult();
const rows: unknown[] = [];
data.on("metadata", metadata => {
console.log(metadata);
});
data.on("error", error => {
console.log(error);
});
for await (const row of data) {
rows.push(row);
}
firebolt-sdk maps SQL data types to their corresponding JavaScript equivalents. The mapping is described in the table below:
Category | SQL type | JavaScript type | Notes |
---|---|---|---|
Numeric | INT | Number | If value cannot be represented by JavaScript Number (determine using Number.isSafeInteger), BigNumber from "bignumber.js" is used |
INTEGER | Number | ||
BIGINT | Number | ||
LONG | Number | ||
FLOAT | Number | ||
DOUBLE | Number | ||
String | VARCHAR | String | |
TEXT | String | ||
STRING | String | ||
Date & Time | DATE | Date |
Engines can be managed by using the resourceManager
object.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const enginesService = firebolt.resourceManager.engine
Returns engine using engine ID
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getById(
"c8a228ea-93df-4784-99f9-a99368518782",
);
Returns engine using engine name.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
Property | Type | Notes |
---|---|---|
id |
{engine_id: string; account_id: string} |
|
name |
string |
|
endpoint |
string |
|
description |
string |
|
current_status_summary |
string |
Starts an engine.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.start()
Stops an engine.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.stop()
Restarts an engine.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.restart()
Databases can be managed by using the resourceManager
object.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const databaseService = firebolt.resourceManager.database
Returns database using database ID
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getById(
"c8a228ea-93df-4784-99f9-a99368518782",
);
Returns database using database name.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name")
Property | Type | Notes |
---|---|---|
id |
{engine_id: string; account_id: string} |
|
name |
string |
|
description |
string |
It is possible to create resourceManager
separately from firebolt client,
providing only auth credentials
import { FireboltResourceManager } from 'firebolt-sdk'
const resourceManager = FireboltResourceManager();
await resourceManager.authenticate({
auth: {
username: process.env.FIREBOLT_USERNAME as string,
password: process.env.FIREBOLT_PASSWORD as string,
},
account: process.env.ACCOUNT_NAME as string
});
const engine = await resourceManager.engine.getByName(
process.env.FIREBOLT_ENGINE_NAME as string
);
The recommended way to consume query results is by using streams.
For convenience, statement.streamResult
also returns meta: Promise<Meta[]>
and statistics: Promise<Statistics>
, which are wrappers over data.on('metadata')
and data.on('statistics')
.
const firebolt = Firebolt();
const connection = await firebolt.connect(connectionParams);
const statement = await connection.execute("SELECT 1");
const {
data,
meta: metaPromise,
statistics: statisticsPromise
} = await statement.streamResult();
const rows: unknown[] = [];
const meta = await metaPromise;
for await (const row of data) {
rows.push(row);
}
const statistics = await statisticsPromise
console.log(meta);
console.log(statistics);
console.log(rows)
To achieve seamless stream pipes to fs
or stdout
, you can use the Transform
stream.
import stream, { TransformCallback } from 'stream';
class SerializeRowStream extends stream.Transform {
public constructor() {
super({
objectMode: true,
transform(
row: any,
encoding: BufferEncoding,
callback: TransformCallback
) {
const transformed = JSON.stringify(row);
this.push(transformed);
this.push('\n')
callback();
}
});
}
}
const serializedStream = new SerializeRowStream()
const firebolt = Firebolt();
const connection = await firebolt.connect(connectionParams);
const statement = await connection.execute("select 1 union all select 2");
const { data } = await statement.streamResult();
data.pipe(serializedStream).pipe(process.stdout);
Or use rowParser
that returns strings or Buffer:
const { data } = await statement.streamResult({
rowParser: (row: string) => `${row}\n`
});
data.pipe(process.stdout);
cp .env.example .env
npm test
Released under Apache License.