Skip to content

this package is wrapper of bigquery-emulator which provides us BigQuery mock working in local machine.

Notifications You must be signed in to change notification settings

m3dev/bqemulatormanager

 
 

Repository files navigation

BiqQueryEmulator Manager

this package is wrapper of bigquery-emulator which provides us BigQuery mock working in local machine.

using this package, you can

  • do unit test of your sql
  • download the schema of big query, and use it to make test data

usage

  1. following instruction, download bigquery-emulator command.

  2. install this package.

pip install bqemulatormanager
  1. test your sql.
import bqemulatormanager as bqem
import pandas as pd

manager = bqem.Manager(project='test', schema_path='resources/schema_example.yaml')

with manager:
    data = pd.DataFrame([
        {'id': 1, 'name': 'sato'},
        {'id': 2, 'name': 'yamada'}
    ])

    manager.load(data, 'dataset1.table_a')

    sql = 'SELECT id, name FROM `dataset1.table_a`'

    df = manager.query(sql)
print(df)

unit test example is here.

automatically detect schema

When called Manager.load, SchemaManager search correspond table schema from schema_path (default is bqem_master_schema.yaml).

If schema definition cannot be found, SchemaManager request it from BigQuery in production environment and update master schema file.

About

this package is wrapper of bigquery-emulator which provides us BigQuery mock working in local machine.

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%