Follow the steps below to setup the local environment for development.
- Azure Subscription
- python version 3.9
Optional:
- Visual Studio Code
- VSCode Extensions (Managing Extensions):
ms-azuretools.vscode-azurefunctionsms-python.python
- docker (If wanting to use Megalinter container)
pip install -r requirements.txtSet the listed env vars in the table below in a
functions/local.settings.jsonfile.
touch functions/local.settings.json| Name | Example Value | Description |
|---|---|---|
FUNCTIONS_WORKER_RUNTIME |
python | The runtime of the Azure Functions |
AzureWebJobsStorage |
DefaultEndpointsProtocol=https;AccountName=<storage_acct>;AccountKey=xx-xx-xx==;EndpointSuffix=core.windows.net |
Storage Account connection string for Azure Web Jobs used by Functions |
datazoom_STORAGE |
DefaultEndpointsProtocol=https;AccountName=<storage_acct>;AccountKey=xx-xx-xx==;EndpointSuffix=core.windows.net |
Storage Account connection string for Input blobs |
MANAGED_CLIENT_ID |
XXX-XXX-XXX |
Azure Managed Service Identity Client ID |
KUSTO_URI |
https://<data_explorer_resource_name>.<region>.kusto.windows.net" |
Azure Data Explorer Kusto Cluster Resource URI. |
KUSTO_DATABASE |
testdata |
The name of the targeted Kusto Database |
SLOW_START_TABLE |
slow_start_anomaly_detection |
Slow Start Anomaly Detection Table |
METRICS_ADVISOR_ENDPOINT |
https://name-metricsadvisor.cognitiveservices.azure.com/ |
Metrics Advisor Endpoint |
METRICS_ADVISOR_SUBSCRIPTION_KEY |
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |
The subscription key to your Metrics Advisor. Can be found in Keys and Endpoint section of metrics advisor resource in the Azure portal |
METRICS_ADVISOR_API_KEY |
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx |
Metrics Advisor API Key. Can be found in Azure Metics Advisor Workspace |
METRICS_ADVISOR_ALERT_CONFIGURATION_ID |
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx |
Metrics Advisor configuration ID. Can be found in Azure Metics Advisor Workspace |
To run the Azure Functions locally, follow this guide: Develop and Code Azure Functions Locally
This project allows for extending the Data Transformation logic into more custom logic that fits your use case.
This doc will walk you through how to extend the Transformation and add custom logic and scripts: Develop Custom Data Transformations
The linter used is the Megalinter.
To use locally, follow this guide: Using Megalinter Locally
There are a few ways to deploy Azure Functions.
Go to the next step to change the data transformation logic based on your project needs. Extending and Customizing Transformation Logic