Data pipeline project (ELT using Microsoft Azure)

Scala code to convert CSV files stored in Azure Blob Storage to Parquet and store into Azure Storage, using Data bricks notebook and ARM template to run the notebook as a Azure Data Factory Job

The future of sustainability and training: involvement and performance of companies and strategic suppliers

This is a project where data ingestion, data transformation, data preparation and other data activities including Azure SQL. Making pipelines production ready, monitoring and CI/CD implementation.

Bugs In Cloud - Elenco dei Video

Contains solutions/versions of Batch Data Pipelines created on Azure Data Factory

Tokyo Olympics Data Analysis: Creating a ETL pipeline using Azure Data Factory to ingest data, transform it using Azure Databricks and querying and building reports using tools like Synapse Analytics and PowerBI

Azure DevOps CICD for Azure Data Factory

This project presents a sophisticated data-driven web application that integrates React for frontend visualization, NodeJS for backend data retrieval, and Microsoft's Cosmos DB for data storage. Leveraging the fault tolerance, partitioning, replication, and global distribution advantages of Cosmos DB.

This repository contains a full setup for a Snowflake and Azure Data Factory pipeline, including a detailed README, a sample sales dataset for data ingestion and transformation, and Snowflake-generated analysis charts. It's ideal for learning Snowflake-Azure integration and automating data workflows.

A Covid-19 Project on Azure Cloud

The aim of this project is to build a cost efficient Data Warehouse on Amazon's Retail sales data and perform Customer lifetime value analyses.

Notes on ADF (Azure Data Factory)

Copying data from Amazon S3 bucket to Azure Blob container by using Azure Data Factory pipeline. This Data is mounted to Databricks and further analysis is done using Spark SQL.

Repository created for programming and development in the Azure Data Enginner.

This project creates an end-to-end data pipeline and interactive dashboard for analyzing mutual funds' performance using Microsoft Azure and Power BI. It leverages Azure Data Factory, Data Lake Storage, SQL Database, and Databricks to build a scalable, efficient pipeline, providing real-time insights and data-driven decision-making.

data pipeline azure databricks

END TO END DATA ENGINEERING PROJECT