Save Your Wardrobe
Data scientist
474 Application(s)
454 Rejected
ETL Migration to Spark and Power BI Dashboards
Project Overview
This project focuses on the migration of our current ETL processes to Apache Spark, a powerful big data processing framework. It also includes the development of data dashboards utilizing Power BI for improved data visualization and analysis. The goal is to modernize our data processing and reporting infrastructure, making it more efficient and capable of handling large volumes of data.
Project Objectives
ETL Migration: Transfer existing ETL processes to Apache Spark, ensuring faster and scalable data processing.
Data Transformation: Implement data transformation logic within Spark to enhance data quality and consistency.
Power BI Dashboards: Design and develop interactive dashboards in Power BI for data visualization, reporting, and analysis.
Automation: Implement automated ETL pipelines and scheduled updates for dashboards.
Tools and Frameworks
Apache Spark, Apache Airflow, Power BI, Big Data, Data Storage, Jira, Git, MongoDB, AWS, Athena, S3
