Azure Databricks Engineer - ETL/Python

We have Immediate Openings on Azure Databricks Engineer Job Description : - Design, develop, and maintain scalable data processing solutions using Azure Databricks and Azure Data Factory. - Build and optimize end-to-end data pipelines for batch and real-time data ingestion, transformation, and loading. - Develop complex ETL processes using PySpark on Databricks, ensuring high performance, scalability, and reliability. - Utilize Python to script and automate data workflows and transformations. - Write optimized SQL queries for querying and managing large datasets in Databricks and other data storage solutions. - Integrate Azure Data Factory pipelines with Databricks and other Azure services to orchestrate data workflows. - Collaborate with data engineers, data scientists, and analysts to understand data requirements and deliver effective data solutions. - Implement data validation, quality checks, and error-handling mechanisms to ensure data accuracy and integrity. - Monitor, troubleshoot, and optimize Databricks jobs and pipelines for performance and cost efficiency. - Stay up to date with the latest developments in Azure data services and best practices in cloud data engineering. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/27/2024