Senior Azure Data Engineer Lead - Python/SQL

Salary : 30 to 50 LPA Job Title : Azure Data Engineer Lead Experience:8 Years min Budget:Up to 35 LPA Location : Mumbai (Powai) or Pune (Viman Nagar) - Hybrid We are seeking an experienced Azure Data Engineer Lead. The ideal candidate will have years of experience in building and managing data pipelines using Azure Data Factory,Azure Databricks, and advanced Python and PySpark skills. You will be responsible for handling large-scale data transformations, ensuring data quality, and optimizing data architecture for scalability. Key Responsibilities : - Lead the design, development, and implementation of data pipelines using Python and PySpark on Azure Databricks. - Manage large-scale data transformations and ensure efficient data processing in the cloud. - Implement, troubleshoot, and optimize data workflows using Azure Data Factory. - Work closely with cross-functional teams to define data requirements and deliver efficient solutions. - Ensure data quality, consistency, and security in all data pipelines. - Collaborate on the continuous improvement of data architecture and processes. Required Skills : - Azure Databricks : Expertise in developing and managing data pipelines on Databricks. - Azure Data Factory (ADF) : Proven experience in creating, running, and troubleshooting data pipelines. - Python & PySpark : Strong hands-on experience in Python, PySpark, and managing large-scale data transformations. - SQL : Strong knowledge of SQL for managing and querying large datasets. - Experience with large-scale data processing and building efficient data workflows. Nice to Have : - Knowledge of CI/CD processes and Azure DevOps. - Experience with Data Lake architecture. - Experience working with financial data or handling high volumes of business-critical data. (ref:hirist.tech)

Location: mumbai, IN

Posted Date: 11/25/2024