N-Systems Technologies - Azure Databricks Engineer

Job description : We are looking for Nos10 - Azure Databricks Engineer with 5 years of relevant experience. The work location is Pune, MH (Onsite) . The ideal candidate should have experience, primarily in Databricks and Python. The candidate should be able to design, develop, and maintain data pipelines and data streams. The candidate should also be able to extract and transform data, especially unstructured data, across various data processing layers using Databricks, Python. Job Responsibilities : - Design, develop, and optimize data pipelines using Azure Databricks - Manage and organize large datasets in Data Lakes, ensuring data quality and integrity - Write complex SQL queries to extract, transform, and load data from various sources. - Monitor and troubleshoot data pipeline performance and resolve issues as they arise. - Stay up to date with the latest advancements in data engineering and propose innovative solutions to improve data processes. - Leverage your deep understanding of Azure services to architect solutions that adhere to best practices for security, scalability, and reliability. - Utilize services like Azure Data Factory, Azure Databricks, Azure Storage, and more to create comprehensive data solutions. Required Qualifications : - Generic Managerial Skills - Experience working in agile development projects and sprint deliver - Experience in generating data pipelines in Azure ADF - Experience in ingesting data from different Sources using in Azure ADF - Experience in transforming the data using Azure Databricks & Delta Lake - Experience in Azure DevOps CI/CD Stories, Bugs, and Issue management - Implement Integration solution in Microsoft Azure - Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues - Experience working in agile development projects and sprint deliver (ref:hirist.tech)

Location: pune, IN

Posted Date: 11/25/2024