Data Engineer - SQL/Python

Job Summary : The purpose of this role is to provide end-to-end data integration for the Product. The Data Engineer works with the business's analysts, software developers, and data analysts engineers in order to understand and aid in the implementation of database integration requirements and troubleshoot any existent issues. Key responsibilities : - Assembles large, complex data sets that meet functional / non-functional business requirements - Identifies, designs and implements internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc - Works with stakeholders including the Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs - Implement security measures to protect sensitive data, essential for maintaining trust and compliance with data privacy regulations. - Works with data analytics experts to strive for greater functionality in our data systems Must have : - Data Architecture & Modeling : Design and maintain scalable, efficient data models and architectures to support data analytics and reporting. - Data Pipeline Engineering : Develop, maintain, and optimize scalable data pipelines that can handle large volumes and various types of data. - Data Quality Assurance : Implement rigorous data cleaning, transformation, and integration processes to ensure data quality and consistency. - Collaboration : Work closely with data analysts, business analysts, and other stakeholders to understand data requirements and implement effective data solutions. - Documentation & Governance : Maintain comprehensive documentation of data procedures, systems, and architectures. Provide guidance and support for data governance practices, including metadata management, data lineage, and data cataloging. Technical Skills : - Strong proficiency in Python, with an emphasis on clean, modular, and well- documented code. - Expertise in SQL, JIRA, Git, and GitHub. - Good Communication Skills: Able to explain complex technical concepts clearly and concisely to both technical and non-technical audiences. - Hands-on experience with designing and implementing scalable and secure data processing pipelines using any cloud service (Azure/ Google/ AWS, etc) Good to have : - Azure Cloud Expertise : Hands-on experience in using Azure cloud services and tools like Azure Data Factory or Azure Synapse Analytics. - Azure Data Management : Experience managing and optimizing data storage within Azure using services like Azure SQL Data Warehouse and Azure Blob Storag (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 11/29/2024