Principal Data Engineer - PySpark/Python

Roles & Responsibilities : - It is a key lead role within data engineering team - Perform a role of technical lead for all the new development and provide technical plan for development. - Responsible for providing sign-off for new designs and highlighting if any risks or challenges. - Responsible to providing technical plan & sizing for new features or BAU items - Perform a role of mentor & coach to guide and support engineers during the development process. - Consistently ensure best standards of coding are maintained and feedback is provided for improvements in sprint meetings. - Responsible for delivering effectively in agile manner, by breaking down the complex stories into actionable & distributed tasks. - Responsible for robust, resilient deliverable - Responsible for technical details in requirement grooming, planning & sprint review stage. - In retro, plays a role of lead to make decisions, resolve conflicts, and provide actionable outcomes for engineering challenges. - Responsible for SDLC process which is not people dependent and has minimal impact due to absence. - People/knowledge dependency challenge is managed. - Responsible for ensuring code encapsulations, testability, easily understandable whilst accounting for known edge cases & coverage. Skills & Experience : - 10 years of design & development experience with big data technologies like Azure. - Preferred is Azure & Databricks, with experience in Azure DevOps - Experience in data visualising technology in DL, like PowerBI - Proficient in Python, PySpark and SQL - Proficient in querying and manipulating data from various DB (relational and big data). - Experience of writing effective and maintainable unit and integration tests for ingestion pipelines. - Experience of using static analysis and code quality tools and building CI/CD pipelines. - Excellent communication, problem-solving, and leadership skills, and be able to work well in a fast-paced, dynamic environment. - Experience of working on high-traffic and large-scale Software Products. Behavioural Fit : - Technical with a keen eye for detail. - Self-Driven, self-motivated and results oriented. - Confident and an ability to challenge if necessary. - Process driven, organised & well-structured. - Ability to work in a cross-functional, multi-cultural team and in a collaborative environment with minimal supervision. - Ability to multi-task and plan, organize and prioritize multiple projects (ref:hirist.tech)

Location: chennai, IN

Posted Date: 4/19/2025