Data Engineer - ETL

We are looking for a talented Data Engineer with experience in ETL processes, database administration, and scripting, as well as expertise in tools such as Snowflake, Azure Data Factory, DBT, and Talend. The ideal candidate will have a strong technical background and be capable of working on various aspects of data engineering, from building data pipelines to optimizing data storage and retrieval. Key Responsibilities : ETL Development and Maintenance : - Design, develop, and maintain scalable ETL processes to extract, transform, and load data from various sources into data warehouses and data lakes. - Utilize Talend and Azure Data Factory to create and manage ETL workflows that ensure data accuracy and efficiency. Database Administration : - Administer and optimize databases, ensuring high performance, availability, and security of data assets. - Manage Snowflake data warehouse, including schema design, data partitioning, and performance tuning. Data Engineering and Scripting : - Develop and maintain Python scripts for data processing, automation, and integration tasks. - Implement and optimize data transformation processes using DBT (Data Build Tool) to ensure data quality and consistency. Data Integration and Management : - Integrate data from multiple sources and ensure seamless data flow and accessibility. - Collaborate with data analysts and data scientists to understand data requirements and deliver relevant solutions. Performance Optimization : - Monitor and optimize data pipelines and database performance to ensure efficient data processing and retrieval. - Troubleshoot and resolve issues related to data pipelines, database operations, and data quality. Documentation and Best Practices : - Document data engineering processes, workflows, and architecture to support knowledge sharing and maintenance. - Implement best practices for data engineering, including data governance, version control, and code review. Collaboration and Communication : - Work closely with cross-functional teams to gather requirements, provide updates, and ensure alignment on data engineering projects. - Communicate technical concepts and solutions effectively to both technical and non-technical stakeholders. Qualifications : Education & Experience : - Bachelor's degree in Computer Science, Data Engineering, or a related field. Advanced degree preferred. - Minimum of 5 years of experience in data engineering or a similar role, with hands-on experience in ETL processes and database administration. Technical Skills : - Proficiency in ETL tools such as Talend and Azure Data Factory. - Experience with Snowflake data warehouse platform. - Strong scripting skills in Python for data processing and automation. - Knowledge of DBT (Data Build Tool) for data transformation and modelling. - Experience with database administration, performance tuning, and optimization. Additional Skills : - Familiarity with cloud platforms (e.g., Azure, AWS, Google Cloud) is a plus. - Strong problem-solving skills and attention to detail. - Ability to work independently and as part of a team in a fast-paced environment. - Excellent communication and interpersonal skills. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 11/28/2024