Data Engineer - ETL/Python

Job Description : Data Engineer Location : Bangalore, India Position : Data Engineer Employment Type : Full-Time (General IST Timings) Experience : 3-6 Years Notice Period : Immediate Joiners or within 15 Days Interview Process : Must be available for a face-to-face final round in Bangalore About the Role : We are seeking a talented and experienced Data Engineer to join our Data Science team in Bangalore. In this role, you will be instrumental in building and maintaining our central repository of financial and market datasets. You will design and implement scalable ETL pipelines, ensure data quality and governance, and support machine learning applications that drive business growth. This position offers a unique opportunity to work with cutting-edge technologies and contribute to high-impact projects. Key Responsibilities : - Design, develop, and optimize data transformation and validation processes using Python, Java, or Scala. - Build and maintain robust ETL pipelines to ingest, process, and store large datasets. - Design and build scalable, high-performance data pipelines on cloud platforms (Azure preferred). - Implement data pipelines that can handle batch and real-time data processing. - Design efficient data models for analytics and reporting, ensuring data integrity and performance. - Develop and maintain data schemas for relational and non-relational databases. - Implement data quality checks and validation processes to ensure data accuracy and consistency. - Establish and enforce data governance policies and best practices. - Maintain clear and comprehensive documentation for data pipelines, data models, and ETL processes. - Communicate effectively with stakeholders and collaborate across teams. - Work closely with the Quant and Development teams to understand data requirements and deliver solutions. - Collaborate with data scientists to support machine learning initiatives. - Implement and maintain web scraping tools to gather data from various online sources. Essential Requirements : - Experience : 3-6 years of experience in ETL development and data engineering. - Strong programming skills in Python, Java, or Scala. - Experience with web scraping for data extraction. - Hands-on experience with Azure (preferred) or other cloud platforms (AWS/GCP). - Expertise in SQL and experience with relational and non-relational databases. - Knowledge of data warehousing concepts and big data technologies (Hadoop, Spark, Kafka). - Strong analytical and problem-solving skills. Preferred Qualifications : - Education : Master's degree in Computer Science or Data Science. - Real-Time Processing : Experience with real-time data processing and streaming frameworks. - Data Governance & Security : Knowledge of data governance and security best practices. Technical Skills : - Python, Java, or Scala programming. - SQL and database management (relational and non-relational). - Azure cloud services (or AWS/GCP). - ETL tools and techniques. - Data modeling and data warehousing. - Big data technologies (Hadoop, Spark, Kafka). - Web scraping tools and libraries. - Data quality and governance principles. Soft Skills : - Strong analytical and problem-solving skills. - Excellent communication and collaboration skills. - Ability to work independently and as part of a team. - Strong attention to detail and accuracy. - Ability to adapt to changing priorities and requirements. -Why Join Us? - Opportunity to work on challenging and impactful projects. - Work with cutting-edge technologies and cloud platforms. - Collaborative and supportive team environment. - Career growth and development opportunities. - Contribute to the development of advanced analytics solutions (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 4/19/2025