Data Engineer - Big Data Solutions

Job Description : Job role : Data Engineer Developer. Experience : 5-8 Years. Location : Chennai, Pune. Skills : Hadoop(HDFS, Hive, HBase, YARN), Spark, Apache, SQL, Kafka (or) Flink, AWS/Azure/GCP. Job Type : On-site. Job Mode : Full Time. Notice Period : Immediate to 15 days. Key Responsibilities : - Design, develop, and implement scalable Big Data solutions using Hadoop, Spark, and other data processing frameworks. - Develop ETL pipelines to ingest, process, and transform large data sets efficiently. - Work with structured, semi-structured, and unstructured data to create data pipelines that enable actionable insights. - Build and maintain data workflows and automated solutions for data processing. - Collaborate with cross-functional teams to gather requirements and deliver data solutions. - Ensure data quality, consistency, and security throughout the data lifecycle. - Optimize performance of data pipelines, ensuring minimal latency and high throughput. - Troubleshoot and resolve data-related issues, ensuring data integrity and availability. - Document system designs, data flows, and key technical details. Technical Skills Required : - Expertise in Hadoop Ecosystem (HDFS, Hive, HBase, YARN, etc. - Proficiency in Apache Spark, Kafka, and Scala/Java/Python. - Strong understanding of SQL and data manipulation techniques. - Experience with cloud platforms like AWS, Azure, or GCP. - Familiarity with data streaming technologies such as Kafka or Flink. - Hands-on experience with data visualization tools and BI platforms is a plus. Qualifications : - Bachelor's/Master's degree in Computer Science, IT, or a related field. - 5-8 years of experience in Big Data development and data engineering. - Strong problem-solving skills with excellent analytical capabilities. - Ability to work independently and as part of a collaborative team environment. Preferred Skills : - Experience with real-time data processing. - Familiarity with containerization technologies like Docker and orchestration tools like Kubernetes. - Knowledge of CI/CD pipelines and DevOps practices. (ref:hirist.tech)

Location: in, IN

Posted Date: 4/19/2025