Big Data Developer - Hadoop/Spark

Role Overview : We are looking for a skilled and experienced Big Data Developer to join our data engineering team. The ideal candidate will have a strong foundation in Big Data technologies and a proven track record of building scalable and efficient data pipelines. You will play a critical role in enabling data-driven decision-making across the Responsibilities : - Design, develop, and optimize big data pipelines using tools and technologies from the Hadoop ecosystem. - Work with large datasets to extract, transform, and load (ETL) data for analytical and operational purposes. - Build scalable data ingestion frameworks for real-time and batch processing. - Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver high-quality solutions. - Ensure data quality, governance, and compliance standards are maintained. - Monitor and troubleshoot big data infrastructure and performance issues. - Document design decisions, processes, and technical Technical Skills : Strong experience with Big Data technologies : - Hadoop, Spark, Hive, HBase, Pig, Sqoop Hands-on experience with data processing frameworks like : - Apache Spark (RDD/DataFrame/SQL APIs), Kafka, Flink Proficiency in programming languages such as : - Java, Scala, Python - Strong knowledge of SQL and NoSQL databases (e.g., MongoDB, Cassandra). - Experience in developing and managing data workflows using Apache Airflow, Oozie, or similar tools. - Familiarity with cloud platforms (AWS / Azure / GCP) and cloud-native big data services (e.g., EMR, Databricks, BigQuery). - Knowledge of CI/CD pipelines, version control tools like Git, and DevOps practices is a plus. - Experience in data modeling, data governance, and performance : - Bachelors or Masters degree in Computer Science, Information Technology, or a related field. - 58 years of experience in Big Data development and data engineering roles. - Strong analytical, problem-solving, and communication skills. - Ability to work in a collaborative, Agile/Scrum environment. (ref:hirist.tech)

Location: pune, IN

Posted Date: 4/19/2025