Data Engineer - Spark/Python

Job Description : - 5-10 years of meaningful experience in data engineering working on building ETL pipelines, big data tech like spark. - Very good expertise in algorithms and data structures - Excellent in at least one programming language: Python, Go, R - Hands-on expertise big data technologies such as Apache Spark, Hadoop, message queues(Kafka), Apache Airflow, Azure ML, Elastic Search - Databases: Hands-on expertise in data modeling, database schema definition, query optimization, usage of relevant data stores (blob storage, data warehouse, MongoDB, Timescale DB), - Hands-on expertise working on analytics platforms such as Azure synapse or equivalent - Good understanding of performance analysis and optimizations - Hands-on experience in at least one cloud platform - Azure(preferred), AWS, or GCP - B.E/B.Tech/M.E/M.Tech/M.S in Computer/Electronics Engineering or equivalent degrees with experience Nice to have : - Knowledge of data visualization tools such as Power BI, Tableau - Working knowledge of the "Internet of Things" desired - Knowledge of internet-scale cloud-based distributed systems - Good understanding of MQTT, Web Sockets, Azure Service Bus, Apache Pulsar - Good knowledge of Domain-Driven Design, Test Driven Development - Experience developing in an Agile / SCRUM Environment Soft skills : - Self-driven, results-oriented, collaborative, comfortable with a fast-paced, always-on, highly ambiguous start-up like environment - Good with rapid problem solving, a solid technical background, and a passion for continuous learning and improvement - Good verbal and written communication skills (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 12/26/2024