Technical Lead - Data Engineering

Role Overview : We are looking for a Technical Lead to drive a large-scale Teradata-to-AWS Databricks migration and play a key role in designing and optimizing cloud-based data pipelines. The ideal candidate will have deep expertise in Databricks, ETL/ELT development, and relational databases, along with a strong background in performance tuning, automation, and team leadership. Key Responsibilities : - Lead and execute a large-scale Teradata-to-AWS Databricks migration strategy, ensuring a seamless and efficient transition. - Architect, design, and implement scalable data pipelines leveraging Databricks tech stack, including Delta Lake, Spark, Unity Catalog, Databricks Workflows, and Lakehouse architecture. - Develop and optimize ETL/ELT workflows, leveraging IBM DataStage, Apache Airflow, FiveTran, and other modern tools to streamline data processing. - Strong expertise in Stored Procedures, PL/SQL, and SQL across relational databases like Oracle, SQL Server, and DB2, with experience in performance tuning and query optimization. - Automate data ingestion, validation, and quality checks, ensuring high accuracy and efficiency in data processing. - Drive best practices in data governance, security, and compliance, ensuring that cloud migration adheres to industry standards. - Collaborate with senior architects, engineers, and business stakeholders to align the migration strategy with business objectives and technical feasibility. - Lead and mentor a technical team, fostering an environment of innovation, continuous learning, and best practices in cloud data engineering. - Work on performance tuning, security enhancements, and cost optimization to ensure efficient use of cloud resources. - Stay up to date with emerging technologies in cloud data engineering, Databricks, and AI-driven optimizations to enhance workflows and improve efficiency (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 4/20/2025