VAYUZ Technologies - Lead Data Engineer

Responsibilities and accountabilities : Data Collection and Integration : - Collect data from various sources, including databases, APIs, external data providers, NoSQL, unstructured data and streaming sources. Design and implement efficient data pipelines to ensure a smooth flow of information into the data warehouse or storage system. Data Storage and Management : - Choose appropriate database systems, optimize data schemas, and ensure data quality and integrity. Consider scalability and performance to handle large volumes of data. ETL (Extract, Transform, Load) Processes : - Design ETL pipelines to transform raw data into a format suitable for analysis. This involves data cleansing, aggregation, and enrichment, ensuring the data is usable. Big Data Technologies and Cloud Computing : - Big data and OLAP technologies such as Google Cloud Big Query, Hadoop and Spark to efficiently process and analyze massive datasets. Enterprise Data Governance : - Define and ensure adherence to the Enterprise Data Governance framework for data practices, policies, and standards both at a technical and a functional level. Work closely with various data owners to develop and implement Data Governance Framework. Distributed Systems : - Handle huge data volumes and ensure fault tolerance. Understanding how distributed systems work is essential for data engineers. Streaming Data (CDC) : - Work with streaming technologies like Qlik Replicate, Informatica CDC, Apache Kafka to handle and analyze data as it flows in. Qualification and experience : - 15 years of hands-on experience in the field of Data Integration and Governance; out of which at least 10 years of leading a team of Data Engineers and Data Governance specialists. - Previously held roles leading Data Engineering teams for dynamic, growing organizations. - Natural enthusiasm to keep up to date on emerging technologies and methodologies to continually improve and advance data solutions where possible. - Knowledge and experience in building end-to-end Data Pipelines. - Knowledge and experience in building Enterprise Data Governance Lifecycle based on the Maturity Model. - Experience in implementing Data Availability, Data Quality, Data Catalogue, Data Lineage & Data Security initiatives across the Enterprise. Having worked in a regulated industry would be a plus. - Experience in defining and enforcement of Data Engineering and Governance principles and policies across the Enterprise. - Knowledge of Relational and NoSQL databases. Hands-on experience in T-SQL / PL-SQL programming. - Bachelor's degree in IT, Computer Science, or a related field; Master's degree will be a plus. (ref:hirist.tech)

Location: mumbai, IN

Posted Date: 1/14/2025