Software Development Engineer II - Data Engineering

SDE2 Data Engineer About : My client is a generative AI company dedicated to transforming workplaces with innovative AI solutions and micro-applications built on the GenAI stack. Our mission is to enhance efficiency and scalability across various industries, including education, legal and compliance, and auditing and taxation. We leverage deep industry relationships and technical expertise to deliver cutting-edge AI products that revolutionize traditional workflows. Job Summary : As an SDE2 Data Engineer , you will play a pivotal role in designing, developing, and maintaining our data infrastructure. You will work closely with cross-functional teams to ensure the efficient collection, storage, and processing of large datasets, enabling advanced analytics and AI applications. This position requires a strong technical background, excellent problem-solving skills, and a passion for leveraging data to drive business outcomes. Key Responsibilities : Data Pipeline Development : - Design, develop, and optimize scalable data pipelines for ETL processes. - Experience with Python/SQL - Implement data integration solutions to gather data from various sources, including APIs, databases, and external systems. - Ensure data quality, integrity, and consistency throughout the pipeline. Database Management : - Design and maintain robust data models and schemas to support business Requirements. - Manage and optimize database performance, ensuring efficient data storage and retrieval. - Implement data partitioning, indexing, and clustering strategies to enhance performance. Data Warehousing : - Build and maintain data warehouses to support analytical and reporting needs. - Develop and implement ETL processes to load data into the data warehouse from various sources. - Ensure data warehouse scalability and reliability. Collaboration and Communication : - Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. - Collaborate with software engineers to integrate data solutions into applications and services. - Communicate technical concepts and solutions effectively to both technical and non-technical stakeholders. Continuous Improvement : - Stay up-to-date with emerging technologies and industry best practices in data engineering. - Identify opportunities for process improvement and automation to enhance data pipeline efficiency. - Contribute to the development of data engineering (ref:hirist.tech)

Location: in, IN

Posted Date: 11/24/2024