The Family Office Company BSC - Senior Data Engineer - Data Pipeline

Job Summary : - Responsible for designing, developing, and maintaining data pipelines, integrating APIs, and managing data-driven solutions within our organization. - Collaborating with cross-functional teams to ensure data availability, quality, and usability for analysis and business insights. - Proficiency in Python programming and a strong willingness to learn and grow in a dynamic environment. - Sense for ownership and drive for improvements. Responsibilities : - Design, develop, and optimize data pipelines and ETL processes using Azure Data Factory (ADF) and Azure Databricks to extract, transform, and load data from various sources into data platforms. - Integrate with external APIs and build data connectors to retrieve and ingest data from third-party systems, ensuring data accuracy and reliability. - Preparing SQL views and procedures for other teams in the company. - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and ensure the availability of relevant datasets for analysis and reporting purposes. - Implement and enforce data governance practices, including data quality, data privacy, and data security measures, following industry best practices and regulatory requirements. - Troubleshoot and resolve data-related issues, performance bottlenecks, and data pipeline failures, ensuring minimal disruption to data workflows and business operations. - Continuously monitor and optimize data processing and storage infrastructure for improved performance, scalability, and cost efficiency. - Ownership of various projects across the engineering team and drive to delivery in time. - Stay updated with the latest trends, technologies, and best practices in data engineering, Azure services, APIs, and data management to propose and implement innovative solutions. Qualifications : Work experience : - Proven work experience as a Data Engineer or a similar role, focusing on data pipeline development, API integration, database design, and data management. - Strong expertise in Azure cloud services, particularly Azure Data Factory (ADF), Azure SQL Database, Azure Blob Storage, Azure Databricks. - Proficient in SQL and Python programming and experience working with relational databases, data warehousing, and data modeling concepts. - Experience with API integration, including RESTful APIs, data extraction, and data ingestion from external systems. - Solid understanding of ETL processes, data integration, and data transformation techniques. - Proficiency in Python programming and experience with data manipulation, automation, and scripting tasks. - Strong problem-solving and analytical skills, with the ability to identify and resolve complex data-related issues. - Experience with leadership and guidance of small team. - Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams. - A keen desire to learn and stay updated with emerging technologies, tools, and methodologies in the data engineering field. Education : - Minimum of bachelor's degree from a recognized university or equivalent experience. Skills & Competencies Required : - Creative thinking. - Desire to work in an information system environment. - Excellent communication (written and oral) and interpersonal skills. - Excellent organizational, multi-tasking, and time-management skills. Interpersonal skills : - Proven ability to build, manage, and foster a team-oriented environment. - Proven ability to work creatively and analytically in a problem-solving environment. - Proven ability to work independently. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 12/28/2024