Unlock Business Consulting

Data Engineer - R/ETL/Python

Click Here to Apply

Job Location

delhi, India

Job Description

Responsibilities : - Responsible for designing, building, and maintaining the infrastructure, systems, processes needed to handle large volumes of data and work with various tools and technologies to develop solutions that enable our organization to process, analyse, and derive insights from massive datasets. - Collaborate with cross-functional teams to identify data needs, design solutions, and enhance data driven decision-making capabilities. - Maintain and ensure the integrity and reliability of data pipelines, implementing best practices in data security and compliance. - Execute end-to-end data engineering projects, from conceptualization to deployment, demonstrating project management and technical prowess. - Set the standard for data engineering practices within the company, guiding the architectural approaches, data pipeline designs, and the integration of cutting-edge technologies to foster a culture of innovation and continuous improvement. - Spearhead the discovery, evaluation, and integration of new datasets, collaborating (incl. pipeline development and data modelling/documentation), and working closely with key data stakeholders to understand their impact and relevance to our core products and the business domain. Skills & Qualifications : - Experience in Data engineering, Data modelling, Structured/Non-Structured Data, ETL, Big database, Data warehouse, Data Lake & administration of data infrastructure, build highly reliable & scalable data systems with good data governance that provide complete, secure, accurate and timely data for analytics. - Hands on administration of big data databases/compute engines (Data Lake, Redshift, EMR, Spark or similar) and optimization/tuning technique - Design, Development & implementation of OLTP & OLAP databases with efficient design, optimization techniques & data ingestion, aggregation, labelling, anonymization, archival, retrival mechanisms. - Proficient in Python, SQL and PL-SQL, query tuning, optimization, ETL, ELT, Data Modelling and Data Warehouse systems. - Strong Proficiency With AWS Technologies Like S3, EC2, Lambda, Batch, CloudWatch, Redshift, RDS, DynamoDB, DMS, Data Lake, Glue, Athena, Kinesis. - Experience in building pipelines, ingesting data, data orchestration from heterogeneous systems onto data lake environments in real time using tools like Airbyte, AirFlow, Trino, Minio, Hive etc. - Proficient in a variety of big-data tools and technologies, including Hadoop, Spark, etc., handling different storage formats like Parquet, Delta/Apache Hudi/Apache Iceberg - Employ advanced SQL, Python, R, Spark, and Bash scripting to optimize data processing, analysis, and automation tasks. Advanced SQL, Python and associated libraries like Pandas, Numpy etc., Pyspark , Shell scripting, Data-Modelling, Big data, Hadoop, Hive, ETL pipelines. - Experience with data visualization tools and analytics technologies (Meta, Power BI, Looker, Tableau, etc.) - Proficient in Business Intelligence (BI), analytic database and products, able to take business requirements and translate into database tables, pipeline design and tasks. - Experience with technology stacks available for Metadata Management, Data Governance, Data Quality, MDM, Data Catalog etc. - Proficient in integrating data from and to diverse platforms using API's - Experience with building data pipelines in streaming and batch mode. - Experience with Git and CI/CD pipelines to deploy cloud applications (ref:hirist.tech)

Location: delhi, IN

Posted Date: 2/23/2025
Click Here to Apply
View More Unlock Business Consulting Jobs

Contact Information

Contact Human Resources
Unlock Business Consulting

Posted

February 23, 2025
UID: 5060973582

AboutJobs.com does not guarantee the validity or accuracy of the job information posted in this database. It is the job seeker's responsibility to independently review all posting companies, contracts and job offers.