Azimuth GRC - Senior Databricks Developer - Big Data Technologies

Job Description : - Conceptualize and communicate Data architecture strategy, technical design and technology roadmap for data platform solutions and services. - Lead design and development of data architecture to support implementation of large-scale data solutions in Databricks to support multiple use cases (delta lake, cloud Datawarehouse migrations, reporting and analytics). - Guiding the organization and development teams on the overall data processes, architectural approaches for data strategy design and implementation. - Providing data solutions that enable business intelligence data integration, data services, self-service analytics, and data-driven digital products and services. - Articulating the value proposition of cloud modernization/transformation to stakeholders; creating detailed documentation that empowers other engineers and customers. - Architecting the solutions for Databricks-specific use cases (streaming and batch) as well as integrating it into other solution components in the customer architecture landscape. - Translating advanced business data, integration and analytics problems into technical approaches that yield actionable recommendations, across multiple, diverse domains. - Implement architectural improvements for existing solutions using legacy modernization and cloud adoptions strategies. - Creating Azure Data Factory processes for SQL and NoSQL data pipelines, and working in Azure SQL, to create tables, views, stored procedures, and other database objects. - Creating applications through Scala, Python, and Java. - Developing Spark tasks for data aggregation and transformation - Writing Scala style documents with each code. - Identifying technical issues (performance bottlenecks, data discrepancies, system defects) in data and software, perform root cause analysis, and communicate results effectively to the development team and other stakeholders. - Providing recommendations on opportunities and improvements and participating in technology and analytics proof-of-concepts - Managing a team Desired Profile : - 8 to 10 years of experience working with Databricks, Apache Spark, and big data technologies. - 8years of experience with Azure Databricks, Azure Data Factory, and Pyspark - Highly proficient in use of Spark and Azure Databricks - 8 years' experience working in Microsoft Azure - Proven experience in designing and implementing data solutions and ETL pipelines. - Strong understanding of data architecture, data modeling, and data warehousing concepts. - Proficiency in Databricks and Spark, including cluster management, job scheduling, and performance tuning. - Experience with cloud platforms such as AWS, Azure, or Google Cloud. - Hands-on working knowledge of topics such as data security, messaging patterns, ELT, Data wrangling and cloud computing and proficiency in data integration/EAI and DB technologies, sophisticated analytics tools, programming languages or visualization platforms - Knowledge of the Databricks ecosystem - Team Management Experience - Strong programming skills in languages such as Python, Scala, or SQL. - Experience with data integration tools and technologies (e.g., Kafka, Delta Lake). - Knowledge of data governance, security, and compliance best practices - Bachelor's degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees or certifications are a plus (ref:hirist.tech)

Location: in, IN

Posted Date: 11/24/2024