Data Engineer - Databricks-Consultant II

                  Databricks Data Engineer– Job description

Job Summary:

We are seeking a highly skilled and experienced Databricks Developer to join our data engineering team. The ideal candidate will have over 4 years of experience working with Databricks on Azure, along with a strong background in data pipelines, and performance optimization. The candidate will be responsible for developing scalable data processing solutions, ensuring data quality, and enabling advanced analytics initiatives.

Key Responsibilities:

• Design, develop, and optimize data pipelines and ETL processes using Azure Databricks (PySpark /Spark SQL).

• Collaborate with data architects, analysts, and other developers to deliver data solutions aligned with business requirements.

• Perform data wrangling, cleansing, transformation, and aggregation from multiple sources.

• Implement and maintain data lake and data warehouse solutions using Azure services (ADLS, Synapse, Delta Lake).

• Monitor pipeline performance, troubleshoot issues, and ensure data integrity and reliability.

• Use Delta Lake for building robust and scalable data architectures.

• Develop and maintain CI/CD pipelines for Databricks workflows and jobs.

• Participate in code reviews, unit testing, and documentation of data processes.

Required Skills & Experience:

• 4+ years of hands-on experience with Databricks on Azure.

• Strong expertise in PySpark, Spark SQL, and Delta Lake.

• Solid understanding of Azure Data Services: Azure Data Lake Storage (ADLS), Azure Data Factory (ADF)

• Proficiency in Python for data processing tasks.

• Experience with data ingestion from various sources (on-prem, cloud, APIs).

• Knowledge of data modeling, data governance, and performance tuning.

• Familiarity with CI/CD tools (Azure DevOps, Git) and job orchestration in Databricks.

• Strong problem-solving skills and ability to work independently or as part of a team.

What we offer:

• EXL Analytics offers an exciting, fast paced and innovative environment, which brings together a group of sharp and entrepreneurial professionals who are eager to influence business decisions. From your very first day, you get an opportunity to work closely with highly experienced, world class analytics consultants.

• You can expect to learn many aspects of businesses that our clients engage in. You will also learn effective teamwork and time-management skills - key aspects for personal and professional growth
• Analytics requires different skill sets at different levels within the organization. At EXL Analytics, we invest heavily in training you in all aspects of analytics as well as in leading analytical tools and techniques.

• We provide guidance/ coaching to every employee through our mentoring program wherein every junior level employee is assigned a senior level professional as advisors.
• Sky is the limit for our team members. The unique experiences gathered at EXL Analytics sets the stage for further growth and development in our company and beyond.