Job Title: Senior Azure Data Engineer

Location: Bangalore, India

Experience: Minimum 5 years

About the Role

We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our dynamic team in Bangalore. As a Senior Azure Data Engineer, you will be responsible for the design, implementation, and optimization of data systems and solutions within the Microsoft Azure ecosystem. You will work closely with data analysts, and other engineers to build and maintain robust data pipelines and ensure seamless data integration for our business intelligence and analytics needs.

Key Responsibilities

• Design, develop, and optimize scalable, high-performance data pipelines using Azure Data Factory (ADF) and Azure Databricks (ADB).
• Lead end-to-end data migration projects, including the movement of data between on- premises systems and Azure cloud services.
• Architect and implement data warehousing solutions, leveraging Azure Synapse Analytics, Azure SQL Data Warehouse, and Azure Data Lake for efficient data storage and analysis.
• Develop and implement ETL/ELT processes using Azure services to transform raw data into structured, consumable formats.
• Optimize data workflows and pipelines for performance, scalability, and cost-effectiveness, ensuring timely delivery of accurate, actionable data.
• Ensure the security, quality, and integrity of data through implementation of best practices in data governance and validation.
• Manage and optimize large datasets, ensuring effective storage, retrieval, and processing through Azure Data Lake and other Azure storage options.
• Collaborate with business stakeholders to understand requirements, define data solutions, and provide timely data-driven insights.
• Provide guidance and mentorship to junior data engineers, promoting best practices and continuous learning.
• Troubleshoot complex data processing workflows, resolve issues, and ensure high system uptime and performance.
• Stay current with the latest trends in Azure technologies and data engineering best practices, continuously improving and optimizing solutions.

Required Qualifications

• Minimum 5 years of Data Engineering experience, including 3+ years with ADF, ADB, and data migration.
• Strong experience in data warehousing, Azure Synapse Analytics, and Azure Data Lake.
• Proficient in developing ETL/ELT pipelines using Azure services, and SQL for large datasets.
• Hands-on experience in migrating data to Azure cloud platforms.
• Expertise in programming languages like Python, Scala, or Spark.
• Strong understanding of data governance and security within Azure.

Preferred Qualifications

• Azure Data Engineer or Azure Databricks Certified.
• Experience with DevOps practices, CI/CD pipelines, and tools like Azure Logic Apps.
• Knowledge of Docker, Kubernetes, and advanced analytics tools like Power BI.