Data Engineering Lead

Mode of work : Hybrid – Mandatory Location: first preference Bangalore Timing: 3PM to 12AM Data Engineer Lead Skill Set: 10+ years in Data Engineering. 5+ years of hands-on experience in Azure Databricks/PySpark. Experience with Microsoft Fabric. Strong SQL optimization expertise. Experience leading teams, architecture discussions, and enterprise data transformation programs. Exposure to DevOps, governance, security, and cloud cost optimization. Hands-on coding is mandatory for this role. The Senior Data Engineering Lead must actively contribute to development, write and optimize production-grade code, troubleshoot complex technical issues, and lead by example rather than focusing solely on team management or architecture. Data Engineering Leadership: Lead the design, development, and delivery of enterprise-scale data engineering solutions, mentoring data engineers and driving technical excellence. Advanced Databricks Development: Strong hands-on expertise in Azure Databricks, PySpark, Spark SQL, Delta Lake, and Medallion Architecture. ETL/ELT Design and Implementation: Design, develop, and optimize scalable ETL/ELT pipelines, incremental data ingestion, CDC, and batch/streaming data processing. SQL Development and Optimization: Expert-level SQL skills, including complex queries, stored procedures, execution plan analysis, indexing, partitioning, and query performance tuning. Azure Data Platform Expertise: Strong experience with Azure Data Factory, Azure Data Lake Storage Gen2, Azure Databricks, and related Azure data services. Microsoft Fabric: Good hands-on experience with Microsoft Fabric, including OneLake, Lakehouse, Warehouse, Data Factory, and Spark-based data engineering capabilities. Data Modeling and Architecture: Expertise in data warehousing, dimensional modeling, star/snowflake schemas, lakehouse architecture, and data quality management. Performance and Cost Optimization: Identify and resolve data processing bottlenecks, optimize Spark workloads, and improve pipeline performance, reliability, and cloud cost efficiency. Technical Design and Team Mentoring: Lead technical design discussions, architecture reviews, code reviews, engineering best practices, and knowledge-sharing initiatives. Enterprise Integration and DevOps: Experience delivering enterprise-scale data integration and analytics solutions, implementing CI/CD, automated testing, monitoring, governance, security, and production Kindly share your updated resume to tabbu@sandboxprofessionals.com