DataOps Consultant, Lakehouse Support

About the role

We are looking for a DataOps Consultant to provide production support and DevOps engineering for our data lakehouse ecosystem. The role combines incident management, automation and AI‑driven diagnostics to keep data pipelines and applications reliable.

Key responsibilities

• Perform deep diagnostics on batch and streaming data pipelines and coordinate resolution with platform engineers, Data Engineers and domain teams (Databricks, Snowflake, Palantir).
• Own L2/L3 incident triage, root‑cause analysis and post‑incident reviews for .NET/C#, Python applications, APIs and event‑driven systems.
• Maintain CI/CD pipelines in Azure DevOps and infrastructure as code using Terraform, Bicep or ARM templates.
• Implement monitoring and observability with Dynatrace, Azure Monitor, OpenTelemetry and data‑observability tools such as Monte Carlo.
• Automate runbooks and diagnostics using AI tools, Python and PowerShell scripts.

Required profile

• Strong experience in production support, incident management and ITIL‑aligned processes.
• Hands‑on expertise with Azure cloud services (App Services, AKS, Functions, Event Hubs, Storage, Key Vault, Azure SQL, Azure Monitor).
• Proficiency in CI/CD (Azure DevOps), IaC (Terraform/Bicep/ARM) and container orchestration (Docker, Kubernetes/AKS).
• Solid knowledge of monitoring stacks (Dynatrace, Azure Monitor) and distributed tracing (OpenTelemetry).
• Experience with data platforms (Databricks, Palantir Foundry, Spark, PySpark) and scheduling/orchestration tools (StoneBranch, Airflow).

Required skills

• .NET/C#
• Python
• Azure DevOps
• Terraform
• Bicep
• ARM templates
• Docker
• Kubernetes / AKS
• Dynatrace
• ServiceNow
• OpenTelemetry
• Monte Carlo
• Palantir Foundry
• Databricks
• Spark / PySpark
• SQL Server
• MongoDB
• PowerShell
• GitHub Copilot