Enterprise Data Engineer · Azure & Databricks · AI-Ready Data Platforms
About
I'm Daniel Conejo Sobrino. I build enterprise-grade data platforms that bridge the gap between raw data and AI-ready insights. My work spans the full data engineering lifecycle—ingesting multi-source pipelines into Azure Data Lake Storage, curating Delta Lake medallion architectures on Databricks, and delivering curated datasets that power downstream machine learning and analytics workloads.
My current focus is designing governance-first data platforms: implementing Unity Catalog for fine-grained access control, defining data contracts between producing and consuming teams, and embedding data quality frameworks that make pipelines reliable enough for production AI workloads.
Alongside platform engineering, I research the governance implications of agentic AI systems—how autonomous agents interact with enterprise data, the audit trails they require, and the policy frameworks organisations need to keep them accountable at scale.
Experience
Viewnext · IBM Group
Data Engineer
2024 – Present
Málaga, Spain · Full Remote
- Work as a Data Engineer at Viewnext, part of IBM, assigned full-time to Repsol as client on the ARiA data platform.
- Implement end-to-end data pipelines across the full data lifecycle, from source extraction and ingestion to curated analytical layers in Azure Data Lake and downstream platform consumption.
- Collaborate with Solution Architects to translate architecture designs into robust, scalable technical implementations, including scope documentation, feasibility validation and test planning.
- Develop and configure batch and streaming ingestion processes across heterogeneous sources such as Oracle, Teradata, SQL Server, APIs, Salesforce, sFTP file sources, Event Hubs and industrial PI System signals.
- Build data transformation and quality workflows using Python, PySpark, SQL, Azure Databricks, Azure Data Factory and platform components for business logic, joins, aggregations, schema validation and semantic normalization.
- Support data modelling across Raw, Processed and Analytical layers, including curated datasets and Synapse Dedicated Pool models optimized for analytical consumption.
- Contribute to SDLC delivery across Development, Test, Acceptance and Production environments, supporting UAT, production readiness and post-release stability.
- Integrate data processes with observability, monitoring and orchestration capabilities to track ingestion health, dependencies and SLA compliance.
- Work with governance and security processes including metadata cataloguing, access requests, ACLs, Azure Key Vault and Service Principals.
- Participate in Agile delivery using ServiceNow and Azure DevOps, including backlog management, CI/CD pipelines, Git workflows and delivery KPIs.
BeoneBe
Data & Cloud Engineer
January 2024 – July 2024
Marbella, Spain
- Built an Azure-backed document intelligence pipeline for automated metadata extraction and authenticity verification across large document corpora
- Deployed processing services in Docker on Linux with TLS/SSL termination and reverse-proxy routing
- Designed REST APIs exposing data pipeline outputs for downstream consumer applications
- Established CI/CD pipelines for automated testing and deployment of data processing services
Education
Universidad Internacional de La Rioja (UNIR)
Higher Degree in Networked Computer Systems Administration
CPIFP Alan Turing, Parque Tecnológico de Andalucía (PTA)
Professional Certificate in Frontend Environment, Professional Certificate Microsoft Azure
ILERNA FP
Higher Degree in Web Applications Development
Research Notes
Notes on agentic AI governance, enterprise data architecture, and platform engineering.