Microsoft Certified: Azure Data Engineer Associate (DP-203)

Architected and maintained data products for renewable energy operations, covering wind turbine, grid-meter, and weather data. Built scalable ETL/ELT pipelines in Azure Databricks using Delta Lake (bronze/silver/gold layers) and processed data in various formats, including structured and semi-structured data. Contributed to a data quality framework supporting table and column documentation, outlier detection, and completeness metrics across all datasets within a data product. In addition, implemented a DORA KPI Databricks dashboard used across all data products. Optimized CI/CD processes in Azure DevOps to streamline deployment across development, test, and production environments.

Technology stack: Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Azure Data Lake, APIs, Dremio, Azure DevOps, YAML, Git, Databricks Workflows, Application Insights, Terraform, OpenAI API, Codex, LLM-assisted workflows

Tasks: Design and operation of end-to-end cloud data platforms for enterprise clients in publishing and finance, including infrastructure automation, pipeline development, monitoring, and data quality.

Activities:

• Built multi-layer data architectures on Databricks (Apache Spark, Delta Lake), BigQuery, and GCP
• Fully automated cloud infrastructure with Terraform across 3 environments (DEV/STG/PRD)
• Developed automated data pipelines with Python, dbt, and GCP services for different data sources
• Built monitoring and alerting systems for real-time platform monitoring
• Implemented data versioning and quality checks at every layer
• Designed automated test and deployment pipelines in GitLab and Bitbucket

Achievements:

• 2× production data processing capacity, reduced spike response time from minutes to ≤15 s, server errors ≈ 0
• Replaced 3,000 lines of manual configuration with a reusable automation module for 7 customer domains, configuration errors to 0
• Delivered a complete end-to-end data platform at ~€10/month infrastructure cost
• Migrated 7 database tables with 0 downstream issues
• Removed 100% exposed credentials, eliminated external vendor dependency
• Delivered integration of 3 teams in 1 sprint

• Consultancy Databricks Lakehouse Platform Architecture
• Migration of existing manufacturing data services to Databricks. Existing services are running on various platforms & tools and are unified on target platform
• Implementation of new Data & AI use cases on Databricks platform (e.g. connecting new systems, building AI Agent prototypes, …)

Tech Stack: Databricks, Azure, PySpark, Python

• Investigate the relationship between Eddy Current Testing (ECT) signals and microstructural properties
• Detect latent patterns in ECT data that reflect intrinsic material characteristics
• Develop and validate predictive models for microstructural classification and quantification, using hardness and case depth as benchmarks
• Apply Bayesian Structural Equation Modeling for advanced data analysis

• Advising on the use of analytics and BI tools and services in the Microsoft Azure stack (e.g. MS Fabric, Synapse Workspaces and dedicated SQL pools, SQL Database, PostgreSQL, Snowflake, Databricks, Data Factory, SSIS, Analysis Services, Function Apps, Power BI, ML)
• Independently designing analytics solutions with Python, SQL, etc.
• Designing and implementing ETLs and data pipelines
• Creating and maintaining APIs
• Independently applying CI/CD, testing, and version control
• Data modeling
• Model development and optimization
• Anomaly detection with AI
• Predictive analytics

Used technologies:

• Snowflake
• Fabric
• Azure Synapse Analytics
• Azure DataFactory
• Azure Data Lake
• Azure DevOps
• Databricks
• Spark
• CI/CD
• SQL Database
• Python
• Power Platform

• AI-powered automation with a focus on efficiency, information processing, and assistant systems
• Automated email classification (OpenAI, FastAPI)
• Contract analysis for LegalTech (Llama 3, LangGraph)
• Internal knowledge search with RAG (VLLM, Hugging Face)
• Anomaly detection on edge devices (LLAVA, TensorRT)
• Agent system for management reports (LangGraph, Zapier)

• Lead Architect to design and implement the data lake and ETL Pipeline using AWS Stack
• Performance Optimization of Data Ingestion of ETL Pipeline
• Development of Data Validation using Great Expectations
• Leading of the data migration for two sources exchanges
• Data Modeling in AWS Redshift

MLOps

• Model inference implementation by mlflow and AWS SageMaker
• Feature Engineering for the running ML Models ( Recommender Engineer, Clustering )
• Implementatino of Model Registry and artifactory using mlflow
• Historization an Profiling of the Input Data Using AWS Glue Crawler and AWS Data Catalog
• Feature importance using mlflow

Tech. Stack: Python 3, AWS Glue, AWS Step Fucntion, AWS Lambda, AWS EventBridge, AWS IAM Role, AWS SageMaker, AWS EC2, AWS Glue Crawler, AWS CloudWatch, MLFlow, ETL, Data lake, GitHub Action, Terraform, Jenkins, Ansible playbooks (Infrastructure as Code), CI/CD, GitLab, SQL, PySparkSCRUM, Agile, Jira, BigData, VSCode, DBeaver, MSSQL, MySQL, grafana, Docker, Linux, Bash, MapReduce, Data Modeling (ORM), Pandas, YAML, SQL-Alchemy

Projects:

Project: RWE – Development of a company-wide Zero Trust cybersecurity architecture (CITADEL) Role: Senior Enterprise Cybersecurity Architect / Zero Trust Architect Company: RWE AG Description: Concept and implementation of the strategic CITADEL cybersecurity target architecture at RWE, based on the Zero Trust architecture principle and aligned with regulatory requirements such as NIS2, ISO 27001 and company-wide security governance policies. The goal was to build a measurable, auditable and scalable security architecture with a strong focus on Identity Governance, compliance transparency and operational manageability. Responsibilities & Achievements:

• Zero Trust architecture design: Developed a company-wide Zero Trust reference architecture (Identity, Device, Network, Application, Data) including trust zones, control points and enforcement mechanisms according to NIS2.
• Identity & Access Governance (IGA): Designed and introduced IGA governance structures including role models, recertification processes, segregation of duties (SoD) and lifecycle management for identities and access.
• Security governance & KPIs: Defined and implemented security KPIs and metrics to manage Zero Trust maturity, identity risks and compliance at the management level.
• Compliance & reporting: Built standardized compliance reports and dashboards to support internal audits, external assessments and regulatory evidence (e.g. NIS2).
• Architecture & stakeholder alignment: Worked closely with Enterprise Architecture, IT operations and business units to integrate the CITADEL architecture into existing IT and security landscapes.
• Strategic security consulting: Advised programs and projects on Zero Trust compliance, identity centricity and regulatory requirements in the energy and critical infrastructure (KRITIS) environment. Technologies & Methods: Zero Trust Architecture, NIS2, Identity Governance & Administration (IGA), IAM, RBAC, SoD, Entra ID, SailPoint, Zscaler, Terraform / IaC, Policy as Code, security KPIs, compliance reporting, NIST 2.0, ISO 27001, Enterprise Security Architecture, governance frameworks, risk & control management

Project: Scalable AI Workbench Platform on Microsoft Azure Role: Cloud Architect & Engineer Company: Siemens Energy Description: Design, development and operation of a secure, modular cloud infrastructure to support Data Science, Machine Learning and AI applications for various engineering teams at Siemens Energy. Responsibilities & Achievements:

• Cloud architecture: Designed and implemented an Infrastructure-as-Code solution (Terraform) for automated provisioning of Azure resources (Resource Groups, Storage Accounts, Cosmos DB, Application Insights, networking, PostgreSQL Flexible Server, Azure Container Apps, Azure Container Registry).
• Developer portal: Used Backstage with custom frontend and backend plugins (Node.js, TypeScript, React.js, PostgreSQL, Container Apps) to enable self-service and empower developers, data scientists and AI/ML engineers.
• Role-based access control: Implemented Azure RBAC to grant targeted access (e.g. Storage Blob Data Contributor, Reader) to engineering groups (e.g. AI Engineers) for relevant resources.
• Data platform engineering: Built and configured a multi-layered storage landscape (Raw, Curated, Vector data), including automated container creation and access control for advanced analytics and AI workloads.
• DevOps integration: Integrated with Azure DevOps for CI/CD pipelines to automate deployment, monitoring and compliance.
• Security & compliance: Implemented Private Endpoints, network policies and Managed Identities to ensure data protection and regulatory compliance.
• Collaboration: Worked closely with cross-functional teams to align the cloud infrastructure with business and technical requirements and drive digital transformation at Siemens Energy. Technologies: Azure, Terraform, Azure DevOps, Cosmos DB, Application Insights, Azure Storage, Private Endpoints, Azure Synapse, Azure Machine Learning, Azure Entra ID, RBAC, Backstage, Node.js, React.js, PostgreSQL, Python (automation), Git