Microsoft Certified: Azure Data Engineer Associate (DP-203)

Architected and maintained data products for renewable energy operations, covering wind turbine, grid-meter, and weather data. Built scalable ETL/ELT pipelines in Azure Databricks using Delta Lake (bronze/silver/gold layers) and processed data in various formats, including structured and semi-structured data. Contributed to a data quality framework supporting table and column documentation, outlier detection, and completeness metrics across all datasets within a data product. In addition, implemented a DORA KPI Databricks dashboard used across all data products. Optimized CI/CD processes in Azure DevOps to streamline deployment across development, test, and production environments.

Technology stack: Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Azure Data Lake, APIs, Dremio, Azure DevOps, YAML, Git, Databricks Workflows, Application Insights, Terraform, OpenAI API, Codex, LLM-assisted workflows

• Built and fine-tuned CNN and RNN architectures using transfer learning for real-world classification tasks — core deep learning skills applicable to BMW's multimodal LLM and GenAI vehicle function development.
• Implemented Dropout, Batch Normalisation, and Early Stopping across deep learning experiments; evaluated rigorously using precision, recall, F1-score, and confusion matrices for production-grade reliability.
• Developed an AI-powered attendance management system using LBPH facial recognition, deployed via Flask web interface with real-time SMS notifications — demonstrating end-to-end AI product delivery for real users.
• Collaborated across cross-functional teams to deliver scalable, documented ML pipelines designed for reproducibility — matching BMW's interdisciplinary team and research environment.
• Integrated AI tooling directly into the development workflow from design through to testing, maintaining high velocity without compromising correctness.

Tasks: Design and operation of end-to-end cloud data platforms for enterprise clients in publishing and finance, including infrastructure automation, pipeline development, monitoring, and data quality.

Activities:

• Built multi-layer data architectures on Databricks (Apache Spark, Delta Lake), BigQuery, and GCP
• Fully automated cloud infrastructure with Terraform across 3 environments (DEV/STG/PRD)
• Developed automated data pipelines with Python, dbt, and GCP services for different data sources
• Built monitoring and alerting systems for real-time platform monitoring
• Implemented data versioning and quality checks at every layer
• Designed automated test and deployment pipelines in GitLab and Bitbucket

Achievements:

• 2× production data processing capacity, reduced spike response time from minutes to ≤15 s, server errors ≈ 0
• Replaced 3,000 lines of manual configuration with a reusable automation module for 7 customer domains, configuration errors to 0
• Delivered a complete end-to-end data platform at ~€10/month infrastructure cost
• Migrated 7 database tables with 0 downstream issues
• Removed 100% exposed credentials, eliminated external vendor dependency
• Delivered integration of 3 teams in 1 sprint

• Consultancy Databricks Lakehouse Platform Architecture
• Migration of existing manufacturing data services to Databricks. Existing services are running on various platforms & tools and are unified on target platform
• Implementation of new Data & AI use cases on Databricks platform (e.g. connecting new systems, building AI Agent prototypes, …)

Tech Stack: Databricks, Azure, PySpark, Python

• Investigate the relationship between Eddy Current Testing (ECT) signals and microstructural properties
• Detect latent patterns in ECT data that reflect intrinsic material characteristics
• Develop and validate predictive models for microstructural classification and quantification, using hardness and case depth as benchmarks
• Apply Bayesian Structural Equation Modeling for advanced data analysis

• Advising on the use of analytics and BI tools and services in the Microsoft Azure stack (e.g. MS Fabric, Synapse Workspaces and dedicated SQL pools, SQL Database, PostgreSQL, Snowflake, Databricks, Data Factory, SSIS, Analysis Services, Function Apps, Power BI, ML)
• Independently designing analytics solutions with Python, SQL, etc.
• Designing and implementing ETLs and data pipelines
• Creating and maintaining APIs
• Independently applying CI/CD, testing, and version control
• Data modeling
• Model development and optimization
• Anomaly detection with AI
• Predictive analytics

Used technologies:

• Snowflake
• Fabric
• Azure Synapse Analytics
• Azure DataFactory
• Azure Data Lake
• Azure DevOps
• Databricks
• Spark
• CI/CD
• SQL Database
• Python
• Power Platform

• AI-powered automation with a focus on efficiency, information processing, and assistant systems
• Automated email classification (OpenAI, FastAPI)
• Contract analysis for LegalTech (Llama 3, LangGraph)
• Internal knowledge search with RAG (VLLM, Hugging Face)
• Anomaly detection on edge devices (LLAVA, TensorRT)
• Agent system for management reports (LangGraph, Zapier)

• Lead Architect to design and implement the data lake and ETL Pipeline using AWS Stack
• Performance Optimization of Data Ingestion of ETL Pipeline
• Development of Data Validation using Great Expectations
• Leading of the data migration for two sources exchanges
• Data Modeling in AWS Redshift

MLOps

• Model inference implementation by mlflow and AWS SageMaker
• Feature Engineering for the running ML Models ( Recommender Engineer, Clustering )
• Implementatino of Model Registry and artifactory using mlflow
• Historization an Profiling of the Input Data Using AWS Glue Crawler and AWS Data Catalog
• Feature importance using mlflow

Tech. Stack: Python 3, AWS Glue, AWS Step Fucntion, AWS Lambda, AWS EventBridge, AWS IAM Role, AWS SageMaker, AWS EC2, AWS Glue Crawler, AWS CloudWatch, MLFlow, ETL, Data lake, GitHub Action, Terraform, Jenkins, Ansible playbooks (Infrastructure as Code), CI/CD, GitLab, SQL, PySparkSCRUM, Agile, Jira, BigData, VSCode, DBeaver, MSSQL, MySQL, grafana, Docker, Linux, Bash, MapReduce, Data Modeling (ORM), Pandas, YAML, SQL-Alchemy