Databricks Experts in Munich

• This project involved extraction of intelligence data to support report writing for a company that provides geopolitical, global, commercial intelligence. The data have been gathered from a number of resources (interview transcripts, online data, internal documents), and then a knowledge base has been build from it. This was the basis of a complex RAG system, that was evaluated against a golden dataset. Agents have been used to find out the contradicting intelligence, the statements supporting each other, and to store back the generated knowledge.

Used: Python, RAG, LangGraph, LangChain, deepeval, MCP

• Building, implementing and selling automated Data Science solutions such as Scorecard Factory and Forecast Factory
• Implementation of automated end-to-end cloud processes
• Development of LLM and NLP models
• Creation of interactive reports
• Support for national and international large corporations as well as medium-sized companies in implementing ML projects

In my role as Senior Cloud DevOps Architect for BP, an oil and gas company, I had the mission to migrate the Electric Vehicle Charging platform of the EV Division from on-premises and Azure to AWS cloud, resulting in a hybrid multi-cloud, multi-tenant SaaS solution.

Deployment with Kubernetes for the application layer meant provisioning Kubernetes clusters managed by EKS and AKS, with a focus on integrating them into a multi-tenant environment. This integration was achieved by using Kubernetes namespaces and access controls to ensure data isolation and privacy enforcement.

In the database layer, we chose an RDS instance with PostgreSQL to support the backend infrastructure of our applications. Tenants shared the same RDS instance, but each had a dedicated schema.

To ingest near real-time data from physical charge points (CPOs), as IoT devices, via the OCPI protocol, we ran into significant delays with batch processing. As a result, we built a real-time streaming data pipeline using Apache Kafka, while prioritizing an event-driven architecture.

Led collaboration across multiple internal teams, external vendors, cloud providers, and on-site partners to integrate over five systems into a unified solution.

Achievements:

• Successfully designed and implemented hybrid multi-cloud solutions, integrating multiple cloud platforms (AWS, Azure) with on-premises infrastructure, using Site-to-Site VPNs, Firewalls, and Load Balancing.
• Led the migration of on-premises infrastructure to multi-cloud, multi-tenant infrastructure, resulting in 30% faster processing times.
• Migrated workloads from VMware and Hyper-V environments to cloud-based VMs, leveraging cloud-native services to optimize performance, cost efficiency, and scalability.
• Designed a multi-tenant Kubernetes platform leveraging the Kubernetes ecosystem, using Karpenter for dynamic EC2 node provisioning, KEDA for event-driven pod autoscaling (e.g., Kafka message lag), and Rancher for centralized monitoring of multiple clusters (EKS, AKS, or on-prem K8s), replacing Microsoft-centric Azure Arc management service.
• Designed and implemented Python-based FastAPI microservices as part of the EV core-backend on AWS EKS application layer, powering data ingestion and customer analytics pipelines.
• Developed asynchronous, event-driven APIs (Python-FastAPI) for real-time integration with CPOs, supporting OCPI 2.3 and OICP protocols.
• Designed and implemented a secure, production-grade Azure Databricks platform using Terraform, ensuring scalability and cost efficiency.
• Migrated on-premises ERP to a hybrid Dynamics 365 architecture with ERP hosted locally and CRM running in Azure, integrated via Azure Arc.
• Automated CI/CD pipelines for Databricks notebooks and jobs using GitHub Actions & Databricks CLI, reducing deployment time. Reduced infrastructure provisioning time by 70% by automating cloud resource deployment with GitOps.
• Ensured compliance with internal audit and data governance standards (GDPR) through OAuth2/OIDC-based authentication and fine-grained role-based access controls.
• Developed a Zero Trust security model, enforcing least-privilege access and microsegmentation, enhancing security posture and compliance with GDPR and NIST.
• Built interactive analytics dashboards in Amazon QuickSight, integrating data from S3 and Redshift to deliver real-time business insights and visualizations with embedded access for multi-tenant users.
• Led cloud security assessments and full-lifecycle cybersecurity integration during M&A, covering AWS, Azure, IAM (Entra ID), and data protection, while aligning security posture with NIST, ISO 27001, and GDPR across hybrid and cloud-native environments.
• Reduced cloud costs by 64% for a client's dev environment by implementing automated start/stop schedules for EC2 and RDS instances via AWS CDK with EventBridge Scheduler or AWS Systems Manager.

Tech stack:

• Infrastructure as Code: Terraform, AWS CDK, Ansible.
• Containers: Kubernetes on EKS, AKS, Docker.
• Streaming Data Processing: Kafka to Confluent Cloud, after AWS MSK.
• Frontend: TypeScript, React, NextJS, Hooks, Styled Components.
• Backend: Python with FastAPI, also Node.js with NestJS.
• Database: Aurora on PostgreSQL with TypeORM, RDS on SQL Server, Azure Databricks full setup and administration, ETL Pipelines.
• CI/CD and GitOps: GitHub Actions, Azure DevOps, ArgoCD.
• Monitoring and Observability: Prometheus and Grafana.
• Virtualization: Hyper-V, VMware Cloud on AWS, Azure Migrate.
• ERP Systems: Odoo, Microsoft Dynamics 365 Business Central on Azure, integrated with Azure Arc.
• Networking: Site-to-Site VPNs, AWS Direct Connect, Azure ExpressRoute, Firewalls (AWS Network Firewall, Azure Firewall).
• Security: IAM, NIST Framework, Zero Trust Security, AWS WAF, AWS Shield, GuardDuty.

• Build and operate an end-to-end MLOps platform on Azure ML and Kubernetes (Kubeflow) for the automated deployment, monitoring, and scaling of forecasting models (including Temporal Fusion Transformer, Informer, Autoformer).
• Implement CI/CD pipelines in Azure DevOps for the full ML lifecycle – from resource provisioning (Terraform), data transformation (Hugging Face Datasets, Pandas, PyTorch, CUDA cluster) through training and evaluation to model registry and endpoint deployment.
• Integrate MLflow for experiment tracking, model versioning, performance monitoring, and automated registration in the Azure Model Registry.
• Develop and containerize PyTorch training jobs (Azure Notebook, Jupyter Notebooks) for price and time series forecasting (PFC models) with automatic rollout via Azure ML Endpoints and REST/gRPC interfaces, Docker containerization, secured with OAuth 2.0.
• Set up monitoring and alerting mechanisms (Prometheus, MLflow Metrics), log centralization, and cost monitoring.
• Automate infrastructure provisioning and model deployment using Terraform, Helm, and Azure CLI; connect to existing market data systems and event pipelines.
• Migrate existing workloads and databases (IONOS → Azure, MongoDB) with integration into central MLOps workflows and internal networks.
• Extend the platform with LLM-based tools (LangChain, LangServe) to integrate GPT-based analysis modules into existing Spring Boot services for market anomaly detection and automated reports.
• Analyze and architect a software solution to process large volumes of data efficiently (>3000 messages/sec.) (market data store).
• Spring Boot / Java 21 container development with RabbitMQ for distributing stock market data via MongoDB (Kubernetes) with fast storage of data in Redis RMaps, deduplication, forwarding messages to Read Model queues, and building Read Models for UI display in MongoDB.
• Integration of RESTHeart to create a REST API for MongoDB.
• Build an Angular frontend to simplify data queries and master data maintenance.
• Agentic coding with remote and local LLMs (Claude Sonnet, Ollama Qwen) and MCP servers.
• Develop Python scripts for transforming and cleaning incoming stock market data (Pandas, scikit-learn).

• Developed & maintained customer analytical reporting solutions to identify revenue trends, performance drivers, risks & optimization opportunities to ensure data driven decision making across Sales, Finance, Product, Data Engineering & Controlling.
• Defined & analyzed customer trends & performance metrics to identify root causes behind variances, anomalies & emerging risks across business domains to deliver actionable recommendations.
• Developed & owned analytical data models & reporting layers to ensure scalability, performance & analytical robustness to support executive & operational reporting across business domains.
• Planned, tracked & executed projects by ensuring adherence to timelines, data accuracy, consistency, deliverables, reliability & data quality standards through rigorous validation & reconciliation processes.
• Raised the analytical maturity by formalizing analytical workflows, documenting data processes & standard operating procedures (SOPs) & conducting training sessions to drive adoption of self-service analytics & embed a data driven culture across operational and business teams.
• Took ownership of the end-to-end lifecycle roadmap from requirement gathering, collection, transformation, developing robust business logics to data storytelling & stakeholder delivery.
• Converted complexity into structured clarity by translating requirements & business processes into analytical recommendations to ensure alignment between non-technical & technical stakeholders.
• Conducted advanced SQL based analysis of complex business datasets to uncover trends, correlations & performance improvement opportunities.
• Drove process automation & efficiency improvements by leveraging Python, SQL optimization & AI assisted tools to reduce processing time & increase reliability across analytical & operational workflows.
• Standardized KPI definitions & reporting logic to ensure consistency & trust across reporting solutions.
• Developed process monitoring dashboards & analyses to identify inefficiencies, bottlenecks & compliance deviations across end-to-end business processes to derive actionable recommendations for process improvement & automation.

• It is like a startup within REWE, where we have to build a new forecasting system on Google Cloud Platform from the scratch. Although, officially my role is called MLOps, my actual tasks also include development of data processing pipelines (data engineering) and data scientists tasks such as feature engineering and model trainings.
• GCP: Terraform (tofu), Vertex AI (Kubeflow), Cloud Run, IAM, Google Cloud Storage, BigQuery, Artifact Registry
• Data engineering: Snowflake as the main data warehouse, Terraform, DBT for data model implementations
• CI/CD: GitLab. We have built a CI/CD pipeline that automates deployments of new releases up to production environment

DWH for customer and financial data

• Extension of the DWH with new data sources
• Report development
• Data quality management

Methodology: Scrum

Tools: Atlassian Confluence & Jira

Databases: Microsoft SQL Server

Programming languages: SQL, T-SQL

ETL: Microsoft SQL Server Integration Services (SSIS)

Frontend platform: PowerBI, Microsoft Reporting Services

• Designed and implemented end-to-end data pipelines (batch & streaming) using Python, SQL, and Apache Spark, Databricks on AWS reducing ETL latency by 40%.
• Developed serverless event-driven ingestion pipelines using AWS Lambda and SQS, ensuring real-time data availability for downstream analytics.
• Leveraged Google Cloud Platform (GCP) services including BigQuery and Dataflow to manage cross-cloud data warehousing and analytics integration.
• Expertise in DMS (CDC, Full Load) and Airflow for scalable data pipeline automation and orchestration.
• Managed and customized data pipelines using Databricks, Airflow. Automation using Docker, Kubernetes, Terraform.
• Automated data quality checks using dbt to modularize transformations and ensure production-grade data lineage, improving reliability by 30%.
• Collaborated with compliance teams to ensure GDPR and SOC2 alignment. Mentored junior engineers and contributed to architecture refactoring for scalability.
• Created and maintained dashboards in Power BI to provide actionable insights.

• Migration of existing data jobs from Cognos Data Manager to Tibco/IBI Datamigrator
• Migration data jobs parametrisation for dynamic runs
• Optimisation and cutting-back
• Regression tests
• Knowledge transfer and documentation

• Conducted a comprehensive due diligence review for a potential acquisition of a Swiss manufacturing company as part of a pre-merger analysis
• Assessed production capacities and technical infrastructure
• Evaluated integration possibilities into existing business processes
• Performed risk assessment and developed recommendations for action
• Documented the findings and presented them to management

• Concept creation and implementing AI Agents in AWS Cloud
• Continuously alignment with stakeholders
• Collaborate with DevOps
• Technologies: Git, CI/CD (GitHub Actions), Python/ML, Streamlit, Deno/typescript, AWS SAM, AWS Bedrock, AWS Lambda, AWS Dynamo DB, AWS S3, AWS Event Bridge etc.

• Co-founded an AI venture focused on building production-grade LLM applications and agentic systems
• Designed and implemented multi-agent AI workflows for financial and trading applications
• Developed LLM-powered copilot architectures for portfolio analysis, trade management, and personalized user coaching
• Built on-device and edge-deployed inference applications, optimizing models for low latency, privacy, and resource-constrained environments
• Led system architecture decisions across model selection, orchestration, state management, and deployment

• Topics: Business Analysis, Data Privacy, Data Management, Stakeholder Management, Conceptualization
• This project focuses on developing and implementing a strategic approach for data deletion in SAP systems. The goal is to identify the relevant data and structures during system migration to ensure both data privacy and IT system efficiency. At the same time, downtime should be minimized and regulatory requirements met.
• Development of a comprehensive approach for data deletion in SAP systems, considering data privacy and business requirements.
• Ensuring efficient and structured data transfer to the new system.
• Optimizing system efficiency and reducing downtimes during migration.
• Creating functional and technical concepts to ensure compliant and sustainable data management.
• Topic preparation: Detailed study of the "data deletion" area to lay the foundation for a structured data migration.
• Definition of project structure: Setting roles, interfaces and the project's organizational structure.
• Regulatory requirements: Analysis of data privacy regulations and business requirements to define deletion criteria.
• Approach: Developing possible scenarios and methods for data cleansing and deletion.
• Deletion concepts: Creating functional and technical deletion concepts that structure the implementation and provide clear guidelines.
• Setting deletion criteria: Defining which data and structures to delete or transfer.
• Responsibilities: Clarifying responsibilities within the project team and among stakeholders.
• Analysis of ongoing activities: Identifying and collecting existing activities in the "data deletion" area.
• Effort, cost and timeline planning: Creating estimates for resources, effort and budget.
• Implementation initiatives: Developing and executing concrete measures to apply the defined deletion strategies.
• IT system efficiency: Analyzing the existing IT infrastructure to identify optimization potential for data deletion and transfer.
• Technology trends: Evaluating new technologies and tools that can support the data cleansing process.
• Cost-benefit analysis: Assessing the financial impact of data cleansing and the introduction of new solution approaches.
• Risk management: Identifying potential risks during implementation and developing appropriate mitigation measures.
• This project lays the foundation for a sustainable and compliant data transfer to a new SAP system. With a clear approach to data deletion, it meets data privacy requirements, reduces downtimes and increases the efficiency of the new system. The results and recommendations will help companies develop a future-proof data strategy that meets legal and business needs.

• Role: Software Engineer & Applied Mathematician (Mathematical optimization for scheduling; duration: 1 months; team setting: Team of 2, remote; technologies: JuMP, Julia, Pluto, Svelte, JavaScript, TypeScript, JetBrains Space, Terraform, Nomad)

• Role: Software & Cloud & Web Engineer (Building scalable data science compute cluster from scratch; duration: 11 months; team setting: Team of 1, on-site; technologies: Terraform, Kubernetes, k8s ingress, k8s services, k8s RBAC, k8s networking, k3s, etcd, S3, DNS, certificates, Julia, Pluto, JavaScript, Tailwind, Astro, npm, Parcel, Preact, MUI, JWT, AWS SQS, AWS RDS, Python, GitLab, GitHub)

• Role: AI & Web Engineer (Custom ChatGPT service; duration: 1 months; team setting: Team of 2, remote; technologies: Python, Poetry, LangChain, Tailwind, ChatGPT API, Flask, FastAPI)

• Role: Architect & Data Engineer (Central datalake setup and ingestion; duration: 9 months; team setting: Team of 5, remote; technologies: Infrastructure-as-code, AWS CDK, Python, Boto3, PySpark, AWS Glue, IAM, S3, ECS, Fargate, Lambda, Apache Hudi, DeltaLake, Databricks, GitHub, Jira, Miro)

• Role: Software Engineer (PoC Julia migration of scikit-decide; duration: 1 months; team setting: Team of 2, remote; technologies: Python, Julia, GitHub)