Amazon EMR Experts in Germany

• Architected and deployed private cloud data platform on VMware vSphere, integrating Greenplum MPP, Apache Kafka, Kubernetes, and Apache Solr, and developed real-time ingestion pipelines with Kafka Connect and Schema Registry.
• Led Oracle Exadata to Greenplum migration, rearchitected data models, optimized storage, implemented RabbitMQ with Debezium for CDC, and deployed VectorDB for Generative AI.
• Designed and executed multi-cloud migration PoC across AWS, Azure, and GCP, defined KPIs for throughput, latency, and cost efficiency, executed bulk data transfers, validated analytics and streaming workloads, and delivered full-scale architecture recommendations.
• Assessed legacy on-premises infrastructure and designed modern cloud-native data platforms using Greenplum and containerized microservices, advising on scalability, disaster recovery, and high-availability.

I designed and developed an AI-powered multi-tenant platform on Azure that transforms SAP process recordings into technical documentation, presentations and automated tests, processing over 15,000 process recordings for enterprise customers like Würth. I owned the architecture, the production releases and the DevOps setup. I also designed a multi-tenant system with SSO and role-based access on Azure. Implemented an MCP Server with Dynamic OAuth Authentication.

Main Tasks:

• Sprint planning and feature preparation
• Design the multi-tenant platform architecture (FastAPI, SQLAlchemy, PostgreSQL row-level security for tenant isolation)
• Develop AI pipelines with Prefect for transcription (Azure Speech API), document generation and SAP screen-recording analysis (Claude, gpt-4-mini)
• Design and implement an MCP server to expose tenant knowledge to LLM clients (Claude), with async retrieval and reranking
• Implement LLM cost tracking, rate limiting and client pooling for Anthropic/OpenAI/Azure OpenAI endpoints
• Set up CI/CD: Docker images to Azure Container Registry, GitHub Actions, Azure Static Web Apps, Alembic migrations in containers
• Manage production releases and execute live data migrations for enterprise customers
• Define engineering standards and architecture patterns for the team

Environment: Azure / Azure Foundry / Python / FastAPI / Prefect / React / PostgreSQL

• Development of high-performance ad distribution via auction
• Holistic (multi-campaign & multi-channel) advertisement placement optimization
• Algorithmic optimization for NP-Hard/NP-e
• Multiple Knapsack Problem with constraints
• Online estimation of parameters in stochastic environments

Tools: Python, R, Kotlin, MILP/SAT/CP Solvers, Pytorch, Pandas, Docker

• Responsible for Openshift/Kubernetes on-prem administration and developer support.
• Developed URP infrastructure automation with Python, Ansible, Kustomize and ArgoCD, Argo Workflow/Events stack.
• Wrote smoke and load tests for URP infrastructure utilizing Python, Kustomize and ApplicationSets.
• Helped to set up and deploy URP infrastructure in Google Cloud, GKE.
• Set up monitoring for URP and ArgoCD stack with Splunk Cloud.
• Performed system administration tasks across RedHat Linux, Kubernetes/Openshift, ArgoCD, GitLab, Bitbucket Enterprise, Kafka and MongoDB.

• Develop and demonstrate solutions using GenAI software like langchain, vercel ai sdk, copilotkit
• Work with customers to understand their challenges and provide the best solutions based on open-source data products
• Build RAG and GraphRAG solutions using Neo4j, lancedb, and Postgres
• Deploy a LLMOps platform using kubernetes, terraform, helmfile, Arize phoenix, mlflow
• Architect and build data pipelines using dbt, Trino, Spark, Iceberg, Airflow, ArgoCD, terraform, kubernetes
• Delivered user-centred technical strategy for Agriculture 4.0 and precision livestock farming, helping my client secure funding from Bpifrance
• Delivered a prospecting tool for a leading French solar carport installer, using geospatial computing (GIS), speeding up the sales process
• Built digital twin architecture for solar carports and EV chargers, making real-time monitoring and smart charging possible

• Advising on the use of analytics and BI tools and services in the Microsoft Azure stack (e.g. MS Fabric, Synapse Workspaces and dedicated SQL pools, SQL Database, PostgreSQL, Snowflake, Databricks, Data Factory, SSIS, Analysis Services, Function Apps, Power BI, ML)
• Independently designing analytics solutions with Python, SQL, etc.
• Designing and implementing ETLs and data pipelines
• Creating and maintaining APIs
• Independently applying CI/CD, testing, and version control
• Data modeling
• Model development and optimization
• Anomaly detection with AI
• Predictive analytics

Used technologies:

• Snowflake
• Fabric
• Azure Synapse Analytics
• Azure DataFactory
• Azure Data Lake
• Azure DevOps
• Databricks
• Spark
• CI/CD
• SQL Database
• Python
• Power Platform

• Head of overall technical integration NSC (National Secure Cloud, project with approx. 60 employees).

• Technical integration of all subprojects into one product, definition of interfaces and basic components of a cloud including hardware, technical architecture of the IABG platform.

• Development of a Cloud Management Platform (CMP) capable of creating private/mixed clouds of any complexity based on a textual description with one click or interactively.

• CMP also includes the complete hardware management lifecycle.

• Kubernetes, OpenStack and Hadoop are used as the foundation.

• The management layer includes Harbor, Gitea, Longhorn, Keycloak, Rancher and Jenkins, which are configured automatically.

• Private cloud can run any customer workloads, including a full Hadoop layer with HDFS, Spark, MapReduce, Mesos, HBase and around 20 additional ML/DL technologies.

• Hadoop worker clusters can also be installed automatically without Kubernetes on bare metal or commodity hardware.

• OpenStack with Nova, Neutron, Ironic, Swift, Cinder, Ceph.

• Development of a Java application Rudi: SOAP, REST, containers, DB.

• Technologies: Kubernetes (K3s, Rke2, Minikube, Harbor, Gitea, Jenkins, Longhorn, Keycloak, Rancher), OpenStack (Nova, Neutron, Keystone, Swift, Ceph, Cinder, Sahara, Magnum, Kayobe, Kolla, Bigrost, Ironic), Hadoop (HDFS, Ambari, Solr, Livy, Ranger, YARN, Tez, HBase, Kafka, Hive, Zookeeper, MapReduce, Spark, Oozie, Flink), virtualization (Kubernetes (K3S), VMware, Oracle), scripting (Ansible, Puppet, Juju, Shell, Groovy, Gradle, Maven).

• Technical consulting for greenfield development of AVIATAR 2.0
• Designing data pipelines and data architecture within the DataVerse Data Lake
• Planning and implementing Data Lake zones, data governance, data retention, and recovery strategies
• Collaboration with the Data Architecture and Platform Team for infrastructure scaling and design
• Close cooperation with stakeholders across Lufthansa Technik, including requirements management
• Hands-on development of data pipelines using Spark Structured Streaming and Databricks
• Proofs of Concept (PoCs) for Graph-Links and real-time analytics
• Provisioning data for AI and ML projects (e.g., predictive analytics)
• Technical design and implementation of near-real-time pipelines based on business requirements

• Role: Software Engineer & Applied Mathematician (Mathematical optimization for scheduling; duration: 1 months; team setting: Team of 2, remote; technologies: JuMP, Julia, Pluto, Svelte, JavaScript, TypeScript, JetBrains Space, Terraform, Nomad)

• Role: Software & Cloud & Web Engineer (Building scalable data science compute cluster from scratch; duration: 11 months; team setting: Team of 1, on-site; technologies: Terraform, Kubernetes, k8s ingress, k8s services, k8s RBAC, k8s networking, k3s, etcd, S3, DNS, certificates, Julia, Pluto, JavaScript, Tailwind, Astro, npm, Parcel, Preact, MUI, JWT, AWS SQS, AWS RDS, Python, GitLab, GitHub)

• Role: AI & Web Engineer (Custom ChatGPT service; duration: 1 months; team setting: Team of 2, remote; technologies: Python, Poetry, LangChain, Tailwind, ChatGPT API, Flask, FastAPI)

• Role: Architect & Data Engineer (Central datalake setup and ingestion; duration: 9 months; team setting: Team of 5, remote; technologies: Infrastructure-as-code, AWS CDK, Python, Boto3, PySpark, AWS Glue, IAM, S3, ECS, Fargate, Lambda, Apache Hudi, DeltaLake, Databricks, GitHub, Jira, Miro)

• Role: Software Engineer (PoC Julia migration of scikit-decide; duration: 1 months; team setting: Team of 2, remote; technologies: Python, Julia, GitHub)

• Implementation of specified DevOps solutions to automate infrastructure (Terraform, Bicep, CloudFormation, Ansible) on-premises datacenter (Ovirt, Proxmox, Ceph Cluster, MinIO) and private cloud.
• Administration, configuration and implementation of CI/CD DevOps pipelines (GitLab, GitFlow) to support development process (Artifactory, Prometheus, Istio, service mesh, Helm Chart, OpenShift (Red Hat Enterprise) / Kubernetes cluster), Red Hat Satellite.
• Administration, setup, monitoring and patching of Linux infrastructure based on Red Hat Enterprise for Dev, Test and QA.
• Use of Scrum and Kanban methods.
• Administration, configuration and implementation of security standards for deploying on Dev, Test, QA and Prod stages of the new ePA applications.
• Development of new plugins and add-ons needed on current infrastructure.
• Database support.
• Data analytics support (Python, Spark, Pandas, Power BI, Splunk Enterprise).
• Implementation of best practices for DevSecOps and BizDevOps using GitOps (ArgoCD), Streamlit framework, Semaphore Ansible UI.
• Configuration and testing of iperf, uperf, sysbench using benchmark-operator for external source data and IoT/MDM devices, creating reports via ELK / OpenSearch.
• Building a new Databricks platform to collect and analyze big data from different sources and IoT devices into Hadoop framework (Python, Pandas, PySpark, Power BI, Apache Airflow).
• Building backend data aggregation and processing to automate configuration deployment between different OpenShift clusters and big data framework (Python, Pandas, PySpark, Apache Spark, PostgreSQL, Django 2, Ansible Automation, Jira JSM).
• Building a new ML pipeline platform using Kubeflow, TensorFlow, KServe.
• Data extraction, transformation and loading from different data sources including structured and unstructured data to analytic DWH / big data cluster using Python, Pandas, Polars, Power BI, Django backend and PostgreSQL.
• Setup of new DevOps Test and QA HashiCorp Vault cluster for PKI and IAM.
• Configuration and testing of automated patching based on CVSS score, SIEM-integrated CVEs.
• Use of Nexpose and InsightVM to scan vulnerability events in network, host, container and application.
• Design and implementation of secure and scalable AWS architectures including VPC, EC2, S3, RDS and Route53 and similar setups on Azure and GCP.
• Automated system provisioning and deployment using CloudFormation templates.
• Configuration of IAM roles, policies and permissions to ensure secure access control.
• Patch management, backup automation and disaster recovery setup on AWS infrastructure.
• Monitoring and optimization of system performance using AWS CloudWatch and AWS Trusted Advisor.
• Support of VMware services (vSphere, Aria, Horizon) and the virtual desktop environment.
• Development and maintenance of CI/CD pipelines using Jenkins, GitLab CI/CD and AWS CodePipeline with interface to Nutanix.
• Configuration of AWS CloudWatch to monitor application performance and system events.
• Planning and execution of migration of on-premises applications to AWS cloud platforms.
• Deployment of containerized applications using Docker and Kubernetes in AWS environments.
• Deployment of internal software packages between availability zones using AWS CodeDeploy.
• Building and deploying ML models using Scikit-learn, XGBoost and Spark MLlib including hyperparameter tuning, model evaluation and production deployment.

• Design and provide best practices on data modeling for dbt, including changing dimensions, late arriving data handling, and testing
• Design the ingestion flow from other systems into S3 and Redshift
• Design and implement new partitions for Dagster and incremental loading with dbt
• Map business requirements to technical architectures
• Instruct junior team members

• Built a decentralized app using Next.js and Convex DB for secure offline Solana transaction signing.

• Architected end-to-end credit risk pipelines by orchestrating Airflow ETLs and training LSTMs/Transformers to predict default and prepayment speeds on MBS portfolios.
• Developed Deep Learning NLP solutions using BERT and LayoutLM for document processing, leveraging Transfer Learning and custom PyTorch loss functions to automate underwriting.
• Optimized R&D lifecycles through Bayesian tuning, Batch Normalization, and MLflow tracking to ensure robust model performance throughout volatile mortgage market cycles.
• Productionized scalable MLOps infrastructure via Docker and INT8 Quantization, deploying low-latency FastAPI microservices on AWS SageMaker with automated CI/CD pipelines.
• Ensured regulatory compliance by integrating SHAP/LIME for explainability and establishing real-time Data Drift monitoring to meet strict FHFA and Fair Lending standards.