Apache Kafka Experts in Frankfurt

I am an experienced data engineer specializing in end‑to‑end data integration, cloud DWH architectures, and high‑quality, governed data products.

I delivered following projects and engagements as a freelancer.

• Data Migration of CRM System for AL-FA Objekt Service Gmbh
• Microsoft Software Resales Partnership

I am looking for freelance roles like: Freelance Data Engineer Cloud Data Warehouse Architect Data Modeling & Architecture Consultant MDM & Data Governance Specialist BI & Analytics Developer

Technical Focus Areas

• Data Engineering & Integration: SQL Server/SSIS, Informatica PowerCenter/IDQ, Talend, Kafka, Azure Data Factory – Delta/CDC/ELT patterns, robust pipelines, monitoring/recovery, data lineage & impact analysis, medallion architecture Bronze/Silver/Gold layers
• DWH & Cloud: Azure SQL / Data Lake / Synapse, AWS Redshift/S3, on‑prem SQL/Oracle – scalable data marts with a strong cost/benefit focus.
• Data Modeling: Atomic (Inmon) and Dimensional (Kimball), Data Vault (Linstedt), Domain‑Driven Design, clear lineage & contracts.
• MDM & Governance: Informatica MDM, IBM MDM, stewardship processes, data quality rules, survivorship/XREF, catalog/glossary, SIF/BES/REST publication.
• Analytics/BI: Power BI, SSAS, Cognos – business‑ready, maintainable data products.

• Architected the Yatta Integration Layer – a config-driven integration platform on Java 25, Spring Boot 4 (WebFlux), Temporal, gRPC and Kafka, enabling new third-party integrations (e.g. AVS fulfillment) via declarative JSON configs with zero code changes.
• Designed and implemented Tink integration with 0Auth IBAN verification to enhance fraud prevention and account validation workflows with Adyen payByBank.
• Architected and implemented an OpenFGA-based authorization model for centralized management of users, groups, and fine-grained access control in the vendor portal.
• Architected and led delivery of the Yatta API Gateway platform using GraphQL Federation, providing a unified enterprise API layer across distributed microservices with centralized authentication, authorization and request orchestration.
• Replaced NGINX + NLB with Istio service mesh and AWS ALB; rolled out WAF, OAuth (Cognito), IP whitelisting and RBAC across environments.
• Migrated CDC from Confluent Cloud connectors to a self-hosted Kafka Connect + Debezium stack, reducing operational cost by ~80% across multiple environments.
• Implemented the Transactional Outbox pattern with Debezium for reliable, exactly-once event publishing to Kafka with Avro and Schema Registry.
• Migrated dunning/payment-recovery workflows from Airflow to Temporal, achieving 99.9% reliability for settlement handling.
• Optimised Apache Airflow with deferrable sensors to handle 1000+ concurrent DAG runs without scaling the worker pool.
• Refactored a monolithic Terraform codebase into 3 modular projects, cutting deployment time by ~45%.
• Stood up full observability with OpenTelemetry, Tempo, Prometheus and Loki; automated dev/staging/prod with ArgoCD, Image Updater and Helm.
• Collaborated with product, operations and engineering stakeholders to define scalable platform architecture and integration standards aligned with long-term business and operational goals.

• Led discovery phase and POC for Shopware to Shopify Plus migration across EMEA markets, evaluating platform suitability, technical architecture, and multi-brand/multi-country capabilities against business requirements.
• Designed reference architecture for Shopify Plus implementation incorporating headless front-end patterns (Vue.js, Nuxt.js), CMS integration (Magnolia), and Azure middleware (APIM, Functions, Logic Apps, Service Bus) for 11 EMEA markets.
• Defined migration strategy analyzing data mapping, cutover approach, and zero-downtime deployment patterns using Varnish caching, GitOps pipelines, and CI/CD orchestration across six vendor teams.
• Architected multi-tenant Shopify Plus governance model with centralized admin, localized storefront customization, and compliance controls (GDPR, data residency).
• Prototyped AI-driven search optimization (LLM.txt, JSON-LD) for product discoverability in Google AI results, demonstrating post-launch performance opportunities.
• Defined EMEA expansion roadmap for 15+ markets through C-level strategic workshops, identifying phased rollout, market-specific configurations, and resource requirements.
• Tech Stack: React, Nuxt.js, Vue.js, Magnolia CMS, Shopware, Shopify Plus, Azure (APIM, Functions, Logic Apps, Service Bus, Front Door), Varnish, SAP, MS Dynamics, Docker, Kubernetes, GitHub Actions, PostgreSQL, Kafka

• Platform established and running stably; deliberately reducing my involvement to refocus on an engineering mandate in the financial sector.
• Built an own SaaS learning platform from the ground up and scaled it to over 20,000 users (over 6,000 courses sold, B2C and B2B); end-to-end ownership from development through infrastructure to operations.
• Built a lab environment that provisions an isolated Linux container per user (Docker, Traefik, Go), including automatic provisioning and a dedicated subdomain per user.
• Integrated LLM features into the product and accelerated development end-to-end with AI-assisted workflows (Claude Code, Codex); CI/CD with automated tests.

Project: SCS (Settlement / Clearing Services)

Settlement platform serving multiple trading and clearing venues — counterparty risk safeguarding, settlement volume reduction, central risk management, and post-trade anonymity.

Technologies: Java 17, Spring Boot 3, Microservices, Spring Data JPA, SonarQube, Fortify (SCST), Mockito, Jenkins, OpenShift, Maven, Podman, GitHub, JIRA, Liquibase, Swagger, AMQP, Apache Camel, Terraform, PostgreSQL, Instana, Graylog.

• Identified and remediated CVEs in third-party libraries using SCA tooling, strengthening the security posture of production components.
• Maintained 90% code coverage with SonarQube, improving code quality and reducing production defects.
• Built and customized Helm charts and values.yaml configurations across 3 environments (ACT, SIMU, PROD), increasing deployment flexibility and consistency.
• Enabled mTLS for database authentication and message broker connections, enforcing encrypted, certificate-validated communication.
• Designed and deployed microservices with asynchronous, REST-based communication between components.
• Automated build and continuous integration pipelines using Maven and Jenkins.
• Used Podman and OpenShift for container orchestration and Liquibase for database version control.
• Optimized Java code and implemented EHCache-based caching, improving application performance.
• Managed application images, JAR versions, and dependencies via DBAG Artifactory Repository Manager.

• Implementation of specified DevOps solutions to automate infrastructure (Terraform, Bicep, CloudFormation, Ansible) on-premises datacenter (Ovirt, Proxmox, Ceph Cluster, MinIO) and private cloud.
• Administration, configuration and implementation of CI/CD DevOps pipelines (GitLab, GitFlow) to support development process (Artifactory, Prometheus, Istio, service mesh, Helm Chart, OpenShift (Red Hat Enterprise) / Kubernetes cluster), Red Hat Satellite.
• Administration, setup, monitoring and patching of Linux infrastructure based on Red Hat Enterprise for Dev, Test and QA.
• Use of Scrum and Kanban methods.
• Administration, configuration and implementation of security standards for deploying on Dev, Test, QA and Prod stages of the new ePA applications.
• Development of new plugins and add-ons needed on current infrastructure.
• Database support.
• Data analytics support (Python, Spark, Pandas, Power BI, Splunk Enterprise).
• Implementation of best practices for DevSecOps and BizDevOps using GitOps (ArgoCD), Streamlit framework, Semaphore Ansible UI.
• Configuration and testing of iperf, uperf, sysbench using benchmark-operator for external source data and IoT/MDM devices, creating reports via ELK / OpenSearch.
• Building a new Databricks platform to collect and analyze big data from different sources and IoT devices into Hadoop framework (Python, Pandas, PySpark, Power BI, Apache Airflow).
• Building backend data aggregation and processing to automate configuration deployment between different OpenShift clusters and big data framework (Python, Pandas, PySpark, Apache Spark, PostgreSQL, Django 2, Ansible Automation, Jira JSM).
• Building a new ML pipeline platform using Kubeflow, TensorFlow, KServe.
• Data extraction, transformation and loading from different data sources including structured and unstructured data to analytic DWH / big data cluster using Python, Pandas, Polars, Power BI, Django backend and PostgreSQL.
• Setup of new DevOps Test and QA HashiCorp Vault cluster for PKI and IAM.
• Configuration and testing of automated patching based on CVSS score, SIEM-integrated CVEs.
• Use of Nexpose and InsightVM to scan vulnerability events in network, host, container and application.
• Design and implementation of secure and scalable AWS architectures including VPC, EC2, S3, RDS and Route53 and similar setups on Azure and GCP.
• Automated system provisioning and deployment using CloudFormation templates.
• Configuration of IAM roles, policies and permissions to ensure secure access control.
• Patch management, backup automation and disaster recovery setup on AWS infrastructure.
• Monitoring and optimization of system performance using AWS CloudWatch and AWS Trusted Advisor.
• Support of VMware services (vSphere, Aria, Horizon) and the virtual desktop environment.
• Development and maintenance of CI/CD pipelines using Jenkins, GitLab CI/CD and AWS CodePipeline with interface to Nutanix.
• Configuration of AWS CloudWatch to monitor application performance and system events.
• Planning and execution of migration of on-premises applications to AWS cloud platforms.
• Deployment of containerized applications using Docker and Kubernetes in AWS environments.
• Deployment of internal software packages between availability zones using AWS CodeDeploy.
• Building and deploying ML models using Scikit-learn, XGBoost and Spark MLlib including hyperparameter tuning, model evaluation and production deployment.

• Hands-on solution architect
• Digital product development - SaaS
• Strategic consulting on software architecture, cloud migration, and DevOps processes
• Coaching SE developers and IT architects

• Co-founded the Real-Time Data team, which grew to 10 members over time.
• Developed and delivered core data products.
• Optimized real-time application performance and implemented monitoring, alerting and logging solutions to ensure system stability.
• Created and maintained deployment pipelines.
• Collaborated with teammates, architects and experts in an agile Scrum environment.
• Operated applications, analyzed, tested and troubleshot software solutions.

• Datastage 11.7, dbt, Oracle 19, Python 3.12 / PySpark 3.5, Azure GitHub, Azure DevOps, Automic
• Development of migration jobs to transfer data from the collection DWH to the new Risk Mart, as well as development of ETL pipelines to migrate historical data from the old Mart to the new Risk Mart.
• Storage of the silver layer on Hadoop and the gold layer in Oracle.
• Translation of DataStage jobs into dbt to publish reporting data in Google Cloud to a PostgreSQL database.
• Creation and optimization of complex SQL queries for data extraction from a data vault, taking into account historical data in the point-in-time tables.
• Creation of Oracle table definitions (DDL) and adjustment of existing stored procedures.
• Versioning changes in GitHub and deployment via the CI/CD portal.
• Refactoring long-running DataStage jobs into Python using PySpark to reduce server load.
• Migration of SAS scripts to PL/SQL, including new development of distribution functions that have no direct equivalent in Oracle.
• Development of Automic jobs to run DataStage pipelines and Python scripts (PySpark jobs) that control the population of the SME and institutional risk tables in the Risk Mart and perform business calculations.
• Participation in the agile process, including creating user stories, estimations, and planning in Azure DevOps.
• Handling Azure DevOps tickets and close collaboration with testers and business teams for error analysis and resolution.

• Advising on the use of analytics and BI tools and services in the Microsoft Azure stack (e.g. MS Fabric, Synapse Workspaces and dedicated SQL pools, SQL Database, PostgreSQL, Snowflake, Databricks, Data Factory, SSIS, Analysis Services, Function Apps, Power BI, ML)
• Independently designing analytics solutions with Python, SQL, etc.
• Designing and implementing ETLs and data pipelines
• Creating and maintaining APIs
• Independently applying CI/CD, testing, and version control
• Data modeling
• Model development and optimization
• Anomaly detection with AI
• Predictive analytics

Used technologies:

• Snowflake
• Fabric
• Azure Synapse Analytics
• Azure DataFactory
• Azure Data Lake
• Azure DevOps
• Databricks
• Spark
• CI/CD
• SQL Database
• Python
• Power Platform

• Database design with Oracle DB, JPA and Flyway considering high performance requirements
• Analysis of multiple architecture drafts and identification of technical risks
• Design and implementation of various interfaces and integrations, especially REST APIs and Kafka topics
• Setup of Elasticsearch for monitoring and analyzing log data
• Establishment of an iterative work approach in the project team facing complex business requirements

• General CTO responsibilities (architectural design, operational setup, external runtime product evaluation, investor buy-in, regulatory compliance).
• Software development oversight (implementation on deep-dive-in) plus workflow design.
• Product co-ownership.
• Tech/tools/frameworks: proprietary software (Java, JavaScript), Kubernetes, Postgres, MiniIO, Ollama (internal), several xAI API (external), OpenTofu (Terraform), Keycloak, Kafka, Prometheus, ELK Stack, GitHub, GitHub Workflows, Argo CD, ISO 27001, BSI-ISM, EU AI Act.

AI-powered platform for automated UX validation and designer support

• Designed and led technical implementation of an enterprise-wide AI solution for automated UX review that improved design quality and significantly reduced manual review processes in teams
• Developed an automated UX validation tool as a Figma plugin and web application that generates test cases based on internal guidelines and reliably checks current designs for consistency and standard compliance
• Implemented an interactive designer chat based on RAG that answers questions about the current design and the company's UX guidelines, and designed the deployment architecture using containerized services
• Python, Azure OpenAI, PostgreSQL, REST API, Docker, OpenShift, Helm, CI/CD, Figma MCP, LLM, RAG, Prompt Engineering, GenAI, XAI, AI Architecture, AI Strategy

• Testing the SekIDP and related components
• Test design, execution and automation, microservices, GitHub Enterprise, Eclipse, Katalon Test Platform, API Testing, Confluence 8, JIRA/Xray, Draw.io, Swagger/OpenAPI, Postman, Docker, Kubernetes, Podman, PuTTY, E-Health, Telematik, Gematik standards, EPA, E-Rezept, sektoraler IDP, encryption, XaDES, PaDES, DICOM, HL7, FHIR, IHE, ICD, SSO, SAML/Shibboleth, OAuth, smartcards, JWT, two factor authentication Android and iOS, Wireshark, BrowserStack, Cypress, gRPC, REST, SOAP, WS-Security, Linux Shell, PowerShell, SSH/SSL, Java, Kotlin, Cordova, Gradle, Groovy, Python, TypeScript