Apache Airflow Experts in Germany

I am an experienced data engineer specializing in end‑to‑end data integration, cloud DWH architectures, and high‑quality, governed data products.

I delivered following projects and engagements as a freelancer.

• Data Migration of CRM System for AL-FA Objekt Service Gmbh
• Microsoft Software Resales Partnership

I am looking for freelance roles like: Freelance Data Engineer Cloud Data Warehouse Architect Data Modeling & Architecture Consultant MDM & Data Governance Specialist BI & Analytics Developer

Technical Focus Areas

• Data Engineering & Integration: SQL Server/SSIS, Informatica PowerCenter/IDQ, Talend, Kafka, Azure Data Factory – Delta/CDC/ELT patterns, robust pipelines, monitoring/recovery, data lineage & impact analysis, medallion architecture Bronze/Silver/Gold layers
• DWH & Cloud: Azure SQL / Data Lake / Synapse, AWS Redshift/S3, on‑prem SQL/Oracle – scalable data marts with a strong cost/benefit focus.
• Data Modeling: Atomic (Inmon) and Dimensional (Kimball), Data Vault (Linstedt), Domain‑Driven Design, clear lineage & contracts.
• MDM & Governance: Informatica MDM, IBM MDM, stewardship processes, data quality rules, survivorship/XREF, catalog/glossary, SIF/BES/REST publication.
• Analytics/BI: Power BI, SSAS, Cognos – business‑ready, maintainable data products.

• Architected the Yatta Integration Layer – a config-driven integration platform on Java 25, Spring Boot 4 (WebFlux), Temporal, gRPC and Kafka, enabling new third-party integrations (e.g. AVS fulfillment) via declarative JSON configs with zero code changes.
• Designed and implemented Tink integration with 0Auth IBAN verification to enhance fraud prevention and account validation workflows with Adyen payByBank.
• Architected and implemented an OpenFGA-based authorization model for centralized management of users, groups, and fine-grained access control in the vendor portal.
• Architected and led delivery of the Yatta API Gateway platform using GraphQL Federation, providing a unified enterprise API layer across distributed microservices with centralized authentication, authorization and request orchestration.
• Replaced NGINX + NLB with Istio service mesh and AWS ALB; rolled out WAF, OAuth (Cognito), IP whitelisting and RBAC across environments.
• Migrated CDC from Confluent Cloud connectors to a self-hosted Kafka Connect + Debezium stack, reducing operational cost by ~80% across multiple environments.
• Implemented the Transactional Outbox pattern with Debezium for reliable, exactly-once event publishing to Kafka with Avro and Schema Registry.
• Migrated dunning/payment-recovery workflows from Airflow to Temporal, achieving 99.9% reliability for settlement handling.
• Optimised Apache Airflow with deferrable sensors to handle 1000+ concurrent DAG runs without scaling the worker pool.
• Refactored a monolithic Terraform codebase into 3 modular projects, cutting deployment time by ~45%.
• Stood up full observability with OpenTelemetry, Tempo, Prometheus and Loki; automated dev/staging/prod with ArgoCD, Image Updater and Helm.
• Collaborated with product, operations and engineering stakeholders to define scalable platform architecture and integration standards aligned with long-term business and operational goals.

Scope: Embedded as FP&A Data & AI Architect within the Finance function of a major European refrigeration manufacturer, leading the transformation of manual, fragmented financial reporting into an automated, governance-driven intelligence platform. Driving the shift from Excel-based controlling to structured data architecture, Power BI analytics, and AI-assisted financial operations.

Financial Data Integrity & ERP Governance

• Initiated and led GL vs. subledger reconciliation investigations, identifying and resolving structural mismatches between General Ledger and subledger data that had gone undetected prior to engagement
• Conducted asset analysis to identify items missing from General Ledger postings, surfacing gaps in fixed asset tracking and period-end completeness
• Validated SAP reports, establishing baseline data quality standards for Finance team consumption
• Established systematic SAP data validation framework ensuring ongoing integrity between ERP postings and downstream reporting outputs

Finance Reporting Transformation

• Designed and implemented a structured Transformation Project approach for converting manual Finance reports into fully automated processes
• Created and owns the Data Reporting Audit Log; a centralized tracking system capturing report owners, stakeholders, data sources, manual effort estimates, and automation opportunity scores across the Finance function
• Mapped the full reporting landscape identifying quick-win automation targets and strategic Power BI migration candidates
• Actively reducing manual Excel and PowerPoint dependency across FP&A workflows; replacing point-in-time snapshots with live, governed data models

Power BI & Analytics Enablement

• Introduced and presented Power BI as the strategic reporting platform to Finance leadership, building internal buy-in for the BI transformation roadmap
• Designed initial Power BI architecture aligned with SAP, Salesforce and Oracle data structures and FP&A reporting requirements
• Established report ownership, governance documentation, and data lineage standards enabling sustainable self-service analytics across the Finance team

Transformation Infrastructure & Collaboration

• Configured and deployed Jira as the transformation project management hub, establishing structured sprint workflows, backlog management, and progress visibility for Finance IT initiatives
• Proposed and initiated a dedicated FP&A Communication & Transformation Hub, a structured cross-functional forum aligning Finance, IT, and business stakeholders around the reporting transformation roadmap
• Positioned the Finance function as an active driver of data governance and digital transformation within the broader organization

Outcomes

• GL/subledger reconciliation gaps identified and investigation framework established within first two weeks of engagement
• Data Reporting Audit Log deployed; first structured inventory of Finance reporting landscape in company history
• Power BI transformation roadmap presented and approved by Finance leadership
• Jira-based project governance live; Finance transformation now tracked with full sprint visibility

Technologies: SAP FI/CO · Power BI · DAX · SQL · Excel (advanced) · Power Query (M) · Power Automate · VBA · Jira · Microsoft 365 · SharePoint · Salesforce (Sales Data) · Oracle HCM · Python

• Client — Podimo ApS (podcast & audiobook streaming): build the finance reporting layer on BigQuery + dbt + Airflow, including the core revenue-transaction fact tables used across finance reporting.
• API automation: design and build a BigQuery → Airflow → Microsoft Dynamics 365 Business Central REST-API pipeline to automate sales-invoice posting, with idempotency and master-data sync between systems.
• Delivery: sole engineer on the engagement — requirements, modelling, orchestration and stakeholder communication with the client finance team, end to end.

• Architected and deployed private cloud data platform on VMware vSphere, integrating Greenplum MPP, Apache Kafka, Kubernetes, and Apache Solr, and developed real-time ingestion pipelines with Kafka Connect and Schema Registry.
• Led Oracle Exadata to Greenplum migration, rearchitected data models, optimized storage, implemented RabbitMQ with Debezium for CDC, and deployed VectorDB for Generative AI.
• Designed and executed multi-cloud migration PoC across AWS, Azure, and GCP, defined KPIs for throughput, latency, and cost efficiency, executed bulk data transfers, validated analytics and streaming workloads, and delivered full-scale architecture recommendations.
• Assessed legacy on-premises infrastructure and designed modern cloud-native data platforms using Greenplum and containerized microservices, advising on scalability, disaster recovery, and high-availability.

• Building, implementing and selling automated Data Science solutions such as Scorecard Factory and Forecast Factory
• Implementation of automated end-to-end cloud processes
• Development of LLM and NLP models
• Creation of interactive reports
• Support for national and international large corporations as well as medium-sized companies in implementing ML projects

Architected and maintained data products for renewable energy operations, covering wind turbine, grid-meter, and weather data. Built scalable ETL/ELT pipelines in Azure Databricks using Delta Lake (bronze/silver/gold layers) and processed data in various formats, including structured and semi-structured data. Contributed to a data quality framework supporting table and column documentation, outlier detection, and completeness metrics across all datasets within a data product. In addition, implemented a DORA KPI Databricks dashboard used across all data products. Optimized CI/CD processes in Azure DevOps to streamline deployment across development, test, and production environments.

Technology stack: Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Azure Data Lake, APIs, Dremio, Azure DevOps, YAML, Git, Databricks Workflows, Application Insights, Terraform, OpenAI API, Codex, LLM-assisted workflows

Led FP&A tech transformation for a 9-figure business – from resolving legacy technical debt to leading AI-native EPM implementation

• Driving end-to-end FP&A transformation, from architecture redesign through EPM tool selection to rollout
• Ran evaluation of 12+ EPM platforms, from vendor negotiation to selection framework tied to long-term planning
• Diagnosed constraints in financial planning architecture, presented findings to the CFO, and secured executive mandate to redesign FP&A infrastructure from the ground up

• Delivered MVP-focused full-stack architecture for a health-sector client: Vite/React frontend, backend services on Google Cloud Run, and Supabase for database plus IAM/authentication.
• Supported product requirements engineering and prioritized cost-aware workload placement, implementing browser-side/edge computation where feasible before moving logic to backend services.

• Mentor team of 6 ML platform engineers through weekly 1:1s, technical design reviews, and best practices, improving team velocity by 35% through structured sprint planning and skill development programs
• Define 2025–2026 ML platform roadmap in collaboration with Data Science, Cloud Engineering, and Product teams, prioritizing automated model governance, cost attribution systems, and multi-environment deployment strategies
• Partner with Data Science, SRE, and Product stakeholders to align ML platform capabilities with business objectives, reducing data scientist deployment friction by 60% through self-service platforms
• Architect and deliver production-grade MLOps platform supporting 50+ models in production with automated promotion pipelines, versioning, and rollback capabilities, achieving 99.5% platform uptime SLA
• Design distributed ML pipeline architecture using Metaflow and Argo Workflows (Vertex Pipelines-compatible), reducing model training time by 30% and deployment cycles from 2 weeks to 3 days through full CI/CD automation
• Build containerized ML services on Kubernetes with auto-scaling policies, resource quotas, and multi-tenancy isolation, optimizing infrastructure costs by $180K annually (25% reduction)
• Implement monitoring, alerting, and performance tracking using Prometheus, Grafana, and custom instrumentation, reducing model debugging time by 50% and establishing model performance SLOs
• Lead development of RAG-based document intelligence platform using LangChain, LangGraph, and vector databases, implementing agentic AI workflows for automated financial document processing
• Implement Infrastructure-as-Code using Terraform for reproducible environment provisioning and GitOps workflows, reducing infrastructure drift incidents by 80%
• Design role-based access control for ML platform, implement model lineage tracking, and establish audit trails for regulatory compliance aligned with enterprise IAM best practices

I designed and developed an AI-powered multi-tenant platform on Azure that transforms SAP process recordings into technical documentation, presentations and automated tests, processing over 15,000 process recordings for enterprise customers like Würth. I owned the architecture, the production releases and the DevOps setup. I also designed a multi-tenant system with SSO and role-based access on Azure. Implemented an MCP Server with Dynamic OAuth Authentication.

Main Tasks:

• Sprint planning and feature preparation
• Design the multi-tenant platform architecture (FastAPI, SQLAlchemy, PostgreSQL row-level security for tenant isolation)
• Develop AI pipelines with Prefect for transcription (Azure Speech API), document generation and SAP screen-recording analysis (Claude, gpt-4-mini)
• Design and implement an MCP server to expose tenant knowledge to LLM clients (Claude), with async retrieval and reranking
• Implement LLM cost tracking, rate limiting and client pooling for Anthropic/OpenAI/Azure OpenAI endpoints
• Set up CI/CD: Docker images to Azure Container Registry, GitHub Actions, Azure Static Web Apps, Alembic migrations in containers
• Manage production releases and execute live data migrations for enterprise customers
• Define engineering standards and architecture patterns for the team

Environment: Azure / Azure Foundry / Python / FastAPI / Prefect / React / PostgreSQL

• Designed and developed AI-powered document processing solutions using Python, OCR, NLP, and Large Language Models (LLMs) to automate extraction, validation, and classification of financial documents, reducing processing time by 75%.
• Built intelligent multi-stage workflow automation pipelines integrating AI services, machine learning models, and enterprise systems to streamline financial operations and improve data quality.
• Developed reusable AI-driven transformation frameworks capable of processing structured and unstructured document formats (XML, CSV, JSON, TXT, DAT) and normalizing them into unified business schemas.
• Designed and developed Python-based REST APIs and backend services supporting enterprise finance applications and high-volume data processing workloads.
• Built scalable data synchronization pipelines between Oracle CFIN and SQL databases, incorporating machine learning models for cash-flow forecasting and AP/AR anomaly detection.
• Architected and deployed Apache Airflow workflows to orchestrate AI-powered data pipelines, automating end-to-end processing from document ingestion through financial system integration.
• Led the migration of critical enterprise integrations from MuleSoft to Python-based services, improving maintainability, performance, and operational flexibility while preserving complete data integrity.
• Managed the full API lifecycle including solution design, implementation, documentation, deployment, monitoring, and production support for mission-critical financial systems.
• Collaborated directly with finance stakeholders to identify business challenges, define solution requirements, and deliver measurable operational improvements through automation and AI-driven workflows.
• Worked closely with cross-functional engineering and business teams to rapidly iterate on features, improve processes, and drive successful adoption of AI-enabled solutions.
• Provided technical leadership through architecture reviews, technology decisions, code reviews, and engineering best practices across integration and automation initiatives.
• Mentored developers, established coding standards, and contributed to improving software quality, maintainability, and delivery effectiveness across projects.
• Provided production support during critical month-end and quarter-close financial processes, performing root-cause analysis and implementing rapid fixes to ensure system reliability and data accuracy.

• Designed and deployed enterprise Retrieval-Augmented Generation (RAG) applications using LangChain, LangGraph, vector databases, embeddings, and open-source LLMs served through vLLM on GCP GPU infrastructure.
• Built agentic AI workflows using LangGraph with planning, reasoning, tool execution, persistent memory, session management, and Human-in-the-Loop approval mechanisms.
• Developed LLM-powered automation systems integrating BigQuery, SQL pipelines, and external advertising APIs including Meta, TikTok, Amazon, Snapchat, Google, and Pinterest, reducing manual operational workflows.
• Architected multi-agent AI systems for enterprise analytics and decision-support workflows, enabling autonomous task execution and intelligent data interactions.
• Implemented retrieval optimization strategies including multi-retriever architectures, semantic search, context optimization, and query improvement techniques, improving response relevance by approximately 40%.
• Engineered structured prompting strategies, function-calling schemas, and validation workflows to improve reliability of multi-step LLM applications.
• Designed scalable AI services using Python, FastAPI, Cloud Run, Pub/Sub, BigQuery, Docker, and cloud-native deployment architectures.

• Build and operate an end-to-end MLOps platform on Azure ML and Kubernetes (Kubeflow) for the automated deployment, monitoring, and scaling of forecasting models (including Temporal Fusion Transformer, Informer, Autoformer).
• Implement CI/CD pipelines in Azure DevOps for the full ML lifecycle – from resource provisioning (Terraform), data transformation (Hugging Face Datasets, Pandas, PyTorch, CUDA cluster) through training and evaluation to model registry and endpoint deployment.
• Integrate MLflow for experiment tracking, model versioning, performance monitoring, and automated registration in the Azure Model Registry.
• Develop and containerize PyTorch training jobs (Azure Notebook, Jupyter Notebooks) for price and time series forecasting (PFC models) with automatic rollout via Azure ML Endpoints and REST/gRPC interfaces, Docker containerization, secured with OAuth 2.0.
• Set up monitoring and alerting mechanisms (Prometheus, MLflow Metrics), log centralization, and cost monitoring.
• Automate infrastructure provisioning and model deployment using Terraform, Helm, and Azure CLI; connect to existing market data systems and event pipelines.
• Migrate existing workloads and databases (IONOS → Azure, MongoDB) with integration into central MLOps workflows and internal networks.
• Extend the platform with LLM-based tools (LangChain, LangServe) to integrate GPT-based analysis modules into existing Spring Boot services for market anomaly detection and automated reports.
• Analyze and architect a software solution to process large volumes of data efficiently (>3000 messages/sec.) (market data store).
• Spring Boot / Java 21 container development with RabbitMQ for distributing stock market data via MongoDB (Kubernetes) with fast storage of data in Redis RMaps, deduplication, forwarding messages to Read Model queues, and building Read Models for UI display in MongoDB.
• Integration of RESTHeart to create a REST API for MongoDB.
• Build an Angular frontend to simplify data queries and master data maintenance.
• Agentic coding with remote and local LLMs (Claude Sonnet, Ollama Qwen) and MCP servers.
• Develop Python scripts for transforming and cleaning incoming stock market data (Pandas, scikit-learn).