ETL Experts in Cologne

Architected and maintained data products for renewable energy operations, covering wind turbine, grid-meter, and weather data. Built scalable ETL/ELT pipelines in Azure Databricks using Delta Lake (bronze/silver/gold layers) and processed data in various formats, including structured and semi-structured data. Contributed to a data quality framework supporting table and column documentation, outlier detection, and completeness metrics across all datasets within a data product. In addition, implemented a DORA KPI Databricks dashboard used across all data products. Optimized CI/CD processes in Azure DevOps to streamline deployment across development, test, and production environments.

Technology stack: Azure Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Azure Data Lake, APIs, Dremio, Azure DevOps, YAML, Git, Databricks Workflows, Application Insights, Terraform, OpenAI API, Codex, LLM-assisted workflows

• Integration of medical modalities (including ultrasound) into the existing IT infrastructure (DICOM, HL7) – put into operation within the planned timeframe.
• Administration and optimization of PACS systems for efficient archiving and distribution of radiology image data across multiple locations.
• Ensuring consistent data quality and seamless interoperability in data exchange between HIS, RIS, and PACS.
• Close collaboration with medical staff to analyze and digitally optimize clinical workflows.
• Requirements management and test coordination when implementing clinical requirements in complex IT structures.

• Analysis of business requirements
• Integration of different data sources such as ERP systems, production systems, and external data sources
• Implementation of load processes and processing logic with MSSQL
• Data modeling and optimization of data models
• Setting up data quality management incl. data profiling with dynamic programming
• Support in designing and establishing data governance, especially in the areas of data quality management and data protection
• Support in developing BI solutions with Tableau to help decision-making processes

Industry: digital sovereignty, public sector

Content:

• Project steering (teams: consulting, development, testing, deployment/operations)

Products and standards:

• Jira, Confluence, Asana, Miro, Mural
• Open Source, Keycloak, Open Xchange, ownCloud

• Independently initiated and deployed automated validation workflows using Python, cutting manual processing by 58% and improving efficiency
• Developed a machine learning model for synthetic defect generation, reducing downtime and production costs; deployed locally and via Databricks and Azure AI Factory
• Utilized a small dataset of image data from the production lines and extended this dataset with training on models like cycleGAN and pix2pix
• Built and optimized the Linux-based development environment for training 3D models; maintained reproducibility via GitHub
• Presented technical insights to cross-functional teams (engineers, QA, project managers), ensuring alignment of ML solutions with operational needs

• Developed and enhanced ETL loading jobs with IBM DataStage and optimized SQL in an IBM DB2 environment as part of the Agree21 data migration
• Analyzed data quality and developed test procedures
• Created PowerShell scripts and documented GIT deployment processes
• Technologies: RedHat Linux, IBM DB2 with DBVisualizer, IBM InfoSphere DataStage 11.7, JIRA, TortoiseGIT, TortoiseSVN, PowerShell scripts

• Designed and led the development of an automated short-video generation system.
• Built a scalable AWS backend using Step Functions, Lambda, S3, ECS Fargate, and DynamoDB.
• Developed video rendering with OpenCV and FFMPEG; ensured maintainable Python code.
• Supervised and mentored a Python developer and trained the client in AI workflows.
• Decreased end-to-end production time from hours to minutes.
• Created a modular, extensible architecture designed to support future AI models.

• Implementation of custom reporting solutions for financial KPIs aligned with specific business requirements

• Development of a machine learning application that achieved a 250% performance improvement

• Data Architect "Production-Oriented Quality Assurance" (03/2024–09/2024):

• Design and implementation of an analytics platform to detect quality deviations in manufacturing

• Build of a scalable data lakehouse architecture on Databricks in combination with SAP ERP data via SAP Datasphere

• Close collaboration with the SAP team to harmonize bill of materials and order data

• Visualization of KPIs to support shopfloor management

• Lead Data Engineer "Sales Performance Monitoring" (08/2023–12/2023):

• Design and implementation of a Databricks-based platform for analyzing sales figures and promotion effects

• Integration of SAP SD data via SAP BW/4HANA

• Use of Azure DevOps to orchestrate ETL jobs and deploy workflows

• Technical Project Lead "Cloud Migration & Data Strategy" (02/2023–05/2023):

• Migration of a heterogeneous data warehouse stack to a modern cloud architecture on Azure with Databricks as the central processing platform

• Development of a governance-compliant data architecture to integrate SAP financial data and non-SAP sources

Technologies: Databricks, Delta Lake, Python, SAP Datasphere, Azure DevOps, PowerBI, SQL

• Developed automated ETL pipelines with AWS Glue and Athena to ensure consistent data quality and governance requirements
• Implemented validation, anonymization, and encryption measures for data in compliance with GDPR
• Optimized cloud costs by introducing FinOps practices and increased transparency for business units
• Monitored performance, performed root cause analyses, and ensured adherence to SLAs
• Supported data and solution architects in building scalable data models for ML and analytics scenarios

• Acted as the primary bridge between Legal, Engineering, and Business units to ensure zero compliance violations while maintaining product velocity.

• Led the development of a GDPR-compliant AI aggregator platform, managing a roadmap that balances legal constraints with aggressive feature delivery.

• Scaled the engineering team from 4 to 9 developers, establishing hiring protocols and technical onboarding processes to support rapid product iteration.

• Boosted the development process by introducing structured sprint cycles and backlog refinement, resulting in a 20% reduction in feature delivery time.

• Architected and prototyped agentic AI workflows with n8n and RAG pipelines on Langchain.

• Technical project management, requirements management, and design to guide the data team in developing a predictive maintenance model for DB InfraGO's maintenance planning.
• Statistical modeling and analysis programming with R Studio for fault analysis in preventive maintenance: multivariate modeling using quasi-Poisson, negative binomial, lasso, offset, splines, RandomForest.
• Implementation of various R Shiny dashboards.
• Sparring partner and requirements management for data engineering, data modeling, and ETL pipeline in Tableau Prep.

• In the citizens' and business customer portal of the Customs Administration, airlines submit applications for departures and landings every month

• These applications are forwarded to the caseworkers via SOAP web service and, after review and processing, any queries or notifications are returned via SOAP web service

• PL/SQL programming of SOAP webservices client requests for HTTP/XML message exchange between the BuG portal and the caseworkers

• New creation and change requests for Forms and Reports 12 modules

• Forms applications start and read SOAP webservice requests via PL/SQL packages, process them and send Reports 12 documents to the airlines

• Extraction and transformation of relevant data from the SAP system developed for the bank and consolidation into the existing reporting environment
• Integration of the data into the existing BI model (extending the star schema to a galaxy schema) for generating the funding committee report and the business development report
• Ensuring data quality
• Knowledge transfer to department staff (SAP, SQL database, Power Pivot, DAX)

• Improvement of Tagetik
• Other tasks