Large Language Model Experts in Dortmund

• Delivered MVP-focused full-stack architecture for a health-sector client: Vite/React frontend, backend services on Google Cloud Run, and Supabase for database plus IAM/authentication.
• Supported product requirements engineering and prioritized cost-aware workload placement, implementing browser-side/edge computation where feasible before moving logic to backend services.

• Architected and deployed end-to-end machine learning pipelines across classification and prediction datasets, ensuring robustness and reproducibility through MLOps best practices.
• Contributed directly to LLM model output accuracy improvement by designing and engineering specialised prompts grounded in end-to-end ML and SciML pipeline logic.
• Developed training data for large language models by formulating coding problems that models could not resolve and subsequently documenting the correct solutions.

• Developed and optimised structured support workflows and evaluation procedures, applying consistent quality standards across high-volume operational tasks.
• Monitored performance metrics to identify systemic issues and drive targeted improvements — a skill directly transferable to LLM performance metric analysis.
• Managed escalations and maintained high accuracy and satisfaction standards in a fully asynchronous, remote-first environment.

Conventional Paradigm Test (CPT) – AI Evaluation & LLM Research

Development of an experimental evaluation approach to examine “paradigmatic closure” in Large Language Models — that is, the question of how far LLMs can recognize the basic assumptions, values, and limits of the paradigms within which they generate answers.

Design and testing of an additional approach to classic AI benchmarks that does not primarily measure factual correctness or task performance, but instead examines a model’s ability to recognize alternative perspectives, make implicit assumptions visible, and reflect on the limits of its own answer or interpretation framework.

Focus areas: development of evaluation criteria and test questions · LLM evaluation and comparative model analysis · prompt and response analysis · qualitative classification of model answers · study of epistemic compression and value leakage · benchmark and literature research · development of structured assessment and analysis methods

As part of CPT, existing AI evaluation approaches and benchmarks were analyzed, and a minimalist test protocol was developed that classifies model answers by response patterns such as DIRECT, CLARIFY, PLURALIST, REFUSE, and META-AWARE. TruthfulQA was used as the basis for experimental application and comparison with existing reference answers.

Technologies & Methods: Large Language Models (LLMs) · Generative AI · Prompt Engineering · AI Evaluation · TruthfulQA · Benchmark Analysis · Human-in-the-Loop Evaluation · Qualitative Content Analysis · Research & Literature Review

• Development of an automated job-scouting workflow for the aggregation and LLM-based evaluation of job postings from RSS feeds and APIs.
• Development of a Telegram bot for voice and text messages, featuring LLM-supported processing, summarization, and structured JSON output.
• Implementation of a personal task-planning assistant using n8n, OpenRouter, and Supabase for automated daily and knowledge organization.
• Development of LLM chatbots for structured knowledge retrieval utilizing prompt constraints, guardrails, and forced output formats.

• Technical lead of a team developing agentic chatbot solutions (React, TypeScript, Python, FastAPI)
• Architecture design for multi-LLM dialog systems - focus on maintainability, UX, and autonomous execution
• Stakeholder alignment, CI/CD processes, and AI integration at enterprise level

• Setting up new desktop computers
• Setting up a VPS for automation, databases and chat interface
• Connecting to Azure AI Foundry
• Developing various n8n workflows for email, social media and document management
• Implementing a QA system (backups, error handling, HITL)
• Setting up and optimizing an inbound voice agent

• Initial assessment of requirements based on a business value prioritization framework
• Identification of issues as well as requirement gathering and evaluation using UML, BPMN, and design thinking methods for iterative requirements analysis through interviews and workshops
• Use of user story mapping in Miro to visualize and align functional requirements (e.g. correct transmission of all application data and attachments to the specialist system) as well as non-functional requirements (e.g. complete and verifiable deletion of an applicant's data) with stakeholders
• Proactive stakeholder management of internal and external stakeholders from public authorities, business units, organizations, and companies
• Preparation of status reports to communicate project progress and upcoming tasks transparently
• Responsibility for a REST-based integration solution (middleware) for secure data exchange between core systems and external specialist applications; ensuring stability and performance in day-to-day operations
• Support for Product Owners in prioritizing backlog items and in product discovery
• Communication of planning to internal and external stakeholders as well as interim assumption of Product Owner tasks and responsibilities during a staff change

• Conduct structured evaluation of LLM outputs using Content Review Standards (CRS) and AI safety frameworks.
• Assess responses across high-risk domains including violence and criminal facilitation.
• Assess responses across high-risk domains including hate speech and harassment.
• Assess responses across high-risk domains including suicide and self-harm.
• Assess responses across high-risk domains including regulated advice (medical, legal, financial).
• Assess responses across high-risk domains including misinformation and fabricated claims.
• Assess responses across high-risk domains including defamation and intellectual property.
• Assess responses across high-risk domains including child safety and sexual exploitation.
• Assess responses across high-risk domains including political and sensitive content.
• Apply youth-protection and age-appropriateness guidelines to prevent unsafe facilitation or restricted substance guidance.
• Classify prompts as adversarial, borderline, or benign based on contextual intent and risk analysis.
• Evaluate model behavior types including correct refusal, partial refusal, over-refusal, under-refusal, improper compliance, and ignorance-based outputs.
• Identify policy misapplications and user-intent misinterpretation patterns.
• Designed structured adversarial and borderline multi-turn conversation flows to stress-test AI boundary enforcement and reasoning stability.
• Identified failure modes including hallucination, unsafe compliance, excessive refusal, contextual drift, and inconsistent safety logic.
• Applied a structured four-dimension evaluation rubric covering accuracy & safety, relevance & completeness, clarity & structure, and tone & appropriateness.
• Provided structured feedback supporting supervised fine-tuning and reinforcement learning from human feedback processes.
• Rewrote unsafe or misaligned outputs into compliant, accurate, and helpful responses.
• Performed Persian ↔ English translation and translation validation of AI-generated content.
• Assessed semantic accuracy, contextual consistency, and safety alignment across languages.
• Identified mistranslations, cultural nuance issues, and cross-lingual policy inconsistencies.
• Recognized with the Above & Beyond Award – Q3 2025 for exceeding quality standards and embracing innovation.

• Setting up and consulting a startup for organizational development (AI-supported)
• Vision and mission workshops
• Marketing roadmap and process design
• Development of sales processes (sales funnel, lead generation)
• Brand building and social media omnichannel strategies
• Content generation for audio, video, and posts (AI-supported)
• Agile consulting and SAFe implementation
• Atlassian consulting and setup of Confluence and Jira (workflows, dashboards, reporting)
• Setup of a podcast and video studio including live streaming setup
• Selection and implementation of hardware and software
• Website creation and brand building
• CMS implementation based on Typo3
• Consulting in sales, marketing, and social media
• Supporting companies in the efficient use of AI technologies
• Strategy development and consulting on implementing AI strategies
• Change management and supporting teams in AI adoption, including training
• Technical expertise in machine learning, data processing, and large language models (LLMs) like Llama, Claude, and OpenAI GPT
• Selection, fine-tuning, and integration of LLMs into business processes
• Consulting on ethics and governance for responsible AI use
• Conducting tailored training for executives and teams
• AI analysis of various large language models (Llama, OpenAI, Mistral)
• Creation and training of custom GPTs focused on organizational development (product vision, OKR, agile maturity)
• Development of a chatbot on agile maturity
• Development of an AI-generated survey to assess organizational agility
• Setup of local offline GPTs with a focus on data protection and GDPR compliance
• Conducting AI workshops for SMEs and mid-sized companies