Document Management System Expert in Dortmund

Conventional Paradigm Test (CPT) – AI Evaluation & LLM Research

Development of an experimental evaluation approach to examine “paradigmatic closure” in Large Language Models — that is, the question of how far LLMs can recognize the basic assumptions, values, and limits of the paradigms within which they generate answers.

Design and testing of an additional approach to classic AI benchmarks that does not primarily measure factual correctness or task performance, but instead examines a model’s ability to recognize alternative perspectives, make implicit assumptions visible, and reflect on the limits of its own answer or interpretation framework.

Focus areas: development of evaluation criteria and test questions · LLM evaluation and comparative model analysis · prompt and response analysis · qualitative classification of model answers · study of epistemic compression and value leakage · benchmark and literature research · development of structured assessment and analysis methods

As part of CPT, existing AI evaluation approaches and benchmarks were analyzed, and a minimalist test protocol was developed that classifies model answers by response patterns such as DIRECT, CLARIFY, PLURALIST, REFUSE, and META-AWARE. TruthfulQA was used as the basis for experimental application and comparison with existing reference answers.

Technologies & Methods: Large Language Models (LLMs) · Generative AI · Prompt Engineering · AI Evaluation · TruthfulQA · Benchmark Analysis · Human-in-the-Loop Evaluation · Qualitative Content Analysis · Research & Literature Review

• Setting up new desktop computers
• Setting up a VPS for automation, databases and chat interface
• Connecting to Azure AI Foundry
• Developing various n8n workflows for email, social media and document management
• Implementing a QA system (backups, error handling, HITL)
• Setting up and optimizing an inbound voice agent

• Setting up and consulting a startup for organizational development (AI-supported)
• Vision and mission workshops
• Marketing roadmap and process design
• Development of sales processes (sales funnel, lead generation)
• Brand building and social media omnichannel strategies
• Content generation for audio, video, and posts (AI-supported)
• Agile consulting and SAFe implementation
• Atlassian consulting and setup of Confluence and Jira (workflows, dashboards, reporting)
• Setup of a podcast and video studio including live streaming setup
• Selection and implementation of hardware and software
• Website creation and brand building
• CMS implementation based on Typo3
• Consulting in sales, marketing, and social media
• Supporting companies in the efficient use of AI technologies
• Strategy development and consulting on implementing AI strategies
• Change management and supporting teams in AI adoption, including training
• Technical expertise in machine learning, data processing, and large language models (LLMs) like Llama, Claude, and OpenAI GPT
• Selection, fine-tuning, and integration of LLMs into business processes
• Consulting on ethics and governance for responsible AI use
• Conducting tailored training for executives and teams
• AI analysis of various large language models (Llama, OpenAI, Mistral)
• Creation and training of custom GPTs focused on organizational development (product vision, OKR, agile maturity)
• Development of a chatbot on agile maturity
• Development of an AI-generated survey to assess organizational agility
• Setup of local offline GPTs with a focus on data protection and GDPR compliance
• Conducting AI workshops for SMEs and mid-sized companies

• Responsible for carrying out sample approval processes