{"uid":"cap_PpY1xbo7twW1gw--YRRit","slug":"knowledge-topic-training-evaluation-text-overlap-detector-a66c9165","name":"Knowledge Topic Training-Evaluation Text Overlap Detector","description":"Knowledge topic - Training evaluation text overlap: Training evaluation text overlap for Knowledge topic. Up to 200 supplied labelled text records. Local dataset tools, evaluation and small naive-Bayes baselines; no hosted model or foundation-model training. One of 25 profiles or calculations across 20 shared operations in this sector. 0.01 USDC includes one bounded training evaluation text overlap deliverable.","url":"https://market.datapackvibe.com/x402/assistant-sector-ml-knowledge-topic-leakage?utm_source=zero.xyz","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"records":{"type":"array","items":{"type":"object","required":["id","text","actual"],"properties":{"id":{"type":"string","default":"","maxLength":80,"minLength":1},"text":{"type":"string","default":"","maxLength":1000},"actual":{"enum":["setup","usage","troubleshooting","reference"],"type":"string"}},"additionalProperties":false},"default":[{"id":"example-0-0","text":"setup sample document number 0 for this supplied dataset.","actual":"setup"},{"id":"example-0-1","text":"setup sample document number 1 for this supplied dataset.","actual":"setup"},{"id":"example-0-2","text":"setup sample document number 2 for this supplied dataset.","actual":"setup"},{"id":"example-1-0","text":"usage sample document number 0 for this supplied dataset.","actual":"usage"},{"id":"example-1-1","text":"usage sample document number 1 for this supplied dataset.","actual":"usage"},{"id":"example-1-2","text":"usage sample document number 2 for this supplied dataset.","actual":"usage"},{"id":"example-2-0","text":"troubleshooting sample document number 0 for this supplied dataset.","actual":"troubleshooting"},{"id":"example-2-1","text":"troubleshooting sample document number 1 for this supplied dataset.","actual":"troubleshooting"},{"id":"example-2-2","text":"troubleshooting sample document number 2 for this supplied dataset.","actual":"troubleshooting"},{"id":"example-3-0","text":"reference sample document number 0 for this supplied dataset.","actual":"reference"},{"id":"example-3-1","text":"reference sample document number 1 for this supplied dataset.","actual":"reference"},{"id":"example-3-2","text":"reference sample document number 2 for this supplied dataset.","actual":"reference"}],"maxItems":200,"minItems":2},"evaluation_texts":{"type":"array","items":{"type":"string","default":"","maxLength":1000},"default":["setup sample document number 0 for this supplied dataset.","A separate supplied document."],"maxItems":200,"minItems":1}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.01","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.01/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.01","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.01","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_reifTak7KyhiNO1b9-y2i","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.01","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Detects text overlap between training and evaluation sets for knowledge topic classification (setup/usage/troubleshooting/reference) using a naive-Bayes baseline, returning leakage metrics.","exampleAgentPrompt":"Check whether my evaluation texts overlap with my training records for a knowledge topic classifier — I have 12 labelled training documents (split across setup, usage, troubleshooting, and reference categories) and 2 evaluation texts I want to test for leakage.","exampleUseCases":[{"title":"Detect benchmark contamination in docs classifier","prompt":"I'm building a knowledge base classifier with categories setup, usage, troubleshooting, and reference. Can you check if any of my evaluation texts — like 'How do I reset my password?' — appear in or are suspiciously similar to my 50 labelled training records before I report accuracy numbers?"},{"title":"Validate train-test split for support ticket triage","prompt":"I have 150 labelled support tickets tagged as setup, usage, troubleshooting, or reference, and a separate set of 30 held-out evaluation texts. Please run a training-evaluation text overlap check to make sure none of my eval examples leaked into the training set."},{"title":"Audit knowledge topic dataset integrity before publishing","prompt":"Before I publish my knowledge topic dataset, can you run an overlap analysis between my 80 labelled training examples across setup, usage, troubleshooting, and reference categories, and my 20 evaluation texts, to flag any data leakage issues?"}],"resultDescription":"Returns an overlap/leakage report quantifying how much textual similarity exists between the supplied labelled training records and the evaluation texts, including which pairs overlap and aggregate leakage metrics — all computed locally using naive-Bayes baseline methods without any hosted model.","failureModes":["Fewer than 2 training records provided — minimum is 2","Fewer than 1 evaluation text provided — minimum is 1","More than 200 records or evaluation texts supplied — exceeds maxItems limit","Record text exceeds 1000 character limit — truncation or rejection","Invalid category label outside enum (setup/usage/troubleshooting/reference) — validation error","Missing required fields (id, text, actual) in a record — schema rejection","Payment not included or insufficient — x402 payment required at $0.01 USDC"],"whenToPreferThis":"Choose this endpoint when you need a lightweight, cheap ($0.01) check for training-evaluation text overlap in a four-class knowledge topic classification setting (setup/usage/troubleshooting/reference) without needing a hosted model or foundation model. Best for small datasets up to 200 records where data integrity validation is the primary goal, not model training.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-15T18:48:47.775Z","isFirstParty":false,"canonicalSlug":"knowledge-topic-training-evaluation-text-overlap-detector-a66c9165"}