{"uid":"cap_42Js2IATKB93KadbMQp9s","slug":"ai-sycophancy-detector-25170608","name":"AI Sycophancy Detector","description":"Compares term-frequency cosine similarity between a leading user position and paired independent/conditioned responses, then reports the positive similarity shift. It is a reproducible surface-form signal, not proof of agreement, truthfulness, motive or sycophancy; paraphrases and legitimate corrections can evade or trigger it.","url":"https://aialign.halowerk.com/v1/sycophancy-detect","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"scenarios":{"type":"array","items":{"type":"object","required":["scenario_id","leading_position","independent_response","conditioned_response"],"properties":{"scenario_id":{"type":"string","maxLength":128,"minLength":1},"leading_position":{"type":"string","maxLength":20000,"minLength":1},"conditioned_response":{"type":"string","maxLength":50000,"minLength":1},"independent_response":{"type":"string","maxLength":50000,"minLength":1}},"additionalProperties":false},"maxItems":5000,"minItems":1},"shift_threshold":{"type":"number","maximum":1,"minimum":0}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.004","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.004/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.004","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.004","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_isgjJP4FWuiDDzf00XIFo","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.004","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Detects sycophantic drift in AI responses by comparing cosine similarity between a leading user position and paired independent vs. conditioned responses, reporting the positive similarity shift.","exampleAgentPrompt":"Can you check if our chatbot is being sycophantic? Here are three scenarios — each has a leading position the user stated, the bot's original independent response, and its response after the user pushed back. Flag any where the similarity shift exceeds 0.3.","exampleUseCases":[{"title":"LLM red-teaming for sycophancy","prompt":"I want to red-team our fine-tuned model for yes-man behavior. I've got 10 scenarios where we gave the model a leading user opinion, recorded its initial response, then re-prompted with the user reiterating their view. Can you run the sycophancy detection on all of them with a shift threshold of 0.25 and tell me which ones show problematic drift?"},{"title":"Chatbot QA before product launch","prompt":"Before we ship our support bot, I need to verify it doesn't just agree with whatever the customer says. Here's a scenario — the leading position is 'your product is definitely broken', the independent response is what the bot said before seeing that, and the conditioned response is what it said after. Does the similarity shift exceed 0.4?"},{"title":"Comparing two model checkpoints","prompt":"We have two versions of our model and want to know which one is more prone to sycophancy under user pressure. I have paired independent and conditioned responses for each — can you compute the cosine similarity shift for both and tell me which model drifts more toward the user's leading position?"}],"resultDescription":"Returns the cosine similarity shift (positive delta) between the independent and conditioned responses relative to the leading user position for each scenario, along with whether the shift exceeds the configured threshold. Results are surface-form signals based on term-frequency cosine similarity — not ground-truth judgments of agreement, truthfulness, or intent.","failureModes":["Empty or missing required fields (scenario_id, leading_position, independent_response, conditioned_response) return validation errors","Exceeding the 5000-scenario batch limit or 50000-character text limits causes rejection","Threshold value outside [0,1] range is invalid","Paraphrased sycophantic responses may evade detection since the method is surface-form","Legitimate corrections by the AI may falsely trigger a high shift score","Very short texts produce unreliable cosine similarity estimates"],"whenToPreferThis":"Choose this endpoint when you need a fast, reproducible, quantitative signal for sycophantic drift in AI-generated responses — especially when comparing paired independent vs. conditioned outputs at scale. It is well-suited for automated QA pipelines, model evaluation benchmarks, and red-teaming workflows where you want a consistent numeric metric rather than a subjective human judgment. Prefer alternatives (e.g. human evaluation, semantic entailment models) when you need to distinguish paraphrase-based agreement from surface-form shift, or when verifying genuine factual correctness.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-14T00:30:37.141Z","isFirstParty":false}