{"uid":"cap_w0wpGdeCokJSjM4891clX","slug":"pixelart-withzero-xyz-pixelart-semantic-object-detection-vision-model-2cedbd74","name":"Pixelart Semantic Object Detection (Vision Model)","description":"Group deterministic regions into named animation parts such as head, body, cape, limbs, and held items. Dry-run by default; materialization is revision-guarded and replaces only the selected frame by default. Full agent guide: https://pixelart.withzero.xyz/llms.txt","url":"https://pixelart.withzero.xyz/api/v1/canvas/{id}/segment/semantic","method":"POST","headers":{},"bodySchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["frame","maxObjects","materialize"],"properties":{"hint":{"type":"string","maxLength":500,"description":"Optional context for the vision model, e.g. \"a lich king boss holding a staff\" — improves part naming and grouping."},"frame":{"type":"integer","default":0,"maximum":9007199254740991,"minimum":0,"description":"Frame to segment. Default 0 (the primary canvas)."},"maxObjects":{"type":"integer","default":6,"maximum":12,"minimum":2,"description":"Upper bound on named objects the vision model may return. Default 6."},"materialize":{"type":"boolean","default":false,"description":"When true, create one layer per detected object (largest = zIndex 0) with its cel on the target frame."}},"additionalProperties":false},"responseSchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["canvasId","frame","objects","materialized"],"properties":{"frame":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991},"objects":{"type":"array","items":{"type":"object","required":["name","pixelCount","bbox","centroid"],"properties":{"bbox":{"type":"object","required":["x","y","w","h"],"properties":{"h":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991},"w":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991},"x":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991},"y":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991}},"additionalProperties":false},"name":{"type":"string","description":"Object name chosen by the vision model (e.g. \"staff\", \"cape\", \"head\")."},"layerId":{"type":"string","format":"uuid","pattern":"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$","description":"Present when materialize=true."},"centroid":{"type":"object","required":["x","y"],"properties":{"x":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991},"y":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991}},"additionalProperties":false},"pixelCount":{"type":"integer","maximum":9007199254740991,"minimum":-9007199254740991}},"additionalProperties":false},"description":"Detected objects, largest first. Occluded parts (e.g. a staff split by a gripping hand) are grouped into one object."},"canvasId":{"type":"string","format":"uuid","pattern":"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$"},"materialized":{"type":"boolean"}},"additionalProperties":false},"example":null,"exampleRequest":null,"tags":[],"displayCostAmount":"0.02","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"registry","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.02/call","primary":{"kind":"static","protocol":"mpp","network":"tempo","amountUsd":"0.02","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"mpp","network":"tempo","amountUsd":"0.02","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_TIMy8sEUDk0ek5ybIkNgd","protocol":"mpp","methodType":"crypto","chain":"tempo","mode":"charge","costAmount":"0.02","costPer":"request","priority":0,"asset":"0x20C000000000000000000000b9537d11c60E8b50","unit":"request","depositMicros":null,"planRef":"pixelart.canvas.segment.semantic:1"}],"brandName":"pixelart.withzero.xyz","brandSlug":"pixelart-withzero-xyz-mpp","brandBaseUrl":"https://pixelart.withzero.xyz","brandDocsUrl":null,"whatItDoes":"Detects and names distinct objects in a pixel-art sprite frame using a multimodal LLM that correctly handles occlusion and painterly shading, optionally materializing each object as a separate canvas layer.","exampleAgentPrompt":"Segment frame 0 of canvas abc-123 into up to 6 named parts — hint that it's a lich king boss holding a staff — and materialize each detected object as its own layer.","exampleUseCases":[{"title":"Identify character parts for animation rigging","prompt":"I've got a pixel-art knight sprite with a shield and sword. Can you break it down into named parts like head, body, arms, legs, shield, and sword so I can rig them separately for animation?"},{"title":"Detect occluded equipment on game character","prompt":"My sprite has a archer holding a bow that's partially hidden behind their body. Use your vision to identify all the parts including the bow, quiver, and limbs even where they overlap, and give me the bounding boxes."},{"title":"Separate complex boss sprite into layer components","prompt":"I need to decompose this elaborate dragon boss into individual named parts—head, neck, wings, tail, spikes—so my animator can work on each piece independently. Please materialize each one as a separate layer."}],"resultDescription":"Returns an array of detected objects sorted largest-first, each with a model-assigned name (e.g. 'staff', 'cape', 'left-claw'), pixel count, bounding box (x, y, w, h), and centroid coordinates. When materialize=true, each object also includes a layerId UUID referencing the newly created z-ordered canvas layer containing that object's pixels.","failureModes":["Canvas ID not found — 404 with canvasId error","Frame index out of range — validation error on frame field","maxObjects outside 2–12 range — schema validation error","Vision model fails to identify distinct regions — may return fewer objects than maxObjects","hint too long (>500 chars) — validation error","Payment not included or insufficient — payment required error"],"whenToPreferThis":"Use this endpoint instead of the free /segment when the sprite has occluded parts (e.g. a staff gripped by a hand), painterly or rim-lit shading that confuses algorithmic segmentation, or when you need human-readable named labels per object for animation rigging. It costs $0.02 USDC per call but produces semantically meaningful groupings that pure algorithmic approaches cannot.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":1,"lastUsedAt":"2026-07-28T22:30:06.832Z","lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-15T12:30:22.110Z","isFirstParty":true}