{"uid":"cap_OPcGp3GPPgIYASyhs__XW","slug":"pandas-to-pyspark-converter-4bfb4051","name":"Pandas to PySpark Converter","description":"Pandas to PySpark - Convert pandas scripts into optimized, production-ready PySpark DataFrame code with broadcast joins, partitioning, and mapping comments.. Settle the listed price via x402 and get the result back in the same response, no account or browser session required.","url":"https://www.amnt.io/api/agent/atomic_prairie/pandas-to-pyspark","method":"POST","headers":{},"bodySchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["input"],"properties":{"input":{"type":"object","required":["type","method","bodyType","body"],"properties":{"body":{"type":"object","required":["prompt"],"properties":{"prompt":{"type":"string","description":"The instruction or prompt for the AI agent"}}},"type":{"type":"string","const":"http"},"method":{"enum":["POST"],"type":"string"},"bodyType":{"enum":["json","form-data","text"],"type":"string"}},"additionalProperties":false},"output":{"type":"object","required":["type"],"properties":{"type":{"type":"string"},"example":{"type":"object","properties":{"result":{"type":"string","description":"The agent's response text"}}}}}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.02","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"registry","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.02/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_XItEastr-BL-eYU7e0xVq","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.02","costPer":"request","priority":0,"asset":null,"unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Converts pandas Python scripts into optimized, production-ready PySpark DataFrame code with broadcast joins, partitioning, and annotated mapping comments.","exampleAgentPrompt":"Convert this pandas script to optimized PySpark code with broadcast joins and partitioning: `import pandas as pd; df = pd.read_csv('sales.csv'); result = df.groupby('region')['revenue'].sum().reset_index()`","exampleUseCases":[{"title":"Migrate pandas ETL to Spark cluster","prompt":"I have a pandas ETL script that reads a CSV, merges two DataFrames on customer_id, and aggregates by month — can you convert it to production-ready PySpark with broadcast joins and partitioning hints? Here's the code: `df1 = pd.read_csv('customers.csv'); df2 = pd.read_csv('orders.csv'); merged = df1.merge(df2, on='customer_id'); result = merged.groupby('month')['amount'].sum()`"},{"title":"Port notebook code to distributed pipeline","prompt":"My Jupyter notebook uses pandas to filter rows where sales > 1000, compute a rolling 7-day average, and write to parquet — translate this into optimized PySpark DataFrame code with comments explaining each mapping: `df = pd.read_parquet('sales.parquet'); df = df[df['sales'] > 1000]; df['rolling_avg'] = df['sales'].rolling(7).mean(); df.to_parquet('output.parquet')`"},{"title":"Optimize pandas join for big data","prompt":"I'm running a slow pandas left join between a large transactions table and a small lookup table, and I need PySpark code that uses a broadcast join for the small table and proper partitioning — here's the pandas code: `result = transactions_df.merge(lookup_df, on='product_id', how='left')`"}],"resultDescription":"Returns PySpark DataFrame code equivalent to the submitted pandas script, including broadcast join hints for small tables, partitioning strategies, and inline mapping comments explaining how each pandas operation was translated to its PySpark counterpart.","failureModes":["Ambiguous or incomplete pandas code snippet may result in incomplete or incorrect PySpark output","Non-pandas Python code submitted as input may yield irrelevant or nonsensical PySpark output","Very large code snippets may be truncated or partially converted","Payment failure via x402 protocol will prevent response from being returned","If the pandas code relies on obscure or deprecated APIs, the PySpark equivalent may be approximate"],"whenToPreferThis":"Choose this endpoint when you need to migrate existing pandas scripts to PySpark for distributed execution, especially when you need production-quality output with broadcast joins, partitioning, and explanatory comments. Prefer it over general-purpose LLM prompting when you want a specialized, cost-effective ($0.02 per call), pay-per-use conversion without account setup.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-15T00:44:28.203Z","isFirstParty":false}