Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

LLM-Redactor: An Empirical Evaluation of Eight Techniques for Privacy-Preserving LLM Requests

About

Coding agents and LLM-powered applications routinely send potentially sensitive content to cloud LLM APIs where it may be logged, retained, used for training, or subpoenaed. Existing privacy tooling focuses on network-level encryption and organization-level DLP, neither of which addresses the content of prompts themselves. We present a systematic empirical evaluation of eight techniques for privacy-preserving LLM requests: (A) local-only inference, (B) redaction with placeholder restoration, (C) semantic rephrasing, (D) Trusted Execution Environment hosted inference, (E) split inference, (F) fully homomorphic encryption, (G) secret sharing via multi-party computation, and (H) differential-privacy noise. We implement all eight (or a tractable research-stage subset where deployment is not yet feasible) in an open-source shim compatible with MCP and any OpenAI-compatible API. We evaluate the four practical options (A, B, C, H) and their combinations across four workload classes using a ground-truth-labelled leak benchmark of 1,300 samples with 4,014 annotations. Our headline finding is that no single technique dominates: the combination A+B+C (route locally when possible, redact and rephrase the rest) achieves 0.6% combined leak on PII and 31.3% on proprietary code, with zero exact leaks on PII across 500 samples. We present a decision rule that selects the appropriate option(s) from a threat-model budget and workload characterisation. Code, benchmarks, and evaluation harness are released at https://github.com/jayluxferro/llm-redactor.

Justice Owusu Agyemang, Jerry John Kponyo, Elliot Amponsah, Godfred Manu Addo Boakye, Kwame Opuni-Boachie Obour Agyekum• 2026

Related benchmarks

TaskDatasetResultRank
Privacy Leakage AnalysisWL2 Secrets
Combined Leak Rate0.00e+0
13
Privacy Leakage EvaluationWL1 PII
Combined Leak Rate0.00e+0
13
Privacy Leakage EvaluationWL3 Implicit
Combined Leak Rate0.00e+0
13
Privacy Leakage EvaluationWL4 Code
Combined Leak Rate0.00e+0
13
Privacy Leakage AnalysisWL1 PII
Exact Leak Rate0.00e+0
8
Privacy Leakage AnalysisWL3 Implicit
Exact Leak Rate0.00e+0
8
Privacy Leakage AnalysisWL4 Code
Exact Leak Rate12.3
8
Showing 7 of 7 rows

Other info

Follow for update