Previous-Token Prediction Based LLM Near-Exact Prompt Reconstruction
A new approach for prompt reconstruction in LLMs using previous-token prediction is introduced.
Large language models (LLMs) generate text by auto-regressively sampling the next token, complicating prompt inference from outputs. This work presents a novel approach to inverting LLMs in a black-box setting, training an explicit inverse language model from scratch on data generated by the target LLM. By using previous-token prediction, it establishes a generative link that enables accurate prompt reconstruction and supports diverse outputs, outperforming previous methods across various evaluation metrics.