Large language models (LLMs) have the potential to produce malicious code, posing risks to assessment infrastructure.
SandboxEval is a test suite designed to evaluate the security and confidentiality of test environments for LLM-generated code.
The suite focuses on vulnerabilities related to sensitive information exposure, filesystem manipulation, external communication, and other dangerous operations.
By deploying SandboxEval, developers can gain valuable insights to improve assessment infrastructure and identify risks associated with LLM execution.