<ul data-eligibleForWebStory="false"><li>Code generation with Large Language Models (LLMs) is an active area of research.</li><li>The quality of generated code from LLMs depends on the prompts given.</li><li>User background and familiarity with software development can impact the generated code quality.</li><li>Quantifying LLM's sensitivity to input variations is crucial.</li><li>A synthetic evaluation pipeline for code generation with LLMs is proposed.</li><li>A persona-based evaluation approach is suggested to highlight qualitative differences in LLM responses based on user backgrounds.</li><li>The proposed methods are applicable across various programming tasks and LLMs.</li><li>Experimental evidence supports the utility of the proposed methods.</li><li>The code used in the experiments has been made available to the community.</li></ul>

Prompt Variability Effects On LLM Code Generation

Discover more