A naukri.com initiative
Github
4w
253
Image Credit: Github
Finding leaked passwords with AI: How we built Copilot secret scanning
- In October 2024, Copilot secret scanning was introduced by GitHub to detect generic passwords in users’ codebases using AI.
- Copilot secret scanning is a part of GitHub Secret Protection and aims to enhance security alerts' precision by utilizing AI for analyzing context.
- The development of Copilot secret scanning underwent challenges related to unconventional file types and structures, requiring iterative improvements.
- Initial iterations involved using large language models like GPT-3.5-Turbo and developing an offline evaluation framework to validate the approach.
- Enhancements to the prompt and evaluation process were made based on feedback, and various experimentations were conducted to improve detection quality.
- The implementation involved using a combination of models, prompting strategies, and request management systems for optimal performance and efficiency.
- The mirror testing framework helped in assessing detection quality before moving Copilot secret scanning to general availability, leading to increased precision.
- Lessons learned include focusing on precision, incorporating diverse test cases, effective resource management, and collaborative innovation for continuous improvement.
- Copilot secret scanning now covers 35% of GitHub Secret Protection repositories and continues to evolve to meet user needs for robust application security.
- GitHub Secret Protection, including Copilot secret scanning, offers enterprise solutions for preventing accidental secret exposure and is available for purchase starting April 1, 2025.
Read Full Article
15 Likes
For uninterrupted reading, download the app