PyGraph is a novel approach to automatically harness the power of CUDA Graphs within PyTorch.PyGraph enables wider deployment of CUDA Graphs and reduces GPU kernel parameter copy overheads.PyGraph selectively deploys CUDA Graphs based on a cost-benefit analysis.PyGraph shows substantial performance improvements over PyTorch2 in machine learning benchmarks.