A novel algorithm is presented for detecting fast moving celestial objects within star fields.The algorithm enhances neural networks by transforming them into physical-inspired neural networks.It leverages the point spread function and observational mode as priors for accurate detection.The algorithm is effective in detecting fast moving celestial objects across different observational modes.