Stochastic differential equations (SDEs) are used for modeling in various fields.A nonparametric framework is presented to learn drift and diffusion terms in SDE systems.The framework is effective in systems with singular covariance matrix and high dimensionality reduction.The framework is demonstrated through examples in physics, biology, and collective dynamics.