Feature transformation is crucial for enhancing data representation by creating new features from the original data.
Generative AI shows promise in this area but struggles with stable and error-free output generation.
Existing methods have limitations in ensuring both valid syntax and stable performance.
A new framework is proposed that combines LLMs' symbolic generation with ML's gradient optimization.
The proposed framework includes steps such as generating high-quality samples, embedding and searching for better feature transformations, distilling knowledge between LLMs, and combining ML and LLM probabilities for stable generation.
Experiments on various datasets show that this framework can improve downstream performance by 5% and reduce error cases by nearly half.
The results highlight the effectiveness and robustness of the collaborative approach.
The study also unveils interesting insights into LLMs' ability to understand original data.