The Learning Curves Database 1.1 (LCDB 1.1) shows that learning curves are less often well-behaved than previously assumed.
Statistically rigorous methods reveal significant ill-behavior in approximately 14% of learning curves, double the previous estimates.
Specific learners are identified to be more ill-behaved than others, and different feature scalings do not usually resolve this issue.
The impact of ill-behavior on tasks like learning curve fitting and model selection is evaluated, indicating significant challenges and the importance of LCDB 1.1 as a benchmark for future research.