GRADSTOP: Early Stopping of Gradient Descent via Posterior Sampling presented at ECAI 2025
The scope of the conference covered core AI topics including machine learning, computer vision, natural language processing, knowledge representation and reasoning, planning and search, as well as multidisciplinary work. Katsiaryna and Arash are delighted that their work was accepted for an oral presentation, and they got a chance to receive feedback from AI community.
When discussing their research, Katsiaryna stated “we provided a theoretically justified method for early stopping that does not need a validation set, allows using all available data for training, and relies only on the gradients provided by gradient descent algorithm. The method is beneficial in data scarce setting, for example, in processing of medical data."
From the abstract "Machine learning models are often learned by minimising a loss function on the training data using a gradient descent algorithm. These models often suffer from overfitting, leading to a decline in predictive performance on unseen data. A standard solution is early stopping using a hold-out validation set, which halts the minimisation when the validation loss stops decreasing. However, this hold-out set reduces the data available for training. This paper presents GRADSTOP, a novel stochastic early stopping method that only uses information in the gradients, which are produced by the gradient descent algorithm “for free.” Our main contributions are that we estimate the Bayesian posterior by the gradient information, define the early stopping problem as drawing sample from this posterior, and use the approximated posterior to obtain a stopping criterion. Our empirical evaluation shows that GRADSTOP achieves a small loss on test data and compares favourably to a validation-set-based stopping criterion. By leveraging the entire dataset for training, our method is particularly advantageous in data-limited settings, such as transfer learning. It can be incorporated as an optional feature in gradient descent libraries with only a small computational overhead. The source code is available at https://github.com/edahelsinki/gradstop."
Link to the paper: https://ebooks.iospress.nl/doi/10.3233/FAIA251043
More information about ECAI 2025 can be found https://ecai2025.org/
Read more news
Emotions help explain why people click phishing emails
People are currently taught to recognise phishing attempts in ways that may be counterproductive. Researchers call for information security training to take emotions into account more effectively.
New podcast explores Finnish contributions to artificial intelligence
In Aalto University's podcast “Ja ihminen loi älyn” (And humankind created intelligence), researchers Arno Solin ja Eric Malmi interview pioneers, entrepreneurs and experts about the AI field's history, disruptive consequences and the probability of humanity's destruction.
AI model captures how humans read, paving the way to personalised text and better augmented reality
Researchers now understand not just how our eyes move when we read, but also how we build meaning from text. The new model could hold the key to developing highly customisable applications that adapt to different types of reader.