Adam Algorithm with Step Adaptation : научное издание

Krutikov Vladimir; Tovbis Elena; Kazakovtsev Lev

Дополнительные ссылки: eLibrary

База данных: Каталог библиотеки СФУ

Библиографическое описание: Adam Algorithm with Step Adaptation : научное издание / V. Krutikov, E. Tovbis, L. Kazakovtsev. - Текст : непосредственный // Algorithms. - 2025. - Т. 18, № 5. - ISSN 1999-4893, DOI 10.3390/a18050268.

Аннотация: Adam (Adaptive Moment Estimation) is a well-known algorithm for the first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. As shown by computational experiments, with an increase in the degree of conditionality of the problem and in the presence of interference, Adam is prone to looping, which is associated with difficulties in step adjusting. In this paper, an algorithm for step adaptation for the Adam method is proposed. The principle of the step adaptation scheme used in the paper is based on reproducing the state in which the descent direction and the new gradient are found during one-dimensional descent. In the case of exact one-dimensional descent, the angle between these directions is right. In case of inexact descent, if the angle between the descent direction and the new gradient is obtuse, then the step is large and should be reduced; if the angle is acute, then the step is small and should be increased. For the experimental analysis of the new algorithm, test functions of a certain degree of conditionality with interference on the gradient and learning problems with mini-batches for calculating the gradient were used. As the computational experiment showed, in stochastic optimization problems, the proposed Adam modification with step adaptation turned out to be significantly more efficient than both the standard Adam algorithm and the other methods with step adaptation that are studied in the work.

Год издания: 2025

Авторы: Krutikov Vladimir , Tovbis Elena , Kazakovtsev Lev

Источник: Algorithms

Выпуск: Т. 18, № 5

Держатель оригинала документа: Kemerovo State University, Reshetnev Siberian State University of Science and Technology

Количество экземпляров:

Твёрдая копия издания отсутствует в фонде библиотеки

Показать дополнительные сведения

Gradient Method with Step Adaptation : научное издание

Krutikov Vladimir и др.
2025 год

Algorithm for Scaling Variables in Minimization Methods : научное издание

Tovbis Elena и др.
2026 год

Relaxation Subgradient Algorithms with Machine Learning Procedures : научное издание

Krutikov Vladimir и др.
2022 год

Newtonian Property of Subgradient Method with Optimization of Metric Matrix Parameter Correction : научное издание

Tovbis Elena и др.
2024 год

A Family of Multi-Step Subgradient Minimization Methods : научное издание

Tovbis Elena и др.
2023 год

Biased parameter adaptation in differential evolution : научное издание

Stanovov V. и др.
2021 год

On the Convergence Rate of Quasi-Newton Methods onStrongly Convex Functions with Lipschitz Gradient : научное издание

Krutikov Vladimir и др.
2023 год

Differential evolution with linear bias reduction in parameter adaptation : научное издание

Stanovov V. и др.
2020 год

The Automatic Design of Multimode Resonator Topology with Evolutionary Algorithms : научное издание

Stanovov Vladimir V. и др.
2022 год

Application of Algorithms with Variable Greedy Heuristics for k-Medoids Problems : статья из журнала

Kazakovtsev Lev и др.
2020 год

COMPARISON OF NEURAL NETWORK TRAINING EFFICIENCY USING THE DECC ALGORITHM AND ITS HYBRID WITH ADAM… : научное издание

Bazin N. R.
2026 год

Soft island model for population-based optimization algorithms : научное издание

Akhmedova S. и др.
2018 год

Hyper-Heuristic Approach for Tuning Parameter Adaptation in Differential Evolution : научное издание

Stanovov Vladimir и др.
2024 год

Design of computational models for hydroturbine units based on a nonparametric regression… : статья из журнала

Bukhtoyarov Vladimir Viktorovich и др.
2021 год

Properties of the Quadratic Transformation of Dual Variables : научное издание

Krutikov Vladimir и др.
2023 год

Crossover Rate Sorting in Adaptive Differential Evolution : научное издание

Stanovov Vladimir и др.
2023 год

Библиотечно-издательский комплекс СФУ

Adam Algorithm with Step Adaptation : научное издание
статья из журнала

Похожие публикации