PNKH-B
PNKH-B: a projected Newton-Krylov method for large-scale bound-constrained optimization. We present PNKH-B, a projected Newton-Krylov method for iteratively solving large-scale optimization problems with bound constraints. PNKH-B is geared toward situations in which function and gradient evaluations are expensive, and the (approximate) Hessian is only available through matrix-vector products. This is commonly the case in large-scale parameter estimation, machine learning, and image processing. In each iteration, PNKH-B uses a low-rank approximation of the (approximate) Hessian to determine the search direction and construct the metric used in a projected line search. The key feature of the metric is its consistency with the low-rank approximation of the Hessian on the Krylov subspace. This renders PNKH-B similar to a projected variable metric method. We present an interior point method to solve the quadratic projection problem efficiently. Since the interior point method effectively exploits the low-rank structure, its computational cost only scales linearly with respect to the number of variables, and it only adds negligible computational time. We also experiment with variants of PNKH-B that incorporate estimates of the active set into the Hessian approximation. We prove the global convergence to a stationary point under standard assumptions. Using three numerical experiments motivated by parameter estimation, machine learning, and image reconstruction, we show that the consistent use of the Hessian metric in PNKH-B leads to fast convergence, particularly in the first few iterations. We provide our MATLAB implementation at url{https://github.com/EmoryMLIP/PNKH-B}.
Keywords for this software
References in zbMATH (referenced in 4 articles , 1 standard article )
Showing results 1 to 4 of 4.
Sorted by year (- Heaton, Howard; Fung, Samy Wu; Lin, Alex Tong; Osher, Stanley; Yin, Wotao: Wasserstein-based projections with applications to inverse problems (2022)
- Heaton, Howard; Wu Fung, Samy; Gibali, Aviv; Yin, Wotao: Feasibility-based fixed point networks (2021)
- Kan, Kelvin; Fung, Samy Wu; Ruthotto, Lars: PNKH-B: a projected Newton-Krylov method for large-scale bound-constrained optimization (2021)
- Wang, Chao; Tao, Min; Nagy, James G.; Lou, Yifei: Limited-angle CT reconstruction via the (L_1/L_2) minimization (2021)