create_experimental_design_subgraph_input_data = {
    "new_method": """
Adaptive Curvature Momentum (ACM) Optimizer Overview Existing adaptive optimizers such as Adam and AdaBelief dynamically adjust the learning rate based on the history of gradients. However, while these methods adapt to the magnitude of the gradients, they do not fully exploit information about the local curvature of the loss landscape. In this proposal, we introduce a new optimizer called Adaptive Curvature Momentum (ACM), which utilizes local quadratic approximations to adaptively adjust the update direction and scale. Method Standard Momentum Update Similar to SGD or Adam, ACM maintains a momentum term based on past gradients. Adaptive Learning Rate Scaling Uses second-order information (approximations of the Hessian) to dynamically adjust the learning rate for each direction. To reduce the computational cost of Hessian calculations, Fisher Information Matrix approximations can be employed. Curvature-Aware Adaptive Adjustment Estimates curvature by using the gradient change rate: Δ 𝑔 = 𝑔 𝑡 − 𝑔 𝑡 − 1 Δg=g t ​ −g t−1 ​ Modifies the learning rate based on curvature: 𝜂 𝑡 = 𝛼 1 + 𝛽 ⋅ Curvature ( 𝑔 𝑡 ) η t ​ = 1+β⋅Curvature(g t ​ ) α ​ where 𝛼 α is the base learning rate, and 𝛽 β controls the influence of curvature. Adaptive Regularization Encourages stable updates by incorporating an adaptive weight decay mechanism. When local curvature is high, the optimizer strengthens regularization to suppress excessive updates. Key Features and Benefits ✅ Combines Adam-style adaptability with curvature-aware updates ✅ Faster convergence: Adapts step sizes dynamically, taking larger steps in flat regions and smaller steps in sharp valleys. ✅ Hessian-free approximation: Utilizes efficient curvature estimation while maintaining low computational overhead. ✅ Scalability: Suitable for large-scale models such as ResNets and Transformers.
""",
    "base_method_text": """\
'{"arxiv_id":"2402.02149v2","arxiv_url":"http://arxiv.org/abs/2402.02149v2","title":"Improving Diffusion Models for Inverse Problems Using Optimal Posterior\\n  Covariance","authors":["Xinyu Peng","Ziyang Zheng","Wenrui Dai","Nuoqian Xiao","Chenglin Li","Junni Zou","Hongkai Xiong"],"published_date":"2024-02-03T13:35:39Z","journal":"","doi":"","summary":"Recent diffusion models provide a promising zero-shot solution to noisy\\nlinear inverse problems without retraining for specific inverse problems. In\\nthis paper, we reveal that recent methods can be uniformly interpreted as\\nemploying a Gaussian approximation with hand-crafted isotropic covariance for\\nthe intractable denoising posterior to approximate the conditional posterior\\nmean. Inspired by this finding, we propose to improve recent methods by using\\nmore principled covariance determined by maximum likelihood estimation. To\\nachieve posterior covariance optimization without retraining, we provide\\ngeneral plug-and-play solutions based on two approaches specifically designed\\nfor leveraging pre-trained models with and without reverse covariance. We\\nfurther propose a scalable method for learning posterior covariance prediction\\nbased on representation with orthonormal basis. Experimental results\\ndemonstrate that the proposed methods significantly enhance reconstruction\\nperformance without requiring hyperparameter tuning.","github_url":"https://github.com/xypeng9903/k-diffusion-inverse-problems","main_contributions":"The paper addresses the problem of improving diffusion models for solving noisy linear inverse problems by optimizing the posterior covariance without retraining. The key findings include the development of plug-and-play solutions for posterior covariance that significantly enhance reconstruction performance across various tasks such as inpainting, deblurring, and super-resolution without requiring hyperparameter tuning.","methodology":"The proposed approach replaces hand-crafted isotropic covariance approximations with more principled covariance determined by maximum likelihood estimation. Methods include a generalized optimization for posterior covariance using reverse covariance prediction or Monte Carlo estimations and a scalable method based on orthonormal basis transformations to learn posterior covariance predictions.","experimental_setup":"Experiments were conducted on FFHQ and ImageNet datasets to validate the proposed methods against existing ones like DPS and ΠGDM. Performance metrics included SSIM, LPIPS, and FID across tasks such as image inpainting (50% masking), Gaussian and motion deblurring, and 4× super-resolution. The experimental setups ensured consistent evaluation by using a unified codebase for implementation.","limitations":"The proposed methods, constrained by diagonal covariance assumptions, may not achieve optimal performance even with perfect model training. Additional challenges include potential estimation errors related to the reverse covariance predictions and limitations in addressing high-dimensional data complexities.","future_research_directions":"Future work could investigate the design of better covariance principles, explore nonlinear transformations for improved correlation reduction, and develop efficient approximation methods similar to Tweedie\'s approach for broader application in inverse problems."}'"
""",
    "base_experimental_code": "",
    "base_experimental_info": "",
}
