Publications

$^*$ denotes equal contribution. (α-β) denotes alphabetical order.

2026

  1. arXiv
    Transformers Are Born Biased: Structural Inductive Biases at Random Initialization and Their Practical Consequences
    Siquan Li*, Yao Tong*, Haonan Wang*, and Tianyang Hu
    arXiv preprint, 2026
    Part of the methodology, SeedPrint, is published in ICLR 2026.
    arXiv
  2. ICLR
    SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained From
    Yao Tong*, Haonan Wang*, Siquan Li, Kenji Kawaguchi, and Tianyang Hu
    International Conference on Learning Representations, 2026
    Abridged in the NeurIPS 2025 Lock-LLM Workshop: Prevent Unauthorized Knowledge Use from Large Language Models.
    arXiv
  3. ICLR
    PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
    Haonan Wang*, Brian Chen*, Siquan Li*, Xinhe Liang, Hwee Kuan Lee, Kenji Kawaguchi, and Tianyang Hu
    International Conference on Learning Representations, 2026
    Abridged in the ICML 2025 Workshop on Test-Time Adaptation: Putting Updates to the Test (PUT).
    arXiv
  4. ICLR
    Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling
    Tianyu Xie, Shuchen Xue, Zijin Feng, Tianyang Hu, Jiacheng Sun, Zhenguo Li, and Cheng Zhang
    International Conference on Learning Representations, 2026
    arXiv
  5. ICLR
    Reinforcing Diffusion Models by Direct Group Preference Optimization
    Yihong Luo, Tianyang Hu, and Jing Tang
    International Conference on Learning Representations, 2026
    arXiv

2025

  1. NeurIPS Workshop
    SRTD: A Symmetric Divergence for Interpretable Comparison of Representation Topology
    Yan Wang, Yue Zhu, and Tianyang Hu
    NeurIPS Workshop on Symmetry and Geometry in Neural Representations, 2025
  2. NeurIPS
    Noise Consistency Training: A Native Approach for One-step Generator in Learning Additional Controls
    Yihong Luo*, Shuchen Xue*, Tianyang Hu, and Jing Tang
    Advances in Neural Information Processing Systems, 2025
    arXiv
  3. NeurIPS
    Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generation
    Yihong Luo*, Tianyang Hu*, Weijian Luo, Kenji Kawaguchi, and Jing Tang
    Advances in Neural Information Processing Systems, 2025
    arXiv
  4. JMLR
    Minimax Optimal Deep Neural Network Classifiers Under Smooth Decision Boundary
    Tianyang Hu, Ruiqi Liu, Zuofeng Shang, and Guang Cheng
    Journal of Machine Learning Research, 26 (136), 2025
    arXiv
  5. ICCV
    Adding Additional Control to One-Step Diffusion with Joint Distribution Matching
    Yihong Luo, Tianyang Hu, Yifan Song, Jiacheng Sun, Zhenguo Li, and Jing Tang
    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025
    arXiv
  6. ICCV
    Learning Few-Step Diffusion Models by Trajectory Distribution Matching
    Yihong Luo, Tianyang Hu, Jiacheng Sun, Yujun Cai, and Jing Tang
    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025
    arXiv
  7. ICML WorkshopOral
    Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecture
    Shuchen Xue, Tianyu Xie, Tianyang Hu, Zijin Feng, Jiacheng Sun, Kenji Kawaguchi, Zhenguo Li, and Zhi-Ming Ma
    ICML Workshop on Efficient Systems for Foundation Models (ES-FoMo III) (Selected for Oral presentation)., 2025
    arXiv
  8. ICML
    Elucidating the design space of language models for image generation
    Xuantong Liu, Shaozhe Hao, Xianbiao Qi, Tianyang Hu, Jun Wang, Rong Xiao, and Yuan Yao
    International Conference on Machine Learning, 2025
    arXiv
  9. ICLR
    You only sample once: Taming one-step text-to-image synthesis by self-cooperative diffusion gans
    Yihong Luo, Xiaolong Chen, Xinghua Qu, Tianyang Hu, and Jing Tang
    International Conference on Learning Representations, 2025
    arXiv
  10. TMLR
    AlgoFormer: An Efficient Transformer Framework with Algorithmic Structures
    Yihang Gao*, Chuanyang Zheng*, Enze Xie, Han Shi, Tianyang Hu, Yu Li, Michael K Ng, Zhenguo Li, and Zhaoqiang Liu
    Transactions on Machine Learning Research, 2025
    arXiv
  11. EMNLP
    The Buffer Mechanism for Multi-Step Information Reasoning in Language Models
    Zhiwei Wang, Yunji Wang, Zhongwang Zhang, Zhangchen Zhou, Hui Jin, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Yaoyu Zhang, and Zhi-Qin John Xu
    Findings of the Association for Computational Linguistics: EMNLP, 2025
    arXiv
  12. NAACL
    Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data
    Haonan Wang*, Minbin Huang*, Runhui Huang, Lanqing Hong, Hang Xu, Tianyang Hu, Xiaodan Liang, Zhenguo Li, Hong Cheng, and Kenji Kawaguchi
    Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2025
    arXiv

2024

  1. NeurIPS Workshop
    In-Context Learning behaves as a greedy layer-wise gradient descent algorithm
    Brian Chen, Tianyang Hu, Hui Jin, Hwee Kuan Lee, and Kenji Kawaguchi
    NeurIPS Workshop on Adaptive Foundation Models: Evolving AI for Personalized and Efficient Learning, 2024
  2. ICML
    Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
    Brian Chen, Tianyang Hu, Hui Jin, Hwee Kuan Lee, and Kenji Kawaguchi
    International Conference on Machine Learning, 2024
    arXiv
  3. ICML
    The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling
    Jiajun Ma*, Shuchen Xue*, Tianyang Hu, Wenjia Wang, Zhaoqiang Liu, Zhenguo Li, Zhi-Ming Ma, and Kenji Kawaguchi
    International Conference on Machine Learning, 2024
    arXiv
  4. ICML
    Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion
    Xuantong Liu, Tianyang Hu, Wenjia Wang, Kenji Kawaguchi, and Yuan Yao
    International Conference on Machine Learning, 2024
    arXiv
  5. CVPR
    Accelerating Diffusion Sampling with Optimized Time Steps
    Shuchen Xue, Zhaoqiang Liu, Fei Chen, Shifeng Zhang, Tianyang Hu, Enze Xie, and Zhenguo Li
    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024
    arXiv
  6. ECCV
    JointDreamer: Ensuring Geometry Consistency and Text Congruence in Text-to-3D Generation via Joint Score Distillation
    Chenhan Jiang*, Yihan Zeng*, Tianyang Hu, Songcen Xu, Wei Zhang, Dit-Yan Yeung, and Hang Xu
    European Conference on Computer Vision, 2024
    arXiv
  7. ICLR
    Elucidating The Design Space of Classifier-Guided Diffusion Generation
    Jiajun Ma, Tianyang Hu, Wenjia Wang, and Jiacheng Sun
    International Conference on Learning Representations, 2024
    arXiv
  8. JMLR
    Random Smoothing Regularization in Kernel Gradient Descent Learning
    (α-β) Liang Ding, Tianyang Hu, Jiahang Jiang, Donghao Li, Wenjia Wang, and Yuan Yao
    Journal of Machine Learning Research, 25 (284), 2024
    arXiv
  9. IJCAI
    Deciphering the Projection Head: Representation Evaluation Self-supervised Learning
    Jiajun Ma, Tianyang Hu, and Wenjia Wang
    International Joint Conference on Artificial Intelligence, 2024
    arXiv

2023

  1. NeurIPSSpotlight
    Complexity Matters: Rethinking the Latent Space for Generative Modeling
    Tianyang Hu, Fei Chen, Haonan Wang, Jiawei Li, Wenjia Wang, Jiacheng Sun, and Zhenguo Li
    Advances in Neural Information Processing Systems, 2023
    arXiv
  2. NeurIPS
    Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models
    Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang
    Advances in Neural Information Processing Systems, 2023
    arXiv
  3. arXiv
    Training Energy-Based Models with Diffusion Contrastive Divergences
    Weijian Luo, Hao Jiang, Tianyang Hu, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang
    arXiv preprint, 2023
    arXiv
  4. CVPR
    ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning
    Hao Yang, Lanqing Hong, Aoxue Li, Tianyang Hu, Zhenguo Li, Gim Hee Lee, and Liwei Wang
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023
    arXiv
  5. AISTATS
    Inducing Neural Collapse in Deep Long-tailed Learning
    Xuantong Liu, Jianfeng Zhang, Tianyang Hu, He Cao, Yuan Yao, and Lujia Pan
    In International Conference on Artificial Intelligence and Statistics, 2023
    arXiv
  6. ICML
    Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization
    Yimeng Chen, Tianyang Hu, Fengwei Zhou, Zhenguo Li, and Zhiming Ma
    In International Conference on Machine Learning, 2023
    arXiv
  7. UAI
    Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classification
    Pawel Piwek, Adam Klukowski, and Tianyang Hu
    In Conference on Uncertainty in Artificial Intelligence, 2023
    arXiv
  8. ICLR
    Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding
    Tianyang Hu, Zhili Liu, Fengwei Zhou, Wenjia Wang, and Weiran Huang
    International Conference on Learning Representations, 2023
    arXiv
  9. TMLR
    Continual Learning by Modeling Intra-Class Variation
    Longhui Yu, Tianyang Hu, Lanqing Hong, Zhen Liu, Adrian Weller, and Weiyang Liu
    Transactions on Machine Learning Research, 2023
    arXiv

2022

  1. NeurIPS
    ZooD: Exploiting Model Zoo for Out-of-Distribution Generalization
    Qishi Dong, Awais Muhammad, Fengwei Zhou, Chuanlong Xie, Tianyang Hu, Yongxin Yang, Sung-Ho Bae, and Zhenguo Li
    Advances in Neural Information Processing Systems, 2022
    arXiv
  2. NeurIPSSpotlight
    Understanding Square Loss in Training Overparametrized Neural Network Classifiers
    Tianyang Hu*, Jun Wang*, Wenjia Wang*, and Zhenguo Li
    Advances in Neural Information Processing Systems, 2022
    arXiv

2021

  1. AISTATS
    Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network
    Tianyang Hu*, Wenjia Wang*, Cong Lin, and Guang Cheng
    In International Conference on Artificial Intelligence and Statistics, 2021
    arXiv

2020

  1. Thesis
    Nonparametric Perspective of Deep Learning
    Tianyang Hu
    Purdue University Graduate School, 2020
    HTML
  2. arXiv
    Sharp Rate of Convergence for Deep Neural Network Classifiers Under the Teacher-student Setting
    Tianyang Hu, Zuofeng Shang, and Guang Cheng
    arXiv preprint, 2020
    arXiv
  3. AJPE
    Inter-rater Reliability of Web-based Calibrated Peer Review within a Pharmacy Curriculum
    Alex N Isaacs, Monica L Miller, Tianyang Hu, Bailey Johnson, and Zach A Weber
    American Journal of Pharmaceutical Education, 2020

2018

  1. arXiv
    Stein Neural Sampler
    Tianyang Hu, Zixiang Chen, Hanxi Sun, Jincheng Bai, Mao Ye, and Guang Cheng
    arXiv preprint, 2018
    arXiv