Tao Hu

Ommer Lab.


Computer Vision & Learning Group

Akademiestr. 7,Munich

Ludwig Maximilian University of Munich

I am a Postdoctoral Research Fellow with Björn Ommer in Ommer-Lab ( Stable Diffusion Lab ), focused on exploring the scalability and generalization ablity of diffusion model. I finished my PhD at VISLab, University of Amsterdam, supervised by Cees Snoek and Pascal Mettes. Closely coorporated with David W Zhang, Basura Fernando.

I am recruiting for Bachelor, Master and PhD supervision in Munich and globally. If you're interested in collaborating, feel free to send an email.

Open to discussion and collaboration, feel free to send an email.

Focused on introducing inductive bias into neural network to achieve data-efficiency by few-shot learning, generative model, etc. Have a conviction that generative modelling will be the future of discriminative modelling.

I'm on the job market. Feel free to contact me.

Publication | GitHub | CV(updated in Nov.2023) |
| Research Note | Chat with me |Wechat |

Tao Hu's wechat


Apr 14, 2024 Co-hosting the ECCV 2024 AVGenL: Audio-Visual Generation and Learning workshop.
Mar 29, 2024 I am honored to be invited as the Area Chair of AI for Content Creation Workshop @ CVPR 2024 and Efficient Large Vision Models Workshop @ CVPR 2024.
Mar 20, 2024 A DiT-style Mamba-based diffusion models has been released, co-authored by Bjorn Ommer and other phd students in CompVis lab. Stay tuned for updates. :sparkles:
Dec 11, 2023 An editing method on latent flow matching is accepted by AAAI 2024:sparkles:
Nov 20, 2023 An extension paper on self-guided diffusion models has been submitted, co-authored by Mathilde Caron and Yuki Asano. Stay tuned for updates. :sparkles:

selected publications

  1. ZigMa: A DiT-style Zigzag Mamba Diffusion Model
    Vincent Tao Hu , Stefan Andreas Baumann , Ming Gui , and 4 more authors
    In Arxiv , 2024
    a DiT-style Mamba-based diffusion models
  2. DepthFM: Fast Monocular Depth Estimation with Flow Matching
    Ming Gui , Johannes S. Fischer , Ulrich Prestel , and 6 more authors
    In Arxiv , 2024
    An exploration of flow matching for blazing fast and zero-shot depth estimation
  3. Guided Flow Vision Transformer from Self-Supervised Diffusion Features
    Vincent Tao Hu , Yunlu Chen , Mathilde Caron , and 3 more authors
    In Submission , 2024
  4. Motion Flow Matching for Human Motion Synthesis and Editing
    Vincent Tao Hu , Wenzhe Yin , Pingchuan Ma , and 7 more authors
    In Submission , 2024
  5. Training Class-Imbalanced Diffusion Model Via Overlap Optimization
    Divin Yan , Lu Qi , Vincent Tao Hu , and 2 more authors
    In arxiv , 2024
  6. ./fm.png
    Latent Space Editing in Transformer-based Flow Matching
    Vincent Tao Hu , David W Zhang , Pascal Mettes , and 3 more authors
    In AAAI 2024. Also appear in ICML 2023 Workshop, New Frontiers in Learning, Control, and Dynamical Systems , 2024
  7. Continuous, Subject-Specific Attribute Control in T2I Models by Identifying Semantic Directions
    Stefan Andreas Baumann , Felix Krause , Michael Neumayr , and 3 more authors
    In Arxiv , 2024
    Prompt Editing in T2I models
  8. Boosting Latent Diffusion with Flow Matching
    Johannes S. Fischer , Ming Gui , Pingchuan Ma , and 4 more authors
    In Arxiv , 2023
    flow matching for super-resolution
  9. ToddlerDiffusion: Flash Interpretable Controllable Diffusion Model
    Eslam Mohamed BAKR , Liangbing Zhao , Vincent Tao Hu , and 3 more authors
    In Submission , 2024
  10. ./scribbleseg.png
    Generative Data Augmentation Improves Scribble-supervised Semantic Segmentation
    Jacob Schnell , Jieke Wang , Lu Qi , and 2 more authors
    In SyntaGen CVPR workshop , 2024
    Explore diffusion model for data augmention in segmentation task.
  11. ./fsinr.png
    On the Few-Shot Generalization of Learning on Implicit Neural Representations
    Vincent Tao Hu , David W Zhang , Yunlu Chen , and 6 more authors
    In ICCV NeRF4ADR Workshop , 2023
    Explore few-shot generalization of INR on images.
  12. ./sgdm-why.png
    Self-Guided Diffusion Models
    Tao Hu* , David W Zhang* , Yuki M. Asano , and 2 more authors
    In CVPR , 2023
    A bridge between the community of self-supervised learning and diffusion models. Short version to appear in NeurIPS 2022 Workshop on Score-Based Methods and NeurIPS 2022 Workshop Self-Supervised Learning Theory and Practice.