Notes

• How to use Huggingface’s pretrained weights in LiBai
  • LiBai Transformer vs Huggingface Transformer
  • QKV slicing logic
  • How to correctly load QKV weights
• Detailed instruction on building Vision Transformer models in LiBai
  • Model Architecture
  • A simple Torch implementation of Vision Transformer
  • Implement 3D parallel Vision Transformer in LiBai
  • Details about LiBai’s implementation of the Vision Transformer model
• How to load pretrained model in LiBai
  • Steps
  • Start Loading
• Use Custom ModelLoader
  • Model Loader for HuggerFace
  • Model Loader for LiBai
• Detailed instruction on using model parallel in LiBai
  • Transfer pytroch model to oneflow.
  • 2、Change the deep learning framework to oneflow.
  • 3、Using Libai’s api.
  • 4、Model parallel in Libai.
• Detailed instruction for using distributed inference in LiBai
  • Check model.py
  • Build config.py
  • Refine pipeline_inference.py
  • Distributed run pipeline_inference.py
• Frequently Asked Questions
  • Training
  • Model