Hongxin Liu
|
97c089daec
|
[feature] impl ulysses-style seq parallel (#20)
* [feature] add ulysses style sp attn
* [test] add sp attn test
* [feature] add zero sp plugin
* [hotfix] fix sp backward
* [test] add test for dit model
|
2024-03-01 14:42:06 +08:00 |
|