8 commits

Author	SHA1	Message	Date
Zangwei Zheng	52aeb3769f	format and some fix	2024-03-30 12:02:48 +08:00
Hongxin Liu	8d5278b99a	[feature] impl cached pos embedding (#1) ... * [feature] impl cached pos embedding * [feature] update pos emb * [feature] update pos emb	2024-03-28 17:30:47 +08:00
Frank Lee	14526479fc	added sp for inference (#76)	2024-03-17 11:00:23 +08:00
Zheng Zangwei (Alex Zheng)	d851a85535	format (#69)	2024-03-15 22:16:20 +08:00
Frank Lee	c4c5d64e49	migrate some new files (#64)	2024-03-15 21:49:38 +08:00
Frank Lee	2f87a9af75	removed old files (#63)	2024-03-15 21:48:36 +08:00
Hongxin Liu	91275b2b5e	[feature] impl fastseq-style seq parallel (#21) ... * [feature] add fastseq-style sp attn * [feature] add overlap fastseq * [test] add test for self attn * [feature] update dit model to fit fastseq * [polish] refactor attn * [feature] update train & benchmark script * [polish] update benchmark script * [polish] update benchmark script	2024-03-01 17:31:59 +08:00
Hongxin Liu	97c089daec	[feature] impl ulysses-style seq parallel (#20) ... * [feature] add ulysses style sp attn * [test] add sp attn test * [feature] add zero sp plugin * [hotfix] fix sp backward * [test] add test for dit model	2024-03-01 14:42:06 +08:00