Commit graph

23 commits

Author SHA1 Message Date
Zangwei Zheng
065774a501 [feat] resize images 2024-04-02 16:55:03 +08:00
Zangwei Zheng
00c20261f0 update data pipeline 2024-04-02 15:55:58 +08:00
Hongxin Liu
c618f77bdc [feature] move t5 text processing to csvutil (#22)
* [feature] move t5 text processing to csvutil

* polish code
2024-04-02 14:56:44 +08:00
Zheng Zangwei (Alex Zheng)
b5414b36b8 Dev/datapipe (#21)
* fix #210

* fix #209

* fix #188

* [docs] add training order

* update data pipeline

---------

Co-authored-by: Sze-qq <68757353+Sze-qq@users.noreply.github.com>
2024-04-02 14:51:21 +08:00
Zangwei Zheng
5ade5e5984 [fix] use decord for tools 2024-04-01 16:45:23 +08:00
Zangwei Zheng
4bfb90295b update csvutil 2024-04-01 11:45:38 +08:00
Zangwei Zheng
cdd3b5eb74 llava fix and config modification 2024-03-31 23:44:37 +08:00
Zheng Zangwei (Alex Zheng)
f1ee27ba2f [feat] llava support image and text (#13)
* [feat] llava support image and text

* add resize for image

* update gpt4 caption

* update prompt for llava image captioning
2024-03-31 20:59:33 +08:00
Zangwei Zheng
987283fa1b [fix] transform may not fit enough 2024-03-30 17:05:15 +08:00
Zangwei Zheng
52aeb3769f format and some fix 2024-03-30 12:02:48 +08:00
Zheng Zangwei (Alex Zheng)
682a699aec Update image process (#5)
* [docs] update tool docs

* update aes
2024-03-29 23:34:10 +08:00
Zangwei Zheng
2d4a5df287 [wip] image wrong with flashattn 2024-03-28 22:04:43 +08:00
Zangwei Zheng
a01f6da20e [feat] support for stdit2 sampling 2024-03-28 21:35:33 +08:00
Zangwei Zheng
7392d2e551 [feat] multiple frames with 360p 2024-03-27 00:24:46 +08:00
Zangwei Zheng
13423c57a7 train dataset support new format 2024-03-25 18:36:56 +08:00
Zangwei Zheng
6140f1bbba update csvutil 2024-03-25 17:08:35 +08:00
Zangwei Zheng
5e3eca2d0f add parallel pandas 2024-03-25 15:36:32 +08:00
Zangwei Zheng
3c5ce7f743 complete data format 2024-03-25 15:12:18 +08:00
Zangwei Zheng
b1f6e128dc update convert_dataset and launch exp 2024-03-24 22:03:31 +08:00
Zangwei Zheng
bc27737f7c [feat] add more csv filter 2024-03-24 16:58:16 +08:00
Zangwei Zheng
24c707bfb9 update csvutil for vidprom 2024-03-23 16:02:26 +08:00
Zangwei Zheng
e2856e6397 add datasets doc 2024-03-17 20:09:58 +08:00
Zangwei Zheng
921e3138b3 update docs 2024-03-17 15:47:48 +08:00