Commit graph

61 commits

Author SHA1 Message Date
xyupeng
f7329dda38 Dev/pxy (#69)
* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scene_cut

* update scene_cut

* update scene_cut[A

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* update readme

* update readme

* extract frames using opencv everywhere

* extract frames using opencv everywhere

* extract frames using opencv everywhere

* filter panda10m

* filter panda10m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* ocr

* add ocr

* add main.sh

* add ocr

* add ocr

* add ocr

* add ocr

* add ocr

* add ocr

* update scene_cut

* update remove main.sh

* update scoring

* update scoring

* update scoring

* update README

* update readme

* update scene_cut

* update readme

* update scoring

* update readme

* update readme

* update filter_panda10m

* update readme

* update readme
2024-04-23 21:05:45 +08:00
zhengzangw
a3346808a6 Merge branch 'main' into dev/v1.1 2024-04-22 09:26:17 +00:00
xyupeng
e933ed8727 Dev/pxy (#61)
* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scene_cut

* update scene_cut

* update scene_cut[A

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* update readme

* update readme

* extract frames using opencv everywhere

* extract frames using opencv everywhere

* extract frames using opencv everywhere

* filter panda10m

* filter panda10m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* ocr

* add ocr

* add main.sh

* add ocr

* add ocr

* add ocr

* add ocr

* add ocr

* add ocr

* update scene_cut

* update remove main.sh
2024-04-22 17:15:55 +08:00
Zangwei Zheng
65804b2e9a fix datautil 2024-04-22 15:27:22 +08:00
Zheng Zangwei (Alex Zheng)
341b12f9bf Dev/v1.0.1 (#58)
* update (#57)

* update

* update datautil

* add VBench prompt

* update eval

* update eval

* update intepolation

* add vbench eval

* Dev/sdedit implementation (#56)

* Update utils.py

* update

* update

* update

---------

Co-authored-by: YuKun Zhou <90625606+1zeryu@users.noreply.github.com>
2024-04-20 21:23:10 +08:00
zhengzangw
d262f14a24 update 2024-04-20 02:43:38 +00:00
Zheng Zangwei (Alex Zheng)
1f99b9ba4b complete eval pipeline (#53) 2024-04-18 15:49:14 +08:00
Zangwei Zheng
50ad9a1ca5 small update 2024-04-15 17:36:20 +08:00
Zangwei Zheng
63e86f6dd6 hotfix inference 2024-04-15 13:44:30 +08:00
Zangwei Zheng
1207af03a1 a bunch of update 2024-04-14 17:05:25 +08:00
Zangwei Zheng
8513ff8734 v1.1 training preparation 2024-04-14 00:20:00 +08:00
xyupeng
7ebe6c2d87 Dev/pxy (#49)
* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scene_cut

* update scene_cut

* update scene_cut[A

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* update readme

* update readme

* extract frames using opencv everywhere

* extract frames using opencv everywhere

* extract frames using opencv everywhere
2024-04-13 15:49:05 +08:00
Zangwei Zheng
f5e347e08e merge 2024-04-13 15:47:42 +08:00
Zangwei Zheng
3b85effe5a a bunch of update for data 2024-04-13 15:44:24 +08:00
xyupeng
4e0f97f374 Dev/pxy (#48)
* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scene_cut

* update scene_cut

* update scene_cut[A

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* update readme

* update readme

* extract frames using opencv everywhere
2024-04-13 15:43:19 +08:00
Zangwei Zheng
8fa108a606 [feat] support ffmpeg 2024-04-11 15:48:48 +08:00
Zangwei Zheng
fff002bede [feat] support parquet 2024-04-11 15:28:49 +08:00
Zangwei Zheng
6e2f42cd9f [feat] faster get file list 2024-04-11 13:34:02 +08:00
Zangwei Zheng
bd6a3e46e4 Merge branch 'dev/v1.0.1' of https://github.com/hpcaitech/Open-Sora-dev into dev/v1.0.1 2024-04-11 11:48:13 +08:00
Zangwei Zheng
da1038ca5c a bunch of update 2024-04-11 11:48:06 +08:00
Tom Young
745c1ed784 . 2024-04-11 10:24:01 +08:00
Tom Young
ed65131c57 . 2024-04-11 10:10:13 +08:00
Zangwei Zheng
aced1fb80f [fix] merge csv 2024-04-09 14:01:10 +08:00
Zangwei Zheng
e00f93d111 [exp] update 2024-04-09 13:55:55 +08:00
Hongxin Liu
4bd85a3851 [faeture] add script to split dataset by bucket (#43) 2024-04-08 16:04:19 +08:00
Zangwei Zheng
a445927680 misc 2024-04-08 14:12:50 +08:00
Zangwei Zheng
882225897b fix bucket index 2024-04-07 16:28:44 +08:00
xyupeng
c4bf35727c Dev/pxy (#38)
* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scene_cut

* update scene_cut

* update scene_cut[A

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* update readme

* update readme
2024-04-06 16:45:59 +08:00
xyupeng
7da8be989a Dev/pxy (#37)
* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scoring/matching

* update scene_cut

* update scene_cut

* update scene_cut[A

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* update scene_cut

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* m

* update readme
2024-04-06 16:40:48 +08:00
Zheng Zangwei (Alex Zheng)
c9b81d8fd6 add camera motion detection (#34) 2024-04-05 23:42:31 +08:00
Shen-Chenhui
0d7c226ce6 add remove duplicates by path function 2024-04-05 17:28:17 +08:00
Shen-Chenhui
f3c417a61e add filter by flow score function 2024-04-05 14:48:46 +08:00
Shen-Chenhui
d8203d5cc1 add intersect func 2024-04-05 11:21:00 +08:00
Zheng Zangwei (Alex Zheng)
1bb49f75e7 update frame extraction (#31) 2024-04-04 15:34:26 +08:00
Frank Lee
04c502f859 added checks for video corruption (#30) 2024-04-04 14:51:36 +08:00
Zangwei Zheng
12dcbeee8e update csvutil & mask 2024-04-03 15:15:04 +08:00
Zangwei Zheng
550ea32fad update csvutil 2024-04-03 13:33:46 +08:00
Hongxin Liu
c18ad83f01 [hotfix] fix caption processing (#27) 2024-04-03 11:32:01 +08:00
Zangwei Zheng
065774a501 [feat] resize images 2024-04-02 16:55:03 +08:00
Zangwei Zheng
00c20261f0 update data pipeline 2024-04-02 15:55:58 +08:00
Hongxin Liu
c618f77bdc [feature] move t5 text processing to csvutil (#22)
* [feature] move t5 text processing to csvutil

* polish code
2024-04-02 14:56:44 +08:00
Zheng Zangwei (Alex Zheng)
b5414b36b8 Dev/datapipe (#21)
* fix #210

* fix #209

* fix #188

* [docs] add training order

* update data pipeline

---------

Co-authored-by: Sze-qq <68757353+Sze-qq@users.noreply.github.com>
2024-04-02 14:51:21 +08:00
Zangwei Zheng
5ade5e5984 [fix] use decord for tools 2024-04-01 16:45:23 +08:00
Zangwei Zheng
4bfb90295b update csvutil 2024-04-01 11:45:38 +08:00
Zangwei Zheng
cdd3b5eb74 llava fix and config modification 2024-03-31 23:44:37 +08:00
Zheng Zangwei (Alex Zheng)
f1ee27ba2f [feat] llava support image and text (#13)
* [feat] llava support image and text

* add resize for image

* update gpt4 caption

* update prompt for llava image captioning
2024-03-31 20:59:33 +08:00
Zangwei Zheng
987283fa1b [fix] transform may not fit enough 2024-03-30 17:05:15 +08:00
Zangwei Zheng
52aeb3769f format and some fix 2024-03-30 12:02:48 +08:00
Zheng Zangwei (Alex Zheng)
682a699aec Update image process (#5)
* [docs] update tool docs

* update aes
2024-03-29 23:34:10 +08:00
Zangwei Zheng
2d4a5df287 [wip] image wrong with flashattn 2024-03-28 22:04:43 +08:00