mirror of
https://github.com/hpcaitech/Open-Sora.git
synced 2026-05-21 11:59:01 +02:00
update datasets.md
This commit is contained in:
parent
c40f6c681b
commit
5218d5a928
|
|
@ -1,16 +1,28 @@
|
|||
# Datasets
|
||||
|
||||
For Open-Sora 1.1, we conduct mixed training with both images and videos. The main datasets we use are listed below.
|
||||
For Open-Sora 1.2, we conduct mixed training with both images and videos. The main datasets we use are listed below.
|
||||
Please refer to [README](/README.md#data-processing) for data processing.
|
||||
|
||||
## Video
|
||||
|
||||
### Webvid-10M
|
||||
|
||||
[Webvid-10M](https://github.com/m-bain/webvid) contains 10 million video-text pairs scraped from the stock footage sites.
|
||||
We first train the model on this dataset (40k hours) for 30k steps (2 epochs).
|
||||
|
||||
### Panda-70M
|
||||
|
||||
[Panda-70M](https://github.com/snap-research/Panda-70M) is a large-scale dataset with 70M video-caption pairs.
|
||||
We use the [training-10M subset](https://github.com/snap-research/Panda-70M/tree/main/dataset_dataloading) for training,
|
||||
which contains ~10M videos of better quality.
|
||||
|
||||
### Mixkit
|
||||
[Mixkit](https://mixkit.co/) is a video website where we obtained 9k videos.
|
||||
|
||||
### Pixabay
|
||||
[Pixabay](https://pixabay.com/videos/) is video website where we obtained 60.5k videos.
|
||||
|
||||
|
||||
### Pexels
|
||||
|
||||
[Pexels](https://www.pexels.com/) is a popular online platform that provides high-quality stock photos, videos, and music for free.
|
||||
|
|
@ -28,6 +40,16 @@ The dataset is proposed for super-resolution tasks. We use the dataset for HQ fi
|
|||
The caption is generated by BLIP-2.
|
||||
We find the scene and the text quality are relatively poor. For OpenSora 1.0, we only use ~350K samples from this dataset.
|
||||
|
||||
### MiraData
|
||||
|
||||
[MiraData](https://github.com/mira-space/MiraData): a high-quality dataset with 77k long videos, mainly from games and city/scenic exploration.
|
||||
|
||||
|
||||
### Vript
|
||||
|
||||
[Vript](https://github.com/mutonix/Vript/tree/main): a densely annotated dataset of 400k videos.
|
||||
|
||||
|
||||
## Image
|
||||
|
||||
### Midjourney-v5-1.7M
|
||||
|
|
|
|||
Loading…
Reference in a new issue