AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
We offer the following datasets (note that you will require 2x the dataset size due to downloading and untar-ing). Data is stored in Internet Archive, and you will need to first install the pip package: pip install internetarchive You can uses the scripts in scripts/download to download each dataset. Sudo apt-get update & sudo apt-get install -y ffmpeg This presents a challenging benchmark for video prediction in partially observable environments where a model must understand what parts of the scenes to re-create versus invent depending on its past observations or generations. In addition, to better understand the capabilities of video prediction models in modeling temporal consistency, we introduce several challenging video prediction tasks consisting of agents randomly traversing 3D scenes of varying difficulty. Our experiments show that TECO outperforms SOTA baselines in a variety of video prediction benchmarks ranging from simple mazes in DMLab, large 3D worlds in Minecraft, and complex real-world videos from Kinetics-600. We use a MaskGit prior for dynamics prediction which enables both sharper and faster generations compared to prior work. In this work, we present \textbfnsistent Video Transformer (TECO), a vector-quantized latent dynamics video prediction model that learns compressed representations to efficiently condition on long videos of hundreds of frames during both training and generation. Although these techniques may produce sharp videos, they have difficulty retaining long-term temporal consistency due to their limited context length. Primarily due to computational limitations, most prior methods limit themselves to training on a small subset of frames that are then extended to generate longer videos through a sliding window fashion. Generating long, temporally consistent video remains an open challenge in video generation. Temporally Consistent Video Transformer for Long-Term Video Prediction
0 Comments
Read More
Leave a Reply. |