1 - Zhejiang University
2 - Tencent
Abstract
We present a novel learning method to predict the cloth deformation for skeleton-based characters with a two-stream network.
The characters processed in our approach are not limited to humans, and can be other skeletal-based representations of non-human
targets such as fish or pets. We use a novel network architecture which consists of skeleton-based and mesh-based residual networks
to learn the coarse and wrinkle features as the overall residual from the template cloth mesh. Our network is used to predict the
deformation for loose or tight-fitting clothing or dresses. We ensure that the memory footprint of our network is low, and thereby
result in reduced storage and computational requirements. In practice, our prediction for a single cloth mesh for the skeleton-based
character takes about 7 milliseconds on an NVIDIA GeForce RTX 3090 GPU. Compared with prior methods, our network can generate
fine deformation results with details and wrinkles.
Results
To evaluate that our network can process more complex and different characters, we applied our network on non-human characters such as a monster, a dolphin, and a cat. The monster character has a skeleton similar to the human character, while the dolphin and the cat have different skeletons. The dolphin character has no leg joints, while the cat model has four legs without hands. We can also simulate the cloth deformation on these characters. The monster character wears a loose robe, and the dolphin and the cat wear tight-fitting clothes designed for these characters.
Paper (PDF 12.1 MB)
Video (30.9 MB) also at Youtube
Yudi Li, Min Tang, Yun Yang, Ruofeng Tong, Shuangcai Yang, Yao Li, Bailin An, Qilong Kou, CTSN: Predicting Cloth Deformation for Skeleton-based Characters with a Two-stream Skinning Network, accepted by Computational Visual Media (Proceedings of CVM 2023), 2023.
@article{ncloth22,
author = {Li, Yudi and Tang, Min and Yang, Yun and Tong, Ruofeng and Yang, Shuangcai and Li, Yao and An, Bailin and Kou, Qilong},
title = {{CTSN}: Predicting Cloth Deformation for Skeleton-based Characters with a Two-stream Skinning Network},
journal = {Computational Visual Media (Proceedings of CVM 2023)},
year = {2023},
}
D-Cloth: Skinning-based Cloth Dynamic Prediction with a Three-stage Network
N-Cloth: Predicting 3D Cloth Deformation with Mesh-Based Networks
I-Cloth: Incremental Collision Handling for GPU-Based Interactive Cloth Simulation
PSCC: Parallel Self-Collision Culling with Spatial Hashing on GPUs
I-Cloth: API for fast and reliable cloth simulation with CUDA
Efficient BVH-based Collision Detection Scheme with Ordering and Restructuring
MCCD: Multi-Core Collision Detection between Deformable Models using Front-Based Decomposition
TightCCD: Efficient and Robust Continuous Collision Detection using Tight Error Bounds
Fast and Exact Continuous Collision Detection with Bernstein Sign Classification
A GPU-based Streaming Algorithm for High-Resolution Cloth Simulation
UNC dynamic model benchmark repository
Collision-Streams: Fast GPU-based Collision Detection for Deformable Models
Fast Continuous Collision Detection using Deforming Non-Penetration Filters
Fast Collision Detection for Deformable Models using Representative-Triangles
DeformCD: Collision Detection between Deforming Objects
Self-CCD: Continuous Collision Detection for Deforming Objects
Interactive Collision Detection between Deformable Models using Chromatic Decomposition
Fast Proximity Computation Among Deformable Models using Discrete Voronoi Diagrams
CULLIDE: Interactive Collision Detection between Complex Models using Graphics Hardware
RCULLIDE: Fast and Reliable Collision Culling using Graphics Processors
Quick-CULLIDE: Efficient Inter- and Intra-Object Collision Culling using Graphics Hardware
This work is supported in part by the National Natural Science Foundation of China under Grant No.: 61972341, Grant No.: 61972342, Grant No.: 61732015, and the Tencent-Zhejiang University joint laboratory.