Unlocking Motion from Large Vision Models with a Semantic and Kinematic Duality for Gait Recognition

Zhanbo Huang ¹ Dingqiang Ye ¹ Xiaoming Liu ^1,2 Yu Kong ¹

¹ Michigan State University ² University of North Carolina, Chapel Hill

Abstract

Existing set-based gait recognition methods achieve remarkable performance by capturing global semantic context. However, their order-invariant nature prevents them from modeling the fine-grained kinematic patterns that unfold over time. To unify the global and process-level representations, we propose GaitMax, a framework capturing both semantic context and kinematic motion.

GaitMax leverages attention-based spatiotemporal modeling to dynamically represent detailed part-level trajectories. While this detailed representation is more powerful, it also captures more nuisance factors (e.g., clothing, viewpoint), leading to potential shortcuts. To mitigate this, we introduce Conditional Decorrelation Loss (CDLoss), which explicitly disentangles the gait embeddings from nuisance factors using vision-language supervision.

This loss requires high-quality nuisance descriptions. We therefore construct GCaption, a new resource that provides natural language annotations for multiple gait datasets, moving beyond simple categorical labels. GCaption not only enables our CDLoss but also serves as a foundation for future context-aware gait analysis.

Approach

Overall framework of GaitMax. A semantic branch encodes global motion context; a kinematic branch models part-wise spatiotemporal dynamics via Gaussian Positional Embedding; GCaption + CDLoss disentangle the gait embedding from nuisance descriptions.

Experiment Results

Switch metrics with the tabs; bars are ranked, and GaitMax is highlighted. Higher is better.

In-domain Evaluation

CCPG

Rank-1 accuracy under four clothing/carrying conditions.

Rank-1 (%)

GaitMax (Ours)

CCGR-MINI

Rank-1, mean Average Precision and mean Inverse Negative Penalty.

(%)

GaitMax (Ours)

Cross-domain Evaluation

Trained on CCPG, tested on unseen datasets without fine-tuning.

CASIA-B

Trained on CCPG, tested cross-dataset without fine-tuning.

Rank-1 (%)

GaitMax (Ours)

SUSTech1K

Trained on CCPG, tested cross-dataset without fine-tuning.

Rank-1 (%)

GaitMax (Ours)

Efficiency

Part-level kinematic modeling avoids dense optical flow, cutting motion-modeling compute by an order of magnitude.

Motion-modeling cost

GFLOPs / frame

Optical Flow + GaitBase 211

GaitMax Kinematic Branch 12.7

16.6× less compute

Part-level kinematic modeling replaces dense optical flow at a fraction of the compute.

Backbone size

Params (M)

DenoisingGait 318

BigGait 21.5

GaitMax 26.6

GaitMax keeps a lightweight backbone on par with BigGait, far below dense-feature pipelines.

BibTeX

@inproceedings{huang2026gaitmax,
  title     = {Unlocking Motion from Large Vision Models with a Semantic and Kinematic Duality for Gait Recognition},
  author    = {Huang, Zhanbo and Ye, Dingqiang and Liu, Xiaoming and Kong, Yu},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026},
}