Skeleton-Based Exercise Quality Assessment Under Domain Shift

Abstract

Skeleton-based models for exercise quality assessment score well when trained and tested on the same recording setup, but it is unclear whether they hold up once deployment moves across laboratory, clinical, motion-capture, and web-video domains. We benchmark this question directly: 4 exercise datasets, 9 skeleton-based models spanning GCN and transformer architectures, and 198 out-of-distribution transfer experiments. The result is that 154 of 198 OOD experiments (77.8%) exhibit full or partial classifier collapse — the model settles on one class almost regardless of input, a failure that overall accuracy and macro-F1 do not reveal on their own.

Overview

Cross-Domain Exercise Quality Assessment

Benchmark pipeline: four cross-dataset inputs are unified into an 18-joint skeleton representation, passed through a suite of six GCN and three transformer models, evaluated under IID, OOD, and Mixed protocols, and scored with failure-aware evaluation that detects classifier collapse.

Four datasets — EC3D, REHAB24, Custom YouTube, and UI-PRMD — are unified under one 18-joint preprocessing protocol, evaluated across nine skeleton models, and scored with a failure-aware protocol that inspects per-class prediction counts rather than aggregate accuracy alone.

Main Result

Cross-Domain Collapse Is the Main Failure Mode

77.8%

154 of 198 OOD transfer experiments exhibit full or partial classifier collapse.

Stacked bar showing 141 full-collapse, 13 partial-collapse, and 44 genuine-transfer outcomes out of 198 OOD experiments.

Full Collapse (FC)One class receives zero predictions across the entire test set.

Partial Collapse (PC)One class receives five or fewer predictions — at most ~3.6% class coverage.

A model can report a non-zero F1 score while effectively predicting almost every sample as the same class.

Architecture Comparison

Newer Architectures Do Not Solve Domain Shift

GCN · 6 models

ST-GCN
CTR-GCN
InfoGCN
HD-GCN
BlockGCN
ProtoGCN

Transformer · 3 models

SkateFormer
HyperFormer
FreqMixFormer

Bar chart of collapse rate by model, ranging from 63.64% to 90.91% across the 22 OOD conditions.

Mean OOD macro-F1 clusters tightly across all nine models — 32.6% to 36.7%. Only 1 of 36 pairwise comparisons is significant before correction, and none remain significant after Bonferroni correction. All experiments are single runs (seed 42), so the results are exploratory.

Bar chart of mean OOD macro-F1 per model, clustered between 32.59% and 36.70%.

Architecture family alone does not reliably predict cross-domain robustness.

Pooled Training

Data Diversity Helps — But It Is Not Unseen-Domain Generalization

collapsed models in Mixed D1

91.4%

best result — InfoGCN

domains pooled in training

Mixed D1 pools data from all four domains before splitting into train / val / test, so it should not be read as generalization to an unseen domain — it shows the task is learnable given broad coverage, not that transfer to a truly novel recording environment is solved.

Citation

BibTeX

@inproceedings{10.1145/3840471.3842592,
  author    = {Vahdati, Monica and Gholizadeh HamlAbadi, Kamran and Etemad, Ali and El Saddik, Abdulmotaleb},
  title     = {Skeleton-Based Exercise Quality Assessment Under Domain Shift: A Benchmark Study of Cross-Domain Failure Modes},
  year      = {2026},
  isbn      = {9798400729256},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  url       = {https://doi.org/10.1145/3840471.3842592},
  doi       = {10.1145/3840471.3842592},
  booktitle = {Proceedings of the 3rd International Workshop on Multimedia Computing for Health and Medicine},
  pages     = {43--52},
  numpages  = {10},
  location  = {Rio de Janeiro, Brazil},
  series    = {MCHM '26}
}