Curieux.JY
  • JungYeon Lee
  • Post
  • ๐Ÿ•ธ๏ธ Graph
  • Lecture
  • Note

On this page

  • ๐Ÿ” Ping Review
  • ๐Ÿ”” Ring Review
    • ํ•œ ์ค„๋กœ ์‹œ์ž‘ํ•˜๋ฉด
    • ๋ฐฐ๊ฒฝ: kinematic ๋‹จ๊ณ„๋Š” ์™œ ๋ฌผ์ฒด๋ฅผ ๋ณด์ง€ ์•Š์•˜๋‚˜ (ยงI, ยงII)
    • ๋ฐฉ๋ฒ• ์ƒ์„ธ (ยงIII)
      • ๋ฌธ์ œ ์„ค์ •
      • III-A. Interaction-graph Laplacian ์ตœ์ ํ™”
      • III-B. ํ•™์Šต๋œ object-aware retargeter
      • III-B1. Supervised ํ•™์Šต
      • III-B2. Object-aware fine-tuning
      • ๋ณธ๋ฌธ๊ณผ ๋ถ€๋ก์ด ์–ด๊ธ‹๋‚˜๋Š” ๊ตฌํ˜„ ๊ฐ’
    • Dynamic retargeting (ยงIV)
    • ์ง๊ด€: ์ ‘์ด‰์„ reference์— ๋„ฃ์œผ๋ฉด ์™œ RL์ด ์‰ฌ์›Œ์ง€๋‚˜
    • ์‹คํ—˜ (ยงV)
      • ์…‹์—… (ยงV-A)
      • Object awareness์˜ ํšจ๊ณผ (ยงV-B, Table I)
      • ๊ตฌ์„ฑ์š”์†Œ ablation (ยงV-C, Table II)
      • ์ผ๋ฐ˜ํ™”: ๋‹ค์‹œ ํ’€์ง€ ์•Š๊ณ  ์˜ฎ๊ธฐ๊ธฐ (ยงV-D, Table III)
      • SOTA ๋น„๊ต (ยงV-E, Table IV, Fig. 4)
      • ์‹ค๊ธฐ ์‹คํ—˜ (ยงV-F, Table V)
    • ์ธก์ •๋œ ๊ฒƒ๊ณผ ์„ ์–ธ๋œ ๊ฒƒ
    • ๋น„ํŒ์ ์œผ๋กœ ๋ณด๋ฉด
      • ๊ฐ•์ 
      • ์•ฝ์ ยทํ•œ๊ณ„
    • ๊ด€๋ จ ์—ฐ๊ตฌ์™€์˜ ์ž๋ฆฌ ๋งค๊น€
    • ์š”์•ฝ
    • ์ด ๊ธ€์— ๋Œ€ํ•ด

๐Ÿ“ƒDexOAK ๋ฆฌ๋ทฐ

dexterity
retargeting
contact
rl
data-generation
DexOAK: Object-Aware Kinematic Retargeting
Published

November 21, 2026

  • Paper Link (arXiv:2610.26762v1)

  • Code (GitHub): ์†Œ์Šค๋Š” ์•„์ง ๊ณต๊ฐœ๋˜์ง€ ์•Š์•˜๋‹ค(README์— โ€œcoming soonโ€).

  • Project

  • Video (ํ”„๋กœ์ ํŠธ ํŽ˜์ด์ง€)

  • Federica Spinola, Robert Katzschmann, Cordelia Schmid

  • Inria ยท ENS-PSL(Spinola, Schmid), ETH Zรผrich Soft Robotics Lab(Katzschmann)

  • arXiv preprint, 2026 (v1 2026-10-19, ํ•™ํšŒ ์ฑ„ํƒ ์ •๋ณด ์—†์Œ)

  1. ๐Ÿ’ก ์‚ฌ๋žŒ ์†-๋ฌผ์ฒด ๋ชจ์บก์„ ๋กœ๋ด‡ ์† ๊ถค์ ์œผ๋กœ ์˜ฎ๊ธฐ๋Š” kinematic retargeting์€ ์ง€๊ธˆ๊นŒ์ง€ ์† ๋ชจ์–‘๋งŒ ๋งž์ถ”๊ณ  ๋ฌผ์ฒด๋Š” ๋’ค์˜ physics ๋‹จ๊ณ„์— ๋ฏธ๋ค˜๋Š”๋ฐ, DexOAK๋Š” kinematic ๋‹จ๊ณ„์—์„œ๋ถ€ํ„ฐ ๋ฌผ์ฒด ๊ธฐ์ค€์˜ ์ ‘์ด‰ ๋ฐฐ์น˜๋ฅผ ๋ณด์กดํ•ด RL์ด ์ ‘์ด‰์„ โ€œ์ฐพ๋Š”โ€ ๋Œ€์‹  dynamics๋งŒ โ€œ๋‹ค๋“ฌ๊ฒŒโ€ ๋งŒ๋“ ๋‹ค.
  2. โš™๏ธ ์†๋ยท์†๋ชฉ๊ณผ ๋ฌผ์ฒด anchor 64๊ฐœ๋ฅผ ํ•˜๋‚˜์˜ interaction graph๋กœ ๋ฌถ์–ด Laplacian ์ขŒํ‘œ๋ฅผ ๋งž์ถ”๋Š” per-frame ์ตœ์ ํ™”๋กœ seed reference๋ฅผ ๋งŒ๋“ค๊ณ , ์ด๋ฅผ feed-forward ๋„คํŠธ์›Œํฌ๋กœ amortizeํ•œ ๋’ค surfaceยทobject-partยทcontact-direction ์†์‹ค๋กœ fine-tuningํ•œ๋‹ค. ๊ฒฐ๊ณผ ๊ถค์ ์€ residual RL์˜ base action์ด ๋œ๋‹ค.
  3. ๐ŸŽฏ ๊ฐ™์€ ManipTrans residual RL ์•„๋ž˜์—์„œ hand-only reference ๋Œ€๋น„ ์„ฑ๊ณต๋ฅ ์ด 20.6% โ†’ 48.2%(Allegro, ์˜ค๋ฅธ์† OakInk-v2)๋กœ ์˜ค๋ฅด๊ณ , OakInk-v2๋กœ ํ•™์Šตํ•œ retargeter๊ฐ€ ARCTIC์— zero-shot์œผ๋กœ 87.9 ADD-AUC๋ฅผ ๋‚ด๋ฉฐ, ์‹ค๊ธฐ open-loop replay๋Š” 5๊ณผ์ œ 46/56์ด ์„ฑ๊ณตํ–ˆ๋‹ค.

๐Ÿ” Ping Review

๐Ÿ” Ping โ€” A light tap on the surface. Get the gist in seconds.

์‚ฌ๋žŒ ์†-๋ฌผ์ฒด ๋ชจ์บก(OakInk-v2, ARCTIC)์„ ๋กœ๋ด‡ ๋ฐ์ดํ„ฐ๋กœ ๋ฐ”๊พธ๋Š” ํŒŒ์ดํ”„๋ผ์ธ์€ ๋‘ ๋‹จ๊ณ„๋กœ ๋‚˜๋‰œ๋‹ค. kinematic retargeting์ด ๋กœ๋ด‡ ์†์ด ์–ด๋””์— ๊ฐ€์•ผ ํ•˜๋Š”์ง€๋ฅผ ์ •ํ•˜๊ณ , dynamic retargeting(๋Œ€๊ฐœ residual RL)์ด ์‹œ๋ฎฌ๋ ˆ์ดํ„ฐ ์•ˆ์—์„œ ์‹ค์ œ๋กœ ๋ฌผ์ฒด๋ฅผ ์›€์ง์ด๊ฒŒ ๋งŒ๋“ ๋‹ค. ๊ธฐ์กด์˜ kinematic ๋‹จ๊ณ„๋Š” AnyTeleopยทDexPilotยทGeoRT์ฒ˜๋Ÿผ ์† keypoint ๋ฒกํ„ฐ๋งŒ ๋งž์ถ”๊ณ  ๋ฌผ์ฒด๋ฅผ ๋ณด์ง€ ์•Š๋Š”๋‹ค. ๋ฌผ์ฒด์™€ ์ ‘์ด‰์€ ManipTransยทDexMachinaยทSPIDER์˜ physics ๋‹จ๊ณ„์—์„œ์•ผ ๋“ค์–ด์˜จ๋‹ค. ๊ทธ๋ž˜์„œ dynamic ๋‹จ๊ณ„๋Š” dynamics๋ฅผ ๋งž์ถ”๋Š” ์ผ๊ณผ โ€œ์–ด๋””๋ฅผ ์ฅ์–ด์•ผ ํ•˜๋Š”๊ฐ€โ€๋ฅผ ์ฐพ๋Š” ์ผ์„ ๋™์‹œ์— ๋– ์•ˆ๋Š”๋‹ค.

DexOAK์˜ ์ฃผ์žฅ์€ ๋‹จ์ˆœํ•˜๋‹ค. kinematic reference๊ฐ€ ์ด๋ฏธ ์‹œ์—ฐ๋œ ์ ‘์ด‰ ์œ„์น˜์— ์†๋์„ ์˜ฌ๋ ค ๋‘๋ฉด, RL์€ ์ ‘์ด‰ ํƒ์ƒ‰์—์„œ ํ’€๋ ค๋‚˜ dynamics ๋ณด์ •์—๋งŒ ์ง‘์ค‘ํ•  ์ˆ˜ ์žˆ๋‹ค. ์ด๋ฅผ ์œ„ํ•ด OmniRetarget์˜ interaction mesh๋ฅผ ์†๊ฐ€๋ฝ ๋‹จ์œ„๋กœ ๊ฐ€์ ธ์™€ seed reference๋ฅผ ๋งŒ๋“ค๊ณ , ๊ทธ ์ตœ์ ํ™”๋ฅผ ๋„คํŠธ์›Œํฌ๋กœ amortizeํ•œ ๋‹ค์Œ, ๋ฏธ๋ถ„ ๊ฐ€๋Šฅํ•œ ๋„คํŠธ์›Œํฌ๋ผ๋Š” ์ ์„ ์ด์šฉํ•ด ๋ฐ์ดํ„ฐ์…‹ ์ „์ฒด์— ๊ฑธ์นœ ์ ‘์ด‰ ์†์‹ค๋กœ ๋‹ค์‹œ fine-tuningํ•œ๋‹ค. ํ•™์Šต์ด ๋๋‚˜๋ฉด ์ƒˆ ์‹œ์—ฐ์—๋Š” per-frame ์ตœ์ ํ™” ์—†์ด forward pass๋งŒ ์“ด๋‹ค.


Fig. 1: ์‚ฌ๋žŒ ์‹œ์—ฐ(Unplug, Pour, Close Laptop)์„ DexOAK reference + RL๋กœ ๋ฐ”๊พผ ๊ถค์ ์„ UR5 + ์˜ค๋ฅธ์† Allegro์—์„œ open-loop๋กœ ์žฌ์ƒํ•œ ์žฅ๋ฉด. Close Laptop์€ retargeter ํ•™์Šต์—์„œ ์ œ์™ธ๋œ ๊ณผ์ œ๋‹ค.

ํ•ต์‹ฌ ๋ฐฉ๋ฒ•๋ก :

ํ”„๋ ˆ์ž„ t์—์„œ ์‚ฌ๋žŒ ์† keypoint(์†๋ยท์†๋ชฉ) M_h๊ฐœ์™€ ๋ฌผ์ฒด anchor M_o๊ฐœ๋กœ source ์ง‘ํ•ฉ \mathcal{S}_t๋ฅผ ๋งŒ๋“ค๊ณ , ์† keypoint๋งŒ ๋กœ๋ด‡์˜ ๋Œ€์‘์  r^i_t = w_t\,\mathrm{FK}^i(q_t)๋กœ ๋ฐ”๊พผ target ์ง‘ํ•ฉ \mathcal{T}_t(w_t,q_t)๋ฅผ ๋งŒ๋“ ๋‹ค. ๋‘ ์ง‘ํ•ฉ์— ๊ฐ™์€ Delaunay ์ด์›ƒ \mathcal{N}(i)๋ฅผ ์“ฐ๊ณ , ๊ฐ ์ ์˜ Laplacian ์ขŒํ‘œ \Delta_i=\frac{1}{|\mathcal{N}(i)|}\sum_{j\in\mathcal{N}(i)}(x_i-x_j)๋ฅผ ๋งž์ถ˜๋‹ค(Eq. 1).

\min_{w_t,\,q_t}\;\underbrace{\tfrac{1}{N}\sum_{i=1}^{N}\bigl\lVert\Delta_i(\mathcal{S}_t)-\Delta_i(\mathcal{T}_t(w_t,q_t))\bigr\rVert_2^2}_{\mathcal{L}_{\mathrm{IG}}}+\mathcal{L}_{\mathrm{coll}}+\mathcal{L}_{\mathrm{temp}}\quad\text{s.t.}\;\; q_t\in\mathcal{Q},\;\; |q_t-q^\star_{t-1}|\preceq \mathbf{v}_{\max}\,\mathrm{d}t

์ด ํ•ด \tau^\star๋ฅผ ์ง€๋„ ์‹ ํ˜ธ๋กœ ๋„คํŠธ์›Œํฌ f_\theta(K_t,P_t,\hat q_{t-1},\hat w_{t-1})=(\hat w_t,\hat q_t)๋ฅผ ํ•™์Šตํ•œ ๋’ค, ์‚ฌ๋žŒ ์† ์ ์ด ๋ฌผ์ฒด ํ‘œ๋ฉด \epsilon=15 mm ์•ˆ์— ์žˆ์„ ๋•Œ๋งŒ ์ผœ์ง€๋Š” gate \gamma^i_t๋กœ ์„ธ ์ ‘์ด‰ ์†์‹ค์„ ๊ฑด๋‹ค(Eq. 2~4).

\mathcal{L}_{\mathrm{surf}}=\sum_i\gamma^i_t\,d(r^i_t)^2,\qquad \mathcal{L}_{\mathrm{part}}=\sum_i\gamma^i_t\bigl\lVert r^i_t-o(k^i_t)\bigr\rVert_2^2,\qquad \mathcal{L}_{\mathrm{dir}}=\sum_i\gamma^i_t\,(1-\cos\theta^i_t)

d(\cdot)๋Š” ๋ฌผ์ฒด ์ ๊ตฐ๊นŒ์ง€์˜ ๊ฑฐ๋ฆฌ, o(k^i_t)๋Š” ์‚ฌ๋žŒ ์ ‘์ด‰์ ์— ๊ฐ€์žฅ ๊ฐ€๊นŒ์šด ๋ฌผ์ฒด ์ , \theta^i_t๋Š” r^i_t-o(k^i_t)์™€ k^i_t-o(k^i_t) ์‚ฌ์ด์˜ ๊ฐ๋„๋‹ค. ์—ฌ๊ธฐ์— ์ถฉ๋Œยท์‹œ๊ฐ„ ํ‰ํ™œยทsupervised ์ถœ๋ ฅ anchor ํ•ญ์„ ๋”ํ•œ ๊ฐ€์ค‘ํ•ฉ์ด fine-tuning ๋ชฉ์ ํ•จ์ˆ˜๋‹ค.

์ฃผ์š” ๊ฒฐ๊ณผ:

  • Object awareness์˜ ํšจ๊ณผ(Table I, Allegro ์˜ค๋ฅธ์† OakInk-v2, ๋ชจ๋‘ ๊ฐ™์€ ManipTrans residual RL): \mathrm{SR}_{\mathrm{MT}} hand-only ์ตœ๊ณ  20.6%(ManipTrans imitator) โ†’ IG optimizer 43.2% โ†’ DexOAK 48.2%. contact completeness 5.8% โ†’ 21.9%, tip-position error 49.2 mm โ†’ 3.21 mm.
  • Cross-dataset transfer(Table III, Allegro ARCTIC, ๋ชจ๋‘ DexMachina RL): OakInk-v2 ๋ชจ๋ธ zero-shot 87.9 ADD-AUC(108.9 Hz), ARCTIC fine-tuning ํ›„ 90.0, ARCTIC in-domain DexOAK 91.3, DexMachina 83.4.
  • SOTA ์‹œ์Šคํ…œ ๋น„๊ต(Table IV, ๊ฐ์ž์˜ RL): Inspire OakInk-v2 \mathrm{SR}_{\mathrm{MT}} 39.5(ManipTrans) โ†’ 48.4, Inspire ARCTIC ADD-AUC 62.7(DexMachina) โ†’ 77.9, \mathrm{SR}_{\mathrm{obj}} 42.0(SPIDER) โ†’ 83.4.
  • ์‹ค๊ธฐ(Table V, UR5 + ์˜ค๋ฅธ์† Allegro): 5๊ณผ์ œ open-loop replay 46/56(ManipTrans 26/40), held-out Close LaptopยทPress Button ๊ฐ 10/12.

๊ฒฐ๋ก :

์ƒˆ๋กœ์šด ๊ฒƒ์€ Laplacian ๋ชฉ์ ํ•จ์ˆ˜ ์ž์ฒด๋ณด๋‹ค โ€œkinematic reference๋ฅผ ์ ‘์ด‰ ๋ชฉ์ ํ•จ์ˆ˜๋กœ ๋‹ค๋“ฌ์„ ์ˆ˜ ์žˆ๋Š” ๋Œ€์ƒโ€์œผ๋กœ ๋ฐ”๊พผ ๊ฒƒ์ด๋‹ค. ์ตœ์ ํ™” ํ•ด๋Š” ํ”„๋ ˆ์ž„๋งˆ๋‹ค ๋”ฐ๋กœ ํ’€๋ฆฌ๊ณ  ์ดˆ๊ธฐ๊ฐ’์— ๋ฏผ๊ฐํ•˜์ง€๋งŒ, ๊ทธ ํ•ด๋ฅผ ํ‰๋‚ด ๋‚ธ ๋„คํŠธ์›Œํฌ๋Š” ๋ฏธ๋ถ„ ๊ฐ€๋Šฅํ•˜๊ณ  ๋ฐ์ดํ„ฐ์…‹ ์ „์ฒด๋ฅผ ๊ณต์œ ํ•˜๋ฏ€๋กœ ์ ‘์ด‰ ์†์‹ค๋กœ ํ•œ ๋ฒˆ ๋” ๋ฐ€์–ด ์˜ฌ๋ฆด ์ˆ˜ ์žˆ๋‹ค. ๋‹ค๋งŒ Table I์„ ๋ถ„ํ•ดํ•˜๋ฉด ์ด๋“์˜ ๋Œ€๋ถ€๋ถ„(+27.6 %p ์ค‘ +22.6 %p)์€ object anchor๋ฅผ ๋„ฃ์€ ์ตœ์ ํ™” ๋‹จ๊ณ„์—์„œ ๋‚˜์˜ค๊ณ , ํ•™์Šต๋œ retargeter์™€ fine-tuning์ด ๋”ํ•˜๋Š” ๋ชซ์€ +5.0 %p๋‹ค. ํ—ค๋“œ๋ผ์ธ SOTA ๋น„๊ต(Table IV)๋Š” reference์™€ RL backend๊ฐ€ ํ•จ๊ป˜ ๋ฐ”๋€ ์‹œ์Šคํ…œ ๋น„๊ต๋ผ์„œ, reference์˜ ์ˆœ์ˆ˜ ๊ธฐ์—ฌ๋Š” Table IยทIII์—์„œ ์ฝ์–ด์•ผ ํ•œ๋‹ค.


๐Ÿ”” Ring Review

๐Ÿ”” Ring โ€” An idea that echoes. Grasp the core and its value.

ํ•œ ์ค„๋กœ ์‹œ์ž‘ํ•˜๋ฉด

DexOAK์˜ ํ†ต์ฐฐ์€ โ€œkinematic retargeting์˜ ๋ชฉํ‘œ๋Š” ์† ๋ชจ์–‘ ์žฌํ˜„์ด ์•„๋‹ˆ๋ผ ๊ณผ์ œ ๊ธฐ๋Šฅ์˜ ๋ณด์กด์ด๋‹คโ€๋ผ๋Š” ๋ฌธ์žฅ ํ•˜๋‚˜๋กœ ์š”์•ฝ๋œ๋‹ค(ยงI: โ€œkinematic retargeting should preserve task function rather than hand-pose fidelityโ€). ์‚ฌ๋žŒ๊ณผ ๋กœ๋ด‡์˜ ์†์€ ์ž์œ ๋„๋„ ํฌ๊ธฐ๋„ ๋‹ค๋ฅด๋ฏ€๋กœ ์† ๋ชจ์–‘์„ ๋‹ฎ๊ฒŒ ๋งŒ๋“ค์–ด๋„ ๊ฐ™์€ ์ ‘์ด‰์ด ๋‚˜์˜ค์ง€ ์•Š๋Š”๋‹ค. ๊ทธ๋ ‡๋‹ค๋ฉด ๋งž์ถฐ์•ผ ํ•  ๊ฒƒ์€ ์† ๊ด€์ ˆ์ด ์•„๋‹ˆ๋ผ ๋ฌผ์ฒด์— ๋Œ€ํ•œ ์†๋์˜ ์ƒ๋Œ€ ๋ฐฐ์น˜, ์ฆ‰ ์–ด๋А ๋ฉด์„ ์–ด๋А ๋ฐฉํ–ฅ์—์„œ ๋ˆ„๋ฅด๋Š”์ง€๋‹ค. ์ด ๊ด€์ ์„ ์„ธ ๊ฒน์œผ๋กœ ๊ตฌํ˜„ํ•œ๋‹ค. ์ฒซ์งธ, ์†๋๊ณผ ๋ฌผ์ฒด ์ ์„ ํ•œ ๊ทธ๋ž˜ํ”„์— ๋„ฃ์–ด ์ƒ๋Œ€ ๋ฐฐ์น˜๋ฅผ Laplacian ์ขŒํ‘œ๋กœ ๊ณ ์ •ํ•˜๋Š” ์ตœ์ ํ™”(seed). ๋‘˜์งธ, ๊ทธ ์ตœ์ ํ™”๋ฅผ ๋„คํŠธ์›Œํฌ๋กœ amortizeํ•ด ์ƒˆ ๋ฐ์ดํ„ฐ์— forward pass๋งŒ์œผ๋กœ ์ ์šฉ(์žฌ์‚ฌ์šฉ์„ฑ). ์…‹์งธ, ๋„คํŠธ์›Œํฌ๊ฐ€ ๋ฏธ๋ถ„ ๊ฐ€๋Šฅํ•˜๋‹ค๋Š” ์ ์„ ์ด์šฉํ•ด ์ ‘์ด‰ ์†์‹ค๋กœ ์ตœ์ ํ™” ํ•ด๋ณด๋‹ค ๋” ๋‚˜์€ reference๋ฅผ ๋งŒ๋“œ๋Š” fine-tuning(ํ’ˆ์งˆ). ๊ทธ๋ฆฌ๊ณ  ์ด reference๋ฅผ residual RL์˜ base action์œผ๋กœ ๋„ฃ์œผ๋ฉด RL์˜ ๋ถ€๋‹ด์ด ์ค„์–ด๋“ ๋‹ค๋Š” ๊ฒƒ์ด ๋…ผ๋ฌธ์˜ ๊ฐ€์„ค์ด์ž ์‹คํ—˜์˜ ์ถ•์ด๋‹ค.

๋ฐฐ๊ฒฝ: kinematic ๋‹จ๊ณ„๋Š” ์™œ ๋ฌผ์ฒด๋ฅผ ๋ณด์ง€ ์•Š์•˜๋‚˜ (ยงI, ยงII)

๋…ผ๋ฌธ์€ ๋ฆฌํƒ€๊ฒŒํŒ… ํŒŒ์ดํ”„๋ผ์ธ์„ ๋‘ ๋‹จ๊ณ„๋กœ ๋‚˜๋ˆˆ๋‹ค. kinematic retargeting์€ ๋กœ๋ด‡ ์†์ด ํ™˜๊ฒฝ์— ๋Œ€ํ•ด ์–ด๋””์— ์žˆ์–ด์•ผ ํ•˜๋Š”์ง€, dynamic retargeting์€ ๋ฌผ์ฒด์™€ ์ƒํ˜ธ์ž‘์šฉํ•˜๊ธฐ ์œ„ํ•ด ์–ด๋–ป๊ฒŒ ๊ตฌ๋™ํ•ด์•ผ ํ•˜๋Š”์ง€๋ฅผ ์ •ํ•œ๋‹ค(ยงI). ๊ธฐ์กด ๊ณ„๋ณด๋ฅผ ์ด ์ถ•์œผ๋กœ ์ •๋ฆฌํ•˜๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

๋‹จ๊ณ„ ๋Œ€ํ‘œ ๋ฐฉ๋ฒ• ๋ฌด์—‡์„ ๋งž์ถ”๋Š”๊ฐ€ ๋ฌผ์ฒด ์ •๋ณด
kinematic, ์ตœ์ ํ™” AnyTeleop(PositionยทVector), DexPilot ์† keypoint ์œ„์น˜ยท๋ฒกํ„ฐ ์—†์Œ
kinematic, ํ•™์Šต GeoRT ์‚ฌ๋žŒ ์†๋ โ†’ ๋กœ๋ด‡ ๊ด€์ ˆ์˜ ๊ธฐํ•˜ ๋ชฉ์ ํ•จ์ˆ˜ ์—†์Œ
kinematic, interaction mesh OmniRetarget ๋ชธ-๋ฌผ์ฒด-์ง€ํ˜• Laplacian ์ขŒํ‘œ ์žˆ์Œ(๋‹จ, ํœด๋จธ๋…ธ์ด๋“œ ์ „์‹ ยท์†๋ชฉ ๋…ธ๋“œ ํ•˜๋‚˜)
dynamic, residual RL ManipTrans ์† ๋ชจ๋ฐฉ ์ •์ฑ… + ๊ณผ์ œ๋ณ„ residual RL physics ๋‹จ๊ณ„์—์„œ๋งŒ
dynamic, RL + curriculum DexMachina ๊ฐ€์ƒ object control์„ ์ ์  ์ œ๊ฑฐํ•˜๋Š” curriculum physics ๋‹จ๊ณ„์—์„œ๋งŒ
dynamic, ์ƒ˜ํ”Œ๋ง SPIDER ๊ฐ€์ƒ contact guidance๋ฅผ ๋‹จ annealed sampling physics ๋‹จ๊ณ„์—์„œ๋งŒ

๋…ผ๋ฌธ์ด ์งš๋Š” ๋นˆ์นธ์€ ๋งˆ์ง€๋ง‰ ์„ธ ์ค„์˜ ๊ณตํ†ต์ ์ด๋‹ค. dynamic ๋‹จ๊ณ„ ๋ฐฉ๋ฒ•๋“ค์€ โ€œreceive an object-agnostic reference and must discover contacts themselvesโ€(ยงII). ์ฆ‰ ๋ฌผ๋ฆฌ์  ํƒ€๋‹น์„ฑ์€ ํšŒ๋ณตํ•˜์ง€๋งŒ ์ถœ๋ฐœ์ ์ด ๋˜๋Š” reference์—๋Š” ์ ‘์ด‰ ๊ธฐํ•˜๊ฐ€ ์—†๋‹ค. OmniRetarget์€ ๋ฌผ์ฒด๋ฅผ kinematic ๋‹จ๊ณ„์— ๋„ฃ์—ˆ์ง€๋งŒ ์†์„ ์†๋ชฉ ๋…ธ๋“œ ํ•˜๋‚˜๋กœ ํ‘œํ˜„ํ•ด ์†๊ฐ€๋ฝ ๋‹จ์œ„ ์ ‘์ด‰์„ ํ’€์ง€ ๋ชปํ•œ๋‹ค.

๋™์‹œ๋Œ€ ์—ฐ๊ตฌ๋„ ๋ช…์‹œํ•œ๋‹ค. TopoRetarget, C2Dex, ๊ทธ๋ฆฌ๊ณ  ์ด ๋ธ”๋กœ๊ทธ์—์„œ ๋‹ค๋ฃฌ Regrind(A Minimalist Retargeting-Guided RL Recipe)๊ฐ€ ๊ฐ™์€ Laplacian ๊ณ„์—ด ๋ชฉ์ ํ•จ์ˆ˜๋กœ ์†-๋ฌผ์ฒด ๊ตฌ์กฐ๋ฅผ ๋ณด์กดํ•œ๋‹ค. DexOAK๊ฐ€ ๋‚ด์„ธ์šฐ๋Š” ์ฐจ๋ณ„์ ์€ ์ด๋“ค์ด โ€œre-optimize each new frame or demonstration before RLโ€์ธ ๋ฐ˜๋ฉด DexOAK๋Š” ์ตœ์ ํ™”๋ฅผ ํ•™์Šต์šฉ seed๋ฅผ ๋งŒ๋“œ๋Š” ๋ฐ๋งŒ ์“ฐ๊ณ , ์ดํ›„์—๋Š” ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ retargeter๋ฅผ ๋ฐ์ดํ„ฐ์…‹ ๋‹จ์œ„ ์ ‘์ด‰ ์†์‹ค๋กœ fine-tuningํ•œ๋‹ค๋Š” ์ ์ด๋‹ค(ยงII). ์‹คํ—˜์—์„œ ์ด ๋™์‹œ๋Œ€ ๋ฐฉ๋ฒ•๋“ค๊ณผ์˜ ์ง์ ‘ ๋น„๊ต๋Š” ์—†๋‹ค.

๋ฐฉ๋ฒ• ์ƒ์„ธ (ยงIII)


Fig. 2: DexOAK ๊ฐœ์š”. ์™ผ์ชฝ์€ ์‚ฌ๋žŒ-๋ฌผ์ฒด graph์™€ ๋กœ๋ด‡-๋ฌผ์ฒด graph์˜ Laplacian ์ขŒํ‘œ๋ฅผ ๋งž์ถ”๋Š” per-frame ์ตœ์ ํ™”๋กœ seed reference๋ฅผ ๋งŒ๋“œ๋Š” ๋‹จ๊ณ„, ์˜ค๋ฅธ์ชฝ์€ ๊ทธ seed๋กœ supervised ํ•™์Šตํ•œ ๋’ค surfaceยทobject-partยทcontact-direction ์†์‹ค๋กœ fine-tuningํ•˜๋Š” ํ•™์Šต๋œ retargeter๋‹ค. ๋„คํŠธ์›Œํฌ๋Š” ๋ฌผ์ฒด feature, ์‚ฌ๋žŒ ์† feature, ์ง์ „ ํ”„๋ ˆ์ž„ ๋กœ๋ด‡ ์ƒํƒœ๋ฅผ ๋ฐ›์•„ ํ˜„์žฌ ๋กœ๋ด‡ ์ƒํƒœ๋ฅผ ๋‚ธ๋‹ค. residual RL ๋‹จ๊ณ„(ยงIV)๋Š” ๊ทธ๋ฆผ์—์„œ ๋น ์ ธ ์žˆ๋‹ค.

๋ฌธ์ œ ์„ค์ •

์ž…๋ ฅ์€ Tํ”„๋ ˆ์ž„ ์‚ฌ๋žŒ ์†-๋ฌผ์ฒด ๋ชจ์บก์ด๋‹ค. ํ”„๋ ˆ์ž„ t์—์„œ SMPL-X์—์„œ ๋ฝ‘์€ ์† keypoint K_t\in\mathbb{R}^{N_h\times 3}(N_h=21)๊ณผ ๋ฌผ์ฒด ๋ฉ”์‹œ์—์„œ ๋ฝ‘์€ ์ ๊ตฐ P_t\in\mathbb{R}^{N_p\times 3}(N_p=1024)์ด ๋ชจ๋‘ ์‚ฌ๋žŒ ์†๋ชฉ ์ขŒํ‘œ๊ณ„(W_t\in\mathrm{SE}(3))๋กœ ์ฃผ์–ด์ง„๋‹ค. ์ถœ๋ ฅ์€ ๋กœ๋ด‡ ์†๋ชฉ ์ž์„ธ w_t=(p_t,R_t)\in\mathrm{SE}(3)์™€ ๊ด€์ ˆ๊ฐ q_t\in\mathcal{Q}=[q_{\mathrm{low}},q_{\mathrm{high}}]\subset\mathbb{R}^J์ด๋‹ค. w_t๊ฐ€ ์†๋ชฉ ์ƒ๋Œ€๋Ÿ‰์ด๋ฏ€๋กœ ์บก์ฒ˜ ์ขŒํ‘œ๊ณ„์˜ ๋กœ๋ด‡ ์†๋ชฉ์€ W_t\,w_t๊ฐ€ ๋˜๊ณ , ์ „์—ญ ๋ฐฐ์น˜์— ๋ถˆ๋ณ€์ด๋‹ค. ๊ถค์ ์€ ์†๋งˆ๋‹ค ํ•˜๋‚˜์”ฉ ๋งŒ๋“ค๋ฉฐ, ์–‘์† ๊ฒฐํ•ฉ์€ dynamic ๋‹จ๊ณ„์—์„œ๋งŒ ์ผ์–ด๋‚œ๋‹ค(ยงIII ์„œ๋‘).

III-A. Interaction-graph Laplacian ์ตœ์ ํ™”

Graph ๊ตฌ์„ฑ. source ์ง‘ํ•ฉ \mathcal{S}_t๋Š” ์‚ฌ๋žŒ ์† keypoint M_h๊ฐœ(์†๋ + ์†๋ชฉ)์™€ ๋ฌผ์ฒด anchor M_o=64๊ฐœ, target ์ง‘ํ•ฉ \mathcal{T}_t๋Š” ๊ฐ™์€ anchor์— ๋กœ๋ด‡ ๋Œ€์‘์ ์„ ๋„ฃ๋Š”๋‹ค. ๊ฐ™์€ ์ธ๋ฑ์Šค๋Š” ํ•ด๋ถ€ํ•™์  ๋Œ€์‘(์—„์ง€โ†”๏ธŽ์—„์ง€)์ด๋‹ค. ๋ถ€๋ก Table S1์— ๋”ฐ๋ฅด๋ฉด Allegro๋Š” ์†๊ฐ€๋ฝ์ด ๋„ท์ด๋ผ ์ƒˆ๋ผ์†๊ฐ€๋ฝ์„ ๋ฒ„๋ ค M_h=5, Inspire๋Š” M_h=6์ด๋‹ค.

์ด์›ƒ๊ณผ Laplacian ์ขŒํ‘œ. OmniRetarget์„ ๋”ฐ๋ผ \mathcal{S}_t์— 3D Delaunay tetrahedralization์„ ๊ฑธ๊ณ , ๊ทธ edge๋ฅผ target์—๋„ ๊ทธ๋Œ€๋กœ ์“ด๋‹ค. ๋กœ๋ด‡์ด ์ƒˆ ์ด์›ƒ ๊ด€๊ณ„๋ฅผ ๋งŒ๋“ค์ง€ ์•Š๊ณ  ์‚ฌ๋žŒ์ด ๋งŒ๋“  ์ด์›ƒ ๊ด€๊ณ„๋ฅผ ์žฌํ˜„ํ•˜๊ฒŒ ํ•˜๋ ค๋Š” ๊ฒƒ์ด๋‹ค. Laplacian ์ขŒํ‘œ \Delta_i๋Š” ์ด์›ƒ ์ค‘์‹ฌ์—์„œ์˜ offset์ด๋ผ ์ „์—ญ ํ‰ํ–‰์ด๋™์— ๋ถˆ๋ณ€์ด๊ณ , ๋ฌผ์ฒด anchor๊ฐ€ ๊ทธ๋ž˜ํ”„์— ๋“ค์–ด ์žˆ์œผ๋ฏ€๋กœ ์†๋์˜ ์ขŒํ‘œ๊ฐ€ ๋ฌผ์ฒด์— ๋Œ€ํ•œ ์ƒ๋Œ€ ์œ„์น˜๋ฅผ ๋‹ด๊ฒŒ ๋œ๋‹ค. ์ด๊ฒƒ์ด โ€œIG (w/o object)โ€์™€ โ€œIG (+ object)โ€๋ฅผ ๊ฐ€๋ฅด๋Š” ์ง€์ ์ด๋‹ค(Fig. 3).

OmniRetarget ๋Œ€๋น„ ์„ธ ๊ฐ€์ง€ ํ™•์žฅ(ยงIII-A).

ํ•ญ๋ชฉ OmniRetarget DexOAK
์† ํ‘œํ˜„ ์†๋ชฉ ๋…ธ๋“œ ํ•˜๋‚˜ ์†๋ + ์†๋ชฉ(์†๊ฐ€๋ฝ ๋‹จ์œ„ ์ ‘์ด‰)
๋ฌผ์ฒด ๋ฌผ์ฒด ํ•˜๋‚˜ + ์ง€ํ˜• ์žฅ๋ฉด์˜ ๋ชจ๋“  ๋ฌผ์ฒด๋ฅผ ํ•œ graph์—
์Šค์ผ€์ผ ์‚ฌ๋žŒ keypoint๋ฅผ ํ‚ค ๋น„์œจ๋กœ ๋กœ๋ด‡์— ๋งž๊ฒŒ ์ถ•์†Œยทํ™•๋Œ€ ๋ฌผ์ฒด์— ๋Œ€ํ•œ ์‹ค์ œ ์Šค์ผ€์ผ ์œ ์ง€

์„ธ ๋ฒˆ์งธ๊ฐ€ ์† ์กฐ์ž‘์—์„œ ํŠนํžˆ ์ค‘์š”ํ•˜๋‹ค. ์† ์ „์ฒด๋ฅผ ๋น„์œจ๋กœ ์ค„์ด๋ฉด ์†๋๊ณผ ๋ฌผ์ฒด ํ‘œ๋ฉด ์‚ฌ์ด ๊ฑฐ๋ฆฌ๊ฐ€ ํ•จ๊ป˜ ๋ฐ”๋€Œ์–ด ์ ‘์ด‰์ด ๊นจ์ง„๋‹ค.

Per-frame ์ตœ์ ํ™”. ๋ชฉ์ ํ•จ์ˆ˜๋Š” Laplacian ํ•ญ, ์ถฉ๋Œ hinge ํ•ญ \mathcal{L}_{\mathrm{coll}}=\sum_c\max(0,-g_c)^2(g_c๋Š” ์ถฉ๋Œ ์Œ ๊ฑฐ๋ฆฌ โˆ’ safety margin, ๋กœ๋ด‡-๋ฌผ์ฒดยท์ž๊ธฐ์ถฉ๋Œ ํฌํ•จ, ์ธ์ ‘ ๋งํฌ ์ œ์™ธ), ์‹œ๊ฐ„ ํ‰ํ™œ ํ•ญ \mathcal{L}_{\mathrm{temp}}=\lVert q_t-q^\star_{t-1}\rVert^2+\lVert p_t-p^\star_{t-1}\rVert^2+d^2_{\mathrm{rot}}(R^\star_{t-1},R_t)์˜ ํ•ฉ์ด๋‹ค. ๊ด€์ ˆ ๋ฒ”์œ„์™€ ๊ด€์ ˆ ์†๋„ ํ•œ๊ณ„๋งŒ hard constraint์ด๊ณ , ์†๋ชฉ ์šด๋™์€ \mathcal{L}_{\mathrm{temp}}๋กœ๋งŒ softํ•˜๊ฒŒ ๋ฌถ์ธ๋‹ค. ๊ฐ ํ”„๋ ˆ์ž„์„ SLSQP๋กœ ํ’€๋˜ ์ง์ „ ํ•ด๋กœ warm-startํ•œ๋‹ค.

์—ฌ๊ธฐ์„œ OmniRetarget๊ณผ์˜ ์ฐจ์ด๋ฅผ ํ•˜๋‚˜ ๋” ์งš์–ด ๋‘”๋‹ค. OmniRetarget์€ ๋น„๊ด€ํ†ต์„ hard constraint๋กœ ๊ฑธ๊ณ  sequential SOCP๋กœ ํ’€์–ด โ€œ์•„ํ‹ฐํŒฉํŠธ๋ฅผ ์ •์˜์ƒ ์ œ๊ฑฐโ€ํ•˜๋Š” ์ชฝ์ด์—ˆ๋‹ค. DexOAK๋Š” ์ถฉ๋Œ์„ soft penalty๋กœ ๋‘”๋‹ค. ๊ทธ๋ž˜์„œ seed reference์—๋„ ๊ด€ํ†ต์ด ๋‚จ์„ ์ˆ˜ ์žˆ๊ณ , ๋’ค์˜ fine-tuning์ด ๊ฐ™์€ \mathcal{L}_{\mathrm{coll}}์„ ๋‹ค์‹œ ์“ฐ๋Š” ์ด์œ ๊ฐ€ ๋œ๋‹ค. ๋…ผ๋ฌธ์€ ์ด ์„ ํƒ์˜ ๊ทผ๊ฑฐ๋ฅผ ๋”ฐ๋กœ ๋งํ•˜์ง€ ์•Š๋Š”๋‹ค.


Fig. 3: OakInk-v2 ๋‘ ์žฅ๋ฉด(์œ„: ignite, Allegro / ์•„๋ž˜: unplug, Inspire ์–‘์†). ์™ผ์ชฝ๋ถ€ํ„ฐ ์‚ฌ๋žŒ ์‹œ์—ฐ, ๋ฌผ์ฒด๋ฅผ ๋ณด์ง€ ์•Š๋Š” AnyTeleop-VectorยทManipTrans imitatorยทIG (w/o object), ๊ทธ๋ฆฌ๊ณ  DexOAK์˜ IG (+ object) โ†’ Object-Aware FT โ†’ RL. hand-only ๋ฐฉ๋ฒ•์€ ์†๋์ด ๋ฌผ์ฒด ๋ฐ–์— ๋œจ๊ฑฐ๋‚˜ ์•ˆ์œผ๋กœ ํŒŒ๊ณ ๋“ค๊ณ , object-aware ๋‹จ๊ณ„๋ฅผ ๊ฑฐ์น ์ˆ˜๋ก ์†๋์ด ์†์žก์ด ์œ„๋กœ ๋ชจ์ธ๋‹ค.

III-B. ํ•™์Šต๋œ object-aware retargeter

์™œ amortizeํ•˜๋Š”๊ฐ€. ๋…ผ๋ฌธ์ด ๋“œ๋Š” ์ด์œ ๋Š” ๋‘ ๊ฐ€์ง€๋‹ค. ์ฒซ์งธ, ์ด ์ตœ์ ํ™”๋Š” ์ดˆ๊ธฐ๊ฐ’์— ๋ฏผ๊ฐํ•˜๊ณ  ์ ‘์ด‰ยท์ถฉ๋Œ ํ•ญ์„ ๋” ๋„ฃ์„์ˆ˜๋ก ๋น„๋ณผ๋ก์„ฑ์ด ์ปค์ง„๋‹ค(ยงI). ๊ทธ๋ž˜์„œ ์ตœ์ ํ™” ์•ˆ์— ์ ‘์ด‰ ์†์‹ค์„ ์ง์ ‘ ๋„ฃ๋Š” ๋Œ€์‹ , ์ตœ์ ํ™” ํ•ด๋ฅผ ํ‰๋‚ด ๋‚ด๋Š” ๋„คํŠธ์›Œํฌ๋ฅผ ๋จผ์ € ๋งŒ๋“ค๊ณ  ๊ทธ ์œ„์—์„œ ์ ‘์ด‰ ์†์‹ค์„ ๊ฑด๋‹ค. ๋‘˜์งธ, ์—ฌ๋Ÿฌ ์‹œํ€€์Šค์™€ ๊ณผ์ œ์— ๊ฑธ์ณ ํ•˜๋‚˜์˜ ํ•จ์ˆ˜๋ฅผ ํ•™์Šตํ•˜๋ฏ€๋กœ ์ƒˆ ๋ฐ์ดํ„ฐ์— ๋‹ค์‹œ ์ตœ์ ํ™”ํ•  ํ•„์š”๊ฐ€ ์—†๋‹ค.

๊ตฌ์กฐ. ๊ฐ keypoint๋Š” ์†๋ชฉ ์ขŒํ‘œ๊ณ„์˜ 12์ฐจ์› ์†์„ฑ(์œ„์น˜ 3 + SMPL-X ๊ด€์ ˆ ๋ฐฉํ–ฅ์˜ 6D rotation 6 + ์„ ์†๋„ 3)์œผ๋กœ ํ‘œํ˜„ํ•œ๋‹ค. ๋ฌผ์ฒด ์ ๊ตฐ์€ PointNet์‹ encoder(๊ณต์œ  pointwise MLP + global max pooling)๋กœ ์š”์•ฝํ•˜๊ณ , keypoint ์†์„ฑ์„ ํŽผ์นœ ๋ฒกํ„ฐ์™€ ์ด์–ด ๋ถ™์—ฌ ํ”„๋ ˆ์ž„ feature \phi_t๋ฅผ ๋งŒ๋“ ๋‹ค. ๊ด€์ ˆ decoder์™€ ์†๋ชฉ decoder๋Š” ๊ฐ๊ฐ \phi_t์— ์ž๊ธฐ ์ง์ „ ์˜ˆ์ธก \hat q_{t-1} ๋˜๋Š” \hat w_{t-1}์„ ์ด์–ด ๋ถ™์—ฌ ์ฝ๋Š”๋‹ค. ์ด autoregressive ์กฐ๊ฑดํ™”๋Š” ์ตœ์ ํ™”์˜ warm-start์— ๋Œ€์‘ํ•˜๋ฉฐ, teacher forcing ์—†์ด free-running์œผ๋กœ ํ•™์Šตํ•ด ํ•™์Šต๊ณผ ๋ฐฐํฌ๊ฐ€ ๊ฐ™์€ ์ž…๋ ฅ ๋ถ„ํฌ๋ฅผ ๋ณด๊ฒŒ ํ•œ๋‹ค. ์ฒซ ํ”„๋ ˆ์ž„์€ 0 ๋ฒกํ„ฐ๋กœ ์กฐ๊ฑดํ™”ํ•œ๋‹ค. ๊ด€์ ˆ ์ถœ๋ ฅ์€ ๊ด€์ ˆ๋ณ„ sigmoid๋กœ \hat q_t=q_{\mathrm{low}}+\sigma(\cdot)(q_{\mathrm{high}}-q_{\mathrm{low}})๋ฅผ ๋งŒ๋“ค์–ด joint-limit ์†์‹ค ์—†์ด ๋ฒ”์œ„๋ฅผ ์ง€ํ‚จ๋‹ค. ์†๋ชฉ์€ ์œ„์น˜ 3 + 6D rotation์„ ๋‚ด๊ณ  \mathrm{SO}(3)๋กœ ๋ณ€ํ™˜ํ•œ๋‹ค.

III-B1. Supervised ํ•™์Šต

์†์‹ค์€ ๊ด€์ ˆ L_1๊ณผ ์†๋ชฉ ์†์‹ค(์œ„์น˜ L_1 + \mathrm{SO}(3) geodesic ๊ฑฐ๋ฆฌ)์˜ ๋™์ผ ๊ฐ€์ค‘ ํ•ฉ์ด๋‹ค. ์ด ๋‹จ๊ณ„์˜ ๋ชฉํ‘œ๋Š” seed๋ฅผ ๊ฐœ์„ ํ•˜๋Š” ๊ฒƒ์ด ์•„๋‹ˆ๋ผ ์žฌํ˜„ํ•˜๋Š” ๊ฒƒ์ด๊ณ (Table I์—์„œ ์‹ค์ œ๋กœ ๊ฑฐ์˜ ๊ฐ™์€ ๊ฐ’์„ ๋‚ธ๋‹ค), ์—ญํ• ์€ fine-tuning์˜ ์ดˆ๊ธฐํ™”๋‹ค. ๋…ผ๋ฌธ์€ ์ด ์ดˆ๊ธฐํ™” ์—†์ด fine-tuning๋งŒ ํ•˜๋ฉด โ€œtoo weakly constrained to convergeโ€๋ผ๊ณ  ์“ฐ๊ณ , Table II์˜ โ€œwithout supervised trainingโ€(CC 10.3%, TPE 36.8 mm)์ด ๊ทธ ๊ทผ๊ฑฐ๋‹ค.

III-B2. Object-aware fine-tuning

Laplacian ์ขŒํ‘œ ์ผ์น˜๋Š” ์†-๋ฌผ์ฒด ์ด์›ƒ ๊ด€๊ณ„๋ฅผ ๋ณด์กดํ•  ๋ฟ, ์†๋์„ ํ‘œ๋ฉด ์œ„์— ์˜ฌ๋ ค๋†“์ง€๋Š” ์•Š๋Š”๋‹ค(๋…ผ๋ฌธ์€ Fig. 3์˜ IG (+ object)์—์„œ ์ ‘์ด‰์ด ์•„์ง ๋ถ€์ •ํ™•ํ•˜๊ฒŒ ๋‚จ๋Š”๋‹ค๊ณ  ์“ด๋‹ค). fine-tuning์€ ์ด ๋งˆ์ง€๋ง‰ ๋ช‡ mm๋ฅผ ์ง์ ‘ ๊ฒจ๋ƒฅํ•œ๋‹ค. ์„ธ ์ ‘์ด‰ ์†์‹ค์˜ ์—ญํ• ์„ ๋‚˜๋ˆ„๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

์†์‹ค ๋Œ์–ด๋‹น๊ธฐ๋Š” ๊ณณ ํ•ด๊ฒฐํ•˜๋ ค๋Š” ์‹คํŒจ
\mathcal{L}_{\mathrm{surf}} ๊ฐ€์žฅ ๊ฐ€๊นŒ์šด ๋ฌผ์ฒด ํ‘œ๋ฉด ์†๋์ด ํ‘œ๋ฉด์—์„œ ๋œธ
\mathcal{L}_{\mathrm{part}} ์‚ฌ๋žŒ ์ ‘์ด‰์ ์— ๊ฐ€์žฅ ๊ฐ€๊นŒ์šด ๋ฌผ์ฒด ์  o(k^i_t) ํ‘œ๋ฉด์—๋Š” ๋ถ™์—ˆ์ง€๋งŒ ์—‰๋šฑํ•œ ๋ถ€์œ„์— ๋ถ™์Œ
\mathcal{L}_{\mathrm{dir}} ์‚ฌ๋žŒ๊ณผ ๊ฐ™์€ ์ชฝ์—์„œ o(k^i_t)๋กœ ์ ‘๊ทผ ์–‡์€ ๋ฌผ์ฒด์˜ ๋ฐ˜๋Œ€ ๋ฉด์— ๋ถ™์Œ

์„ธ ํ•ญ ๋ชจ๋‘ ์‚ฌ๋žŒ ์† ์ ์ด ํ‘œ๋ฉด \epsilon ์•ˆ์— ์žˆ์„ ๋•Œ๋งŒ ์ผœ์ง„๋‹ค. ์ ‘๊ทผยท๋†“๋Š” ๊ตฌ๊ฐ„์—์„œ๋Š” ์ ‘์ด‰ ์†์‹ค์ด ์ •๋ณด๊ฐ€ ์—†๊ณ  ์žก์Œ๋งŒ ๋”ํ•œ๋‹ค๋Š” ์ด์œ ๋‹ค. \mathcal{L}_{\mathrm{surf}}๋Š” ํ‘œ๋ฉด์— ๋Œ€ํ•ด ๋Œ€์นญ์ด๋ผ ๊ด€ํ†ต์„ ๋ง‰์ง€ ๋ชปํ•˜๋ฏ€๋กœ ๊ด€ํ†ต์€ \mathcal{L}_{\mathrm{coll}}์ด ๋งก๋Š”๋‹ค. ์ •๊ทœํ™” ํ•ญ์€ \mathcal{L}_{\mathrm{coll}}, ์—ฐ์† ์˜ˆ์ธก ์‚ฌ์ด์˜ \mathcal{L}_{\mathrm{temp}}, ๊ทธ๋ฆฌ๊ณ  supervised ๋„คํŠธ์›Œํฌ์˜ ๊ฐ™์€ ํ”„๋ ˆ์ž„ ์ถœ๋ ฅ์— ๋ฌถ๋Š” \mathcal{L}_{\mathrm{anc}}์ด๋‹ค. ์ „์ฒด ๋ชฉ์ ์€ \mathcal{L}_{\mathrm{ft}}=\sum_\ell\lambda_\ell\mathcal{L}_\ell, \ell\in\{\mathrm{surf},\mathrm{part},\mathrm{dir},\mathrm{coll},\mathrm{anc},\mathrm{temp}\}์ด๊ณ  ๊ฐ€์ค‘์น˜๋Š” (5,10,1,10,1,1)์ด๋‹ค(ยงV-A). ์ด ๋‹จ๊ณ„๋Š” ๋ ˆ์ด๋ธ” ์—†์ด ์ง„ํ–‰๋œ๋‹ค.

๋ณธ๋ฌธ๊ณผ ๋ถ€๋ก์ด ์–ด๊ธ‹๋‚˜๋Š” ๊ตฌํ˜„ ๊ฐ’

๋…ผ๋ฌธ์€ ยงV-A โ€œImplementationโ€๊ณผ ๋ถ€๋ก Table S2์— ๊ฐ™์€ ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ๋‘ ๋ฒˆ ์ ๋Š”๋ฐ, ๋ช‡ ๊ตฐ๋ฐ๊ฐ€ ๋‹ค๋ฅด๋‹ค. HTML๊ณผ PDF์˜ ๊ฐ’์€ ๊ฐ™๋‹ค.

ํ•ญ๋ชฉ ๋ณธ๋ฌธ ยงV-A / ยงIII ๋ถ€๋ก Table S2 / ยงS2
๋กœ๋ด‡-๋ฌผ์ฒด ์ถฉ๋Œ margin 5 mm 2 mm
fine-tuning epoch 24 25
decoder ๋‘ MLP decoder ๋ชจ๋‘ 512 unit hidden layer 3๊ฐœ Allegro 8์ธตยทํญ 1024, Inspire 4์ธตยทํญ 512
Delaunay tetrahedralization ๋งค ํ”„๋ ˆ์ž„ ๋‹ค์‹œ ๊ณ„์‚ฐ(ยงIII-A) source ๋…ธ๋“œ์—์„œ ํ•œ ๋ฒˆ ๋งŒ๋“ค์–ด ์žฌ์‚ฌ์šฉ(ยงS2)
autoregressive ์กฐ๊ฑดํ™” ์ง์ „ ํ”„๋ ˆ์ž„ ์˜ˆ์ธก ํ•˜๋‚˜ โ€œWindow length 9 frames, autoregressiveโ€
keypoint ์ž…๋ ฅ ์† ํ•˜๋‚˜์˜ N_h=21๊ฐœ, ๊ถค์ ์€ ์†๋งˆ๋‹ค โ€œKeypoint tokens 2\times21=42โ€
์ตœ์ ํ™” frame step (๋ณธ๋ฌธ ๋ฏธ๊ธฐ์žฌ) ๋ฐ์ดํ„ฐ๋Š” 60 Hz๋กœ subsample(ยงS1) \mathrm{d}t=1/30 s
fine-tuning ๊ฐ€์ค‘์น˜ ์ถœ์ฒ˜ ยงV-A์— ์ˆ˜์น˜ ๋ช…์‹œ โ€œloss weights: see Table IIโ€(Table II์—๋Š” ๊ฐ€์ค‘์น˜ ์ˆ˜์น˜๊ฐ€ ์—†๋‹ค)

๋Œ€๋ถ€๋ถ„์€ ์žฌํ˜„์ž์—๊ฒŒ ์–ด๋А ๊ฐ’์„ ๋ฏฟ์–ด์•ผ ํ• ์ง€ ๋ชจ๋ฅด๊ฒŒ ๋งŒ๋“œ๋Š” ์ˆ˜์ค€์˜ ํ‘œ๊ธฐ ๋ถˆ์ผ์น˜๋‹ค. ๊ทธ์ค‘ tetrahedralization๊ณผ keypoint ์ž…๋ ฅ ๋‘ ์ค„์€ ๋ฐฉ๋ฒ•์˜ ์˜๋ฏธ๋ฅผ ๋ฐ”๊ฟ€ ์ˆ˜ ์žˆ๋‹ค. ๋งค ํ”„๋ ˆ์ž„ ์žฌ๊ณ„์‚ฐ์ด๋ฉด ์ด์›ƒ ๊ด€๊ณ„๊ฐ€ ์ ‘์ด‰ ๋ณ€ํ™”์— ๋”ฐ๋ผ ๋ฐ”๋€Œ๊ณ , ํ•œ ๋ฒˆ๋งŒ ๋งŒ๋“ค๋ฉด ์ฒซ ํ”„๋ ˆ์ž„์˜ ์ด์›ƒ์ด ์‹œํ€€์Šค ๋‚ด๋‚ด ๊ณ ์ •๋œ๋‹ค. 42๊ฐœ token์ด๋ฉด retargeter๊ฐ€ ์–‘์†์„ ํ•จ๊ป˜ ๋ณธ๋‹ค๋Š” ๋œป์ผ ์ˆ˜ ์žˆ๋Š”๋ฐ, ๋ณธ๋ฌธ์€ โ€œone trajectory per handโ€๋ผ๊ณ  ํ•œ๋‹ค. ์ฝ”๋“œ๊ฐ€ ๊ณต๊ฐœ๋˜๋ฉด ๊ฐ€์žฅ ๋จผ์ € ํ™•์ธํ•  ๋‘ ํ•ญ๋ชฉ์ด๋‹ค.

Dynamic retargeting (ยงIV)

kinematic reference \hat\tau๋Š” residual RL์˜ base action์ด ๋œ๋‹ค. ์ •์ฑ…์€ ์‹œ๋ฎฌ๋ ˆ์ดํ„ฐ ์ƒํƒœ s_t, reference์˜ ํ•œ ์Šคํ… ์•ž ๋ชฉํ‘œ \hat\tau_{t+1}, ์‹œ์—ฐ ๋ฌผ์ฒด ์ƒํƒœ x^o_{t+1}์„ ๋ณด๊ณ  residual์„ ๋‚ธ๋‹ค.

\delta a_t=\pi(s_t,\hat\tau_{t+1},x^o_{t+1}),\qquad a_t=\bar a_t+\delta a_t

์†๊ฐ€๋ฝ์€ base action์— residual์„ ๋”ํ•ด PD ์œ„์น˜ ๋ชฉํ‘œ๋กœ, ์†๋ชฉ์€ PID๊ฐ€ ์œ„์น˜ยท๋ฐฉํ–ฅ ์˜ค์ฐจ๋ฅผ ํž˜ยทํ† ํฌ๋กœ ๋ฐ”๊พผ ๋’ค residual์„ ๋”ํ•œ๋‹ค. episode๋Š” reference์˜ ์ฒ˜์Œ 98% ์ค‘ ๋ฌด์ž‘์œ„ ํ”„๋ ˆ์ž„์—์„œ ์‹œ์ž‘ํ•˜๋Š” reference-state initialization(RSI)์„ ์“ฐ๊ณ , ๊ธฐ์กด ์—ฐ๊ตฌ(DexMachina, SPIDER)์™€ ๋‹ฌ๋ฆฌ ์‹œ๋ฎฌ๋ ˆ์ดํ„ฐ์—์„œ ์ ‘์ด‰์„ ๋ฏธ๋ฆฌ ํ’€์–ด ๋‘๋Š” ์ „์ฒ˜๋ฆฌ๋ฅผ ํ•˜์ง€ ์•Š๋Š”๋‹ค(ยงIV, ยงS3).

์‹คํ—˜๋งˆ๋‹ค RL backend๊ฐ€ ๋‹ค๋ฅด๋‹ค๋Š” ์ ์ด ๊ฒฐ๊ณผ ํ•ด์„์—์„œ ๊ฐ€์žฅ ์ค‘์š”ํ•˜๋‹ค.

์‹คํ—˜ RL backend ๋ชฉ์ 
Table I(OakInk-v2 ablation) ManipTrans residual RL reference๋งŒ ๋ฐ”๊ฟ” ๋น„๊ต
Table III(ARCTIC ์ผ๋ฐ˜ํ™”) DexMachina RL reference๋งŒ ๋ฐ”๊ฟ” ๋น„๊ต
Table IV(SOTA), Table V(์‹ค๊ธฐ) ํ†ตํ•ฉ residual RL: DeepMimic tracking + ManipTrans ๋ณด์ƒ + DexMachina object curriculum ์™„์„ฑ๋œ ์‹œ์Šคํ…œ๋ผ๋ฆฌ ๋น„๊ต

ํ†ตํ•ฉ RL์— DexMachina curriculum์„ ๋„ฃ์€ ์ด์œ ๋Š” โ€œresidual RL needs the object curriculum on ARCTICโ€์ด๋ฉฐ, ํ•˜๋‚˜์˜ ํ”„๋กœํ† ์ฝœ๋กœ ๋ณด๊ณ ํ•˜๋ ค๊ณ  OakInk-v2์—๋„ ์ผฐ๋‹ค(ยงIV). RL ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ๋Š” Table S3์— ์žˆ๋‹ค: 4,096 env, PPO, sim step 1/60 s, episode 1,200 step, \gamma=0.99, GAE \lambda=0.95, clip 0.2, rollout 32, minibatch 1,024, epoch 5, lr 5e-4, H100 ํ•œ ์žฅ. ๋„๋ฉ”์ธ ๋žœ๋คํ™”๋Š” ๋ฌผ์ฒด ๋งˆ์ฐฐ [0.5,3]๋ฐฐ, ์งˆ๋Ÿ‰ ยฑ20%๋‹ค.

์ง๊ด€: ์ ‘์ด‰์„ reference์— ๋„ฃ์œผ๋ฉด ์™œ RL์ด ์‰ฌ์›Œ์ง€๋‚˜

์•„๋ž˜๋Š” ํ•„์ž์˜ ํ•ด์„์ด๋‹ค. residual RL์€ base action ์ฃผ๋ณ€์„ ํƒ์ƒ‰ํ•œ๋‹ค. base action์ด ์†๋์„ ๋ฌผ์ฒด์—์„œ 3~5 cm ๋–จ์–ด์ง„ ๊ณณ์— ๋‘”๋‹ค๋ฉด(hand-only reference์˜ TPE๊ฐ€ 49~78 mm๋‹ค), residual ์ •์ฑ…์€ ์–ด๋””๋ฅผ ์ฅ์–ด์•ผ ํ•˜๋Š”์ง€๋ถ€ํ„ฐ ํƒ์ƒ‰ํ•ด์•ผ ํ•œ๋‹ค. ๋ฌผ์ฒด๋ฅผ ์ฅ๊ธฐ ์ „์—๋Š” ๋ฌผ์ฒด ์ถ”์ข… ๋ณด์ƒ์ด ๊ฑฐ์˜ ์›€์ง์ด์ง€ ์•Š์œผ๋ฏ€๋กœ ์ด ํƒ์ƒ‰์€ sparse reward ๋ฌธ์ œ์— ๊ฐ€๊น๋‹ค. ๋ฐ˜๋Œ€๋กœ reference๊ฐ€ ์†๋์„ ์ด๋ฏธ ์ ‘์ด‰์  ๊ทผ๋ฐฉ(TPE 3 mm๋Œ€)์— ๋‘”๋‹ค๋ฉด residual์ด ํ•  ์ผ์€ ์–ผ๋งˆ๋‚˜ ์„ธ๊ฒŒ, ์–ด๋А ์ˆœ๊ฐ„์— ๋ˆ„๋ฅผ์ง€ ๊ฐ™์€ dynamics ๋ณด์ •์ด ๋œ๋‹ค. ์ด์ชฝ์€ ๋ณด์ƒ์ด ์ด˜์ด˜ํ•œ ๊ตญ์†Œ ๋ฌธ์ œ๋‹ค.

๋…ผ๋ฌธ์ด ๋‚ด์„ธ์šฐ๋Š” ๊ฐ€์„ค๋„ ์ด๊ฒƒ์ด๋‹ค(ยงI: โ€œWe hypothesize that object-aware references let it focus on dynamics rather than inferring contact locationsโ€). ์ €์ž๋Š” โ€œhypothesizeโ€๋ผ๊ณ  ์“ฐ๊ณ , ๊ทผ๊ฑฐ๋Š” Table I์˜ reference๋ณ„ RL ์„ฑ๊ณต๋ฅ  ์ƒ๊ด€์ด๋‹ค. ํƒ์ƒ‰ ๊ณก์„ ์ด๋‚˜ ํ•™์Šต ์†๋„ ๋น„๊ต ๊ฐ™์€ ์ง์ ‘ ์ฆ๊ฑฐ๋Š” ์—†๋‹ค.

์ด ์ง๊ด€์ด ์–ด๋””์„œ ๊นจ์ง€๋Š”์ง€๋„ ๋…ผ๋ฌธ์ด ์Šค์Šค๋กœ ๋ณด์—ฌ ์ค€๋‹ค. reference๋Š” kinematic์ด๋ผ ์šด๋™๋Ÿ‰์„ ๋ชจ๋ฅธ๋‹ค. ํ”๋“ค๊ธฐ์ฒ˜๋Ÿผ ๋™์ ์ธ ๊ณผ์ œ์—์„œ๋Š” ์ ‘์ด‰ ์œ„์น˜๊ฐ€ ๋งž์•„๋„ ์ฅ๋Š” ํž˜๊ณผ ๊ฐ€์†์„ RL์ด ํ˜ผ์ž ๋งŒ๋“ค์–ด์•ผ ํ•˜๊ณ , ๊ทธ๋•Œ ๊ทธ๋ฆฝ์ด ํ’€๋ฆฐ๋‹ค(Fig. 4 ๋งˆ์ง€๋ง‰ ํ–‰, ยงV-E). ์ฆ‰ object-aware reference๋Š” โ€œ์–ด๋””โ€๋ฅผ ํ•ด๊ฒฐํ•˜์ง€๋งŒ โ€œ์–ด๋–ป๊ฒŒโ€๋Š” ์—ฌ์ „ํžˆ RL ๋ชซ์ด๋‹ค.

์‹คํ—˜ (ยงV)

์…‹์—… (ยงV-A)

ํ•ญ๋ชฉ OakInk-v2 ARCTIC
์„ฑ๊ฒฉ ์–‘์†ยทํ•œ์† ๋ฌผ์ฒด ์กฐ์ž‘ ์–‘์† articulated object ์กฐ์ž‘
ํ•™์Šต ๋ฐ์ดํ„ฐ ManipTrans ์„ธํŠธ 77 ์‹œํ€€์Šค, ๋ฌผ์ฒด 42๊ฐœ(์–‘์† 39ยท์˜ค๋ฅธ์† 27ยท์™ผ์† 11) 30 ์‹œํ€€์Šค(๋ฌผ์ฒด 5 ร— ํด๋ฆฝ 6)
ํ‰๊ฐ€ ์–‘์† ์„ธํŠธ(SOTA, Table IV), ์˜ค๋ฅธ์† ์„ธํŠธ(ablation, Table IยทII) DexMachinaยทSPIDER๊ฐ€ ์“ด 7ํด๋ฆฝ ๋ถ€๋ถ„์ง‘ํ•ฉ(Table IIIยทIV)
์ „์ฒ˜๋ฆฌ 120 โ†’ 60 Hz subsample, DexManipNet ๋ถ„ํ• ์„ ๋”ฐ๋ฅธ manipulation primitive ๋ถ€๋ก Table S4ยทS5์— ํด๋ฆฝ ๋ชฉ๋ก

๋กœ๋ด‡ ์†์€ Allegro(์†๊ฐ€๋ฝ 4, ์ œ์–ด DoF 16)์™€ Inspire(์†๊ฐ€๋ฝ 5, ์ œ์–ด DoF 6 + mimic ๊ด€์ ˆ 6)๋‹ค. ์ง€ํ‘œ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

์ง€ํ‘œ ์ •์˜ ์ถœ์ฒ˜
TPE (mm) ๋กœ๋ด‡ ์†๋๊ณผ ์‚ฌ๋žŒ ์†๋์˜ ํ‰๊ท  ์œ ํด๋ฆฌ๋“œ ๊ฑฐ๋ฆฌ ์ด ๋…ผ๋ฌธ
Contact completeness, CC (%) ์‚ฌ๋žŒ ์†๋์ด ํ‘œ๋ฉด 10 mm ์•ˆ์ธ ์ ‘์ด‰์ด ํ•˜๋‚˜๋ผ๋„ ์žˆ๋Š” ํ”„๋ ˆ์ž„ ์ค‘, ์ ‘์ด‰ํ•œ ๋ชจ๋“  ๋กœ๋ด‡ ์†๋์ด ํ‘œ๋ฉด 10 mm ์•ˆ์— ์žˆ๋Š” ํ”„๋ ˆ์ž„ ๋น„์œจ ์ด ๋…ผ๋ฌธ
\mathrm{SR}_{\mathrm{MT}} ๋ฌผ์ฒด ํšŒ์ „ 30ยฐ, ๋ฌผ์ฒด ์ด๋™ 3 cm, ๊ด€์ ˆ ์œ„์น˜ 8 cm, ์†๋ 6 cm ์ดํ•˜ ManipTrans
\mathrm{SR}_{\mathrm{obj}} ๋ฌผ์ฒด๋งŒ, ํšŒ์ „ 28.6ยฐ(0.5 rad)ยท์ด๋™ 10 cm SPIDER
ADD-AUC ๋‹ฌ์„ฑ ์ž์„ธ์™€ ์‹œ์—ฐ ์ž์„ธ์˜ ๋ฌผ์ฒด ํ‘œ๋ฉด ์  ๊ฑฐ๋ฆฌ ๊ธฐ๋ฐ˜ ์ถ”์ข… ์ ์ˆ˜ DexMachina

๋…ผ๋ฌธ์€ TPE์™€ CC๊ฐ€ โ€œdiagnostic rather than complete predictors of downstream successโ€๋ผ๊ณ  ์Šค์Šค๋กœ ์„ ์„ ๊ธ‹๋Š”๋‹ค. ํ‰๊ฐ€๋Š” Table I~III์ด 3 seed ํ‰๊ท , Table IV๊ฐ€ 5 seed ํ‰๊ท ์ด๊ณ  ์ •์ฑ…๋งˆ๋‹ค 256 rollout์ด๋‹ค. Table IIIยทIV์˜ ๋ฒ ์ด์Šค๋ผ์ธ ๊ฐ’์€ ์› ๋…ผ๋ฌธ์—์„œ ๊ฐ€์ ธ์˜จ ๊ฒƒ์ด๋‹ค. ๋ชจ๋“  ํ‘œ์— ํ‘œ์ค€ํŽธ์ฐจ๋‚˜ ์‹ ๋ขฐ๊ตฌ๊ฐ„์€ ์—†๋‹ค.

Object awareness์˜ ํšจ๊ณผ (ยงV-B, Table I)

Allegro ์˜ค๋ฅธ์† OakInk-v2 27๊ณผ์ œ, ๋ชจ๋‘ ๊ฐ™์€ ManipTrans residual RL์ด๋‹ค. ๋ฒ ์ด์Šค๋ผ์ธ ์ˆ˜์น˜๋Š” ์ €์ž๋“ค์˜ ์žฌํ˜„๊ฐ’์ด๋‹ค.

Kinematic reference CC (%) โ†‘ TPE (mm) โ†“ \mathrm{SR}_{\mathrm{MT}} (%) โ†‘
Hand-only reference
AnyTeleop-Vector 3.5 59.5 18.3
AnyTeleop-Position 0.5 77.9 11.1
AnyTeleop-DexPilot 0.3 53.8 0.0
GeoRT 1.0 58.6 0.8
ManipTrans (imitator) 5.8 49.2 20.6
Object-aware reference (DexOAK)
IG optimizer 19.6 3.50 43.2
+ supervised training 19.2 3.50 43.6
+ Object-Aware FT (Full) 21.9 3.21 48.2

๋ณธ๋ฌธ์€ 48.2%๊ฐ€ ManipTrans ๊ฒฐ๊ณผ์˜ 2.3๋ฐฐ์ด๊ณ (48.2/20.6 = 2.34, ํ•„์ž ๊ฒ€์‚ฐ ์ผ์น˜) fine-tuning์ด ์ตœ์ ํ™” ๋Œ€๋น„ +5.0 %p๋ฅผ ๋”ํ•œ๋‹ค๊ณ  ์“ด๋‹ค. ์ด ํ‘œ๋ฅผ ๋‹จ๊ณ„๋ณ„๋กœ ๋ถ„ํ•ดํ•˜๋ฉด ์ด ๋…ผ๋ฌธ์˜ ๊ธฐ์—ฌ ๊ตฌ์กฐ๊ฐ€ ๋“œ๋Ÿฌ๋‚œ๋‹ค(ํ•„์ž ๊ณ„์‚ฐ).

๋‹จ๊ณ„ \mathrm{SR}_{\mathrm{MT}} ์ฆ๋ถ„ ์ „์ฒด +27.6 %p ์ค‘ ๋น„์œจ
hand-only ์ตœ๊ณ (ManipTrans 20.6) โ†’ IG optimizer(43.2) +22.6 %p 82%
IG optimizer โ†’ + supervised(43.6) +0.4 %p 1%
+ supervised โ†’ + Object-Aware FT(48.2) +4.6 %p 17%

์ฆ‰ ์„ฑ๊ณต๋ฅ ์„ ๋‘ ๋ฐฐ๋กœ ๋งŒ๋“  ๊ฒƒ์€ ๋ฌผ์ฒด anchor๋ฅผ ๋„ฃ์€ interaction-graph ์ตœ์ ํ™”์ด๊ณ , ์ด๋Š” OmniRetarget ๋ชฉ์ ํ•จ์ˆ˜๋ฅผ ์†๊ฐ€๋ฝ ๋‹จ์œ„๋กœ ํ™•์žฅํ•œ ๋ถ€๋ถ„์ด๋‹ค. ํ•™์Šต๋œ retargeter์™€ ์ ‘์ด‰ fine-tuning์€ ๊ทธ ์œ„์— ์•ฝ 5 %p๋ฅผ ๋”ํ•œ๋‹ค. 3 seed ํ‰๊ท ์— ๋ถ„์‚ฐ์ด ๋ณด๊ณ ๋˜์ง€ ์•Š์•„ ์ด 5 %p๊ฐ€ seed ์žก์Œ๋ณด๋‹ค ํฐ์ง€๋Š” ํ‘œ๋งŒ์œผ๋กœ ํŒ๋‹จํ•  ์ˆ˜ ์—†๋‹ค.

ํ‘œ์—์„œ ํ•˜๋‚˜ ๋” ๋ˆˆ์— ๋„๋Š” ๊ฒƒ์€ hand-only ๋ฐฉ๋ฒ•์˜ TPE๋‹ค. TPE๋Š” ๋กœ๋ด‡ ์†๋๊ณผ ์‚ฌ๋žŒ ์†๋ ์‚ฌ์ด ๊ฑฐ๋ฆฌ์ธ๋ฐ, ์†๋ ์œ„์น˜๋ฅผ ์ง์ ‘ ๋งž์ถ”๋Š” AnyTeleop-Position์ด 77.9 mm๋กœ ๊ฐ€์žฅ ๋‚˜์˜๋‹ค. ๋…ผ๋ฌธ์€ ์ด ์—ญ์ „์„ ์„ค๋ช…ํ•˜์ง€ ์•Š๋Š”๋‹ค. AnyTeleop๋ฅ˜๋Š” ์†๋ชฉ์„ ์‚ฌ๋žŒ ์†๋ชฉ์— ๊ณ ์ •ํ•˜๊ณ  ์†๊ฐ€๋ฝ ๊ด€์ ˆ๋งŒ ์ตœ์ ํ™”ํ•˜๊ฑฐ๋‚˜ ์† ํฌ๊ธฐ๋ฅผ ์Šค์ผ€์ผ๋งํ•˜๋ฏ€๋กœ, ํฌ๊ธฐ๊ฐ€ ๋‹ค๋ฅธ ๋กœ๋ด‡ ์†์—์„œ๋Š” ์†๋ชฉ ๊ธฐ์ค€์˜ ์ƒ๋Œ€ ๋ฐฐ์น˜๊ฐ€ ๋งž์•„๋„ ์ ˆ๋Œ€ ์†๋ ์œ„์น˜๊ฐ€ cm ๋‹จ์œ„๋กœ ์–ด๊ธ‹๋‚  ์ˆ˜ ์žˆ๋‹ค(ํ•„์ž์˜ ํ•ด์„). ๋ฐ˜๋Œ€๋กœ DexOAK๋Š” ์†๋ชฉ ์ž์„ธ๊นŒ์ง€ ์ตœ์ ํ™” ๋ณ€์ˆ˜์ด๊ณ  ์‹ค์ œ ์Šค์ผ€์ผ์„ ์œ ์ง€ํ•˜๋ฏ€๋กœ TPE๊ฐ€ ์ •์˜์ƒ ์œ ๋ฆฌํ•˜๋‹ค. TPE์™€ CC๋Š” ๊ฐ๊ฐ \mathcal{L}_{\mathrm{IG}}ยท\mathcal{L}_{\mathrm{part}}, \mathcal{L}_{\mathrm{surf}}์™€ ๊ฑฐ์˜ ๊ฐ™์€ ์–‘์„ ์žฌ๋ฏ€๋กœ, ์ด ๋‘ ์—ด์—์„œ์˜ ์••์Šน์€ ์ƒ๋‹น ๋ถ€๋ถ„ ๋ชฉ์ ํ•จ์ˆ˜๊ฐ€ ๊ณง ์ง€ํ‘œ์ธ ๊ตฌ์กฐ์—์„œ ๋‚˜์˜จ๋‹ค. ๋…๋ฆฝ์ ์ธ ๊ทผ๊ฑฐ๋Š” RL ์„ฑ๊ณต๋ฅ  ์—ด์ด๋‹ค.

๊ตฌ์„ฑ์š”์†Œ ablation (ยงV-C, Table II)

๊ฐ™์€ Allegro ์˜ค๋ฅธ์† ๊ณผ์ œ, kinematic ์ง€ํ‘œ๋งŒ(RL ์—†์Œ). ๊ฐ ํ–‰์€ Full ๋ชจ๋ธ์—์„œ ํ•˜๋‚˜๋งŒ ๋ฐ”๊พผ๋‹ค.

๋ณ€ํ˜• CC (%) โ†‘ TPE (mm) โ†“
Interaction-graph Laplacian optimizer
without object anchors 2.8 41.5
without wrist node 19.6 3.51
Architecture and training
without object cloud 20.0 3.83
without autoregression 16.1 9.29
single-sequence training 20.9 3.53
without supervised training 10.3 36.8
Object-aware losses (leave-one-out)
without surface-contact 13.2 6.84
without object-part 19.2 7.71
without contact-direction 18.8 7.25
Loss-weight configurations
uniform (1,1,1,1,1,1) 19.2 5.03
contact-heavy (50,100,20,10,1,1) 20.8 4.11
regularization-heavy (5,10,1,100,50,10) 19.3 3.85
Full model (5,10,1,10,1,1) 21.9 3.21

์ฝ์„๊ฑฐ๋ฆฌ๋Š” ๋„ค ๊ฐ€์ง€๋‹ค.

  1. ๋ฌผ์ฒด anchor๊ฐ€ ์ „๋ถ€๋‹ค. anchor๋ฅผ ๋นผ๋ฉด CC 2.8%, TPE 41.5 mm๋กœ hand-only ๋ฒ ์ด์Šค๋ผ์ธ ์ˆ˜์ค€์œผ๋กœ ๋–จ์–ด์ง„๋‹ค. Table I ๋ถ„ํ•ด์™€ ๊ฐ™์€ ๊ฒฐ๋ก ์ด๋‹ค.
  2. supervised ์ดˆ๊ธฐํ™” ์—†์ด๋Š” fine-tuning์ด ๋ฌด๋„ˆ์ง„๋‹ค(10.3%, 36.8 mm). ์ ‘์ด‰ ์†์‹ค๋งŒ์œผ๋กœ๋Š” ์†์„ ์–ด๋””์— ๋‘˜์ง€ ์ •ํ•˜์ง€ ๋ชปํ•œ๋‹ค๋Š” ๋œป์ด๊ณ , seed ์ตœ์ ํ™”๊ฐ€ ๋‹จ์ˆœํ•œ โ€œ๊ฐ€์†์šฉ ๋Œ€์ฒด๋ฌผโ€์ด ์•„๋‹ˆ๋ผ ํ•™์Šต ๋ฌธ์ œ๋ฅผ ์„ฑ๋ฆฝ์‹œํ‚ค๋Š” ์กฐ๊ฑด์ž„์„ ๋ณด์—ฌ ์ค€๋‹ค.
  3. ๋„คํŠธ์›Œํฌ ์ž…๋ ฅ์—์„œ ๋ฌผ์ฒด ์ ๊ตฐ์„ ๋นผ๋„ ์†์‹ค์ด ์ž‘๋‹ค(20.0%, 3.83 mm). ๋ณธ๋ฌธ์€ โ€œobject information still enters through the lossesโ€๋ผ๊ณ  ์„ค๋ช…ํ•œ๋‹ค. ์ด๊ฒƒ์€ ํฅ๋ฏธ๋กœ์šด ์ž๊ธฐ ์ œํ•œ์ด๋‹ค. ์ œ๋ชฉ์˜ โ€œobject-awareโ€๊ฐ€ ๋„คํŠธ์›Œํฌ์˜ ์ž…๋ ฅ๋ณด๋‹ค ํ•™์Šต ์‹ ํ˜ธ์—์„œ ์ฃผ๋กœ ์˜จ๋‹ค๋Š” ๋œป์ด๊ณ , ๋ฌผ์ฒด ํ˜•์ƒ์„ ๋ณด์ง€ ์•Š๋Š” ๋„คํŠธ์›Œํฌ๋„ ํ•™์Šต ๋ถ„ํฌ ์•ˆ์—์„œ๋Š” ๊ฑฐ์˜ ๊ฐ™์€ reference๋ฅผ ๋‚ธ๋‹ค. ๋‹ค๋งŒ ARCTIC ์ „์ด์ฒ˜๋Ÿผ ์ฒ˜์Œ ๋ณด๋Š” ๋ฌผ์ฒด์—์„œ ๋ฌผ์ฒด ์ž…๋ ฅ์˜ ๊ธฐ์—ฌ๊ฐ€ ์ปค์ง€๋Š”์ง€๋Š” ์ด ํ‘œ๋กœ ์•Œ ์ˆ˜ ์—†๋‹ค(๊ทธ ์กฐ๊ฑด์˜ ablation์€ ์—†๋‹ค).
  4. ๊ฐ€์ค‘์น˜ ๋ฐฐ์น˜์— ๋‘”๊ฐํ•˜๋‹ค. ์„ธ ๋Œ€์•ˆ ๋ชจ๋‘ hand-only๋ณด๋‹ค ํ›จ์”ฌ ๋‚ซ๋‹ค. ๋‹ค๋งŒ contact-heavy๊ฐ€ ์ ‘์ด‰ ๊ฐ€์ค‘์น˜๋ฅผ 10๋ฐฐ ํ‚ค์› ๋Š”๋ฐ๋„ CC๊ฐ€ Full๋ณด๋‹ค ๋‚ฎ๋‹ค(20.8 < 21.9)๋Š” ๋น„๋‹จ์กฐ์„ฑ์€ ์–ธ๊ธ‰๋˜์ง€ ์•Š๋Š”๋‹ค.

ํ•ด์„์ƒ ํ™•์ธ์ด ํ•„์š”ํ•œ ๊ณณ๋„ ์žˆ๋‹ค. ๋งจ ์œ„ ๋‘ ํ–‰์€ โ€œoptimizerโ€ ๋ธ”๋ก์— ์žˆ์ง€๋งŒ ํ‘œ ์บก์…˜์€ โ€œFull model์—์„œ ํ•˜๋‚˜์”ฉ ๋ฐ”๊พผ๋‹คโ€๊ณ  ํ•œ๋‹ค. โ€œwithout wrist nodeโ€(19.6, 3.51)๋Š” Table I์˜ IG optimizer ํ–‰(19.6, 3.50)๊ณผ ๊ฑฐ์˜ ๊ฐ™์•„์„œ, ์ด ๋‘ ํ–‰์ด ์ตœ์ ํ™” ์ถœ๋ ฅ์—์„œ ์žฐ ๊ฐ’์ธ์ง€ ๊ทธ seed๋กœ ํ•™์Šตํ•œ Full ํŒŒ์ดํ”„๋ผ์ธ์—์„œ ์žฐ ๊ฐ’์ธ์ง€๊ฐ€ ๋ชจํ˜ธํ•˜๋‹ค. ๊ทธ๋ฆฌ๊ณ  ์ด ํ‘œ์—๋Š” RL ์„ฑ๊ณต๋ฅ ์ด ์—†์œผ๋ฏ€๋กœ, ๊ฐ ๊ตฌ์„ฑ์š”์†Œ๊ฐ€ downstream ์„ฑ๊ณต์— ์–ผ๋งˆ๋‚˜ ๊ธฐ์—ฌํ•˜๋Š”์ง€๋Š” ์ธก์ •๋˜์ง€ ์•Š์•˜๋‹ค.

์ผ๋ฐ˜ํ™”: ๋‹ค์‹œ ํ’€์ง€ ์•Š๊ณ  ์˜ฎ๊ธฐ๊ธฐ (ยงV-D, Table III)

Allegro, ARCTIC, ๋ชจ๋‘ DexMachina RL. ์†๋„๋Š” โ€œํ‰๊ฐ€ ํ”„๋ ˆ์ž„ ์ˆ˜ รท target ๋ฐ์ดํ„ฐ์˜ kinematic ๋‹จ๊ณ„ ์ด wall-clockโ€์ด๋ฉฐ, ์ตœ์ ํ™”ยทํ•™์Šตยทfine-tuning์„ ํฌํ•จํ•˜๊ณ  RL์€ ์ œ์™ธํ•œ๋‹ค.

๋ฐฉ๋ฒ• Speed (Hz) โ†‘ ADD-AUC (%) โ†‘
In-domain on ARCTIC
DexMachina 13.4 83.4
Ours: IG optimizer 15.5 88.9
Ours: DexOAK 1.4 91.3
Generalization OakInk-v2 โ†’ ARCTIC
Zero-shot 108.9 87.9
+ Object-Aware FT on ARCTIC 57.2 90.0

OakInk-v2 ๋ชจ๋ธ์ด ARCTIC์— zero-shot์œผ๋กœ 87.9๋ฅผ ๋‚ด DexMachina(83.4)๋ฅผ ๋„˜๊ณ , in-domain ์ตœ์ ํ™”๋ณด๋‹ค 7๋ฐฐ ๋น ๋ฅด๋‹ค(108.9/15.5 = 7.03, ํ•„์ž ๊ฒ€์‚ฐ ์ผ์น˜). ARCTIC์—์„œ fine-tuning๋งŒ ํ•˜๋ฉด 90.0์œผ๋กœ in-domain DexOAK(91.3)์— ๊ทผ์ ‘ํ•œ๋‹ค.

์ด ํ‘œ๋Š” ์ •์งํ•˜๊ฒŒ ์ฝ์œผ๋ฉด ๋‘ ๊ฐ€์ง€๋ฅผ ๋” ๋งํ•œ๋‹ค.

  • In-domain DexOAK๋Š” ๊ฐ€์žฅ ๋А๋ฆฌ๋‹ค(1.4 Hz). 30 ์‹œํ€€์Šค์งœ๋ฆฌ ARCTIC์—์„œ seed ์ตœ์ ํ™” + 600 epoch supervised + fine-tuning์„ ๋ชจ๋‘ target ๋ฐ์ดํ„ฐ ๋น„์šฉ์œผ๋กœ ์…Œ๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค. amortization์€ ๊ฐ™์€ retargeter๋ฅผ ์—ฌ๋Ÿฌ ๋ฒˆ ์žฌ์‚ฌ์šฉํ•  ๋•Œ๋งŒ ์ด๋“์ด๊ณ , ๋ฐ์ดํ„ฐ๊ฐ€ ์ž‘์œผ๋ฉด ์ตœ์ ํ™”(15.5 Hz)๋ฅผ ๊ทธ๋ƒฅ ์“ฐ๋Š” ํŽธ์ด ๋น ๋ฅด๋‹ค.
  • Zero-shot์˜ 108.9 Hz๋Š” OakInk-v2 ์ชฝ ๋น„์šฉ์„ ํฌํ•จํ•˜์ง€ ์•Š๋Š”๋‹ค. ์ •์˜์ƒ โ€œtarget-dataโ€ ๋น„์šฉ๋งŒ ์„ธ๋ฏ€๋กœ ๋งž๋Š” ๊ณ„์‚ฐ์ด์ง€๋งŒ, ๋…์ž๊ฐ€ โ€œ7๋ฐฐ ๋น ๋ฅธ ๋ฐฉ๋ฒ•โ€์œผ๋กœ ์ฝ์œผ๋ฉด source ์ชฝ ์ตœ์ ํ™”ยทํ•™์Šต ๋น„์šฉ์ด ์‚ฌ๋ผ์ง„๋‹ค. ์ •ํ™•ํžˆ๋Š” ์ด๋ฏธ ํ•™์Šต๋œ retargeter์˜ ํ•œ๊ณ„ ๋น„์šฉ์ด๋‹ค.

ADD-AUC ๋ถ„ํ•ด๋„ Table I๊ณผ ๊ฐ™์€ ๋ชจ์–‘์ด๋‹ค(ํ•„์ž ๊ณ„์‚ฐ). DexMachina 83.4 ๋Œ€๋น„ in-domain DexOAK์˜ +7.9 ์ค‘ IG optimizer๊ฐ€ +5.5(70%), ํ•™์Šต + fine-tuning์ด +2.4(30%)๋‹ค.

SOTA ๋น„๊ต (ยงV-E, Table IV, Fig. 4)

๊ฐ ๋ฐฉ๋ฒ•์€ ์ž๊ธฐ dynamic refinement๋ฅผ ์“ฐ๊ณ , DexOAK+RL์€ ยงIV์˜ ํ†ตํ•ฉ residual RL์„ ์“ด๋‹ค. ๋ฒ ์ด์Šค๋ผ์ธ ๊ฐ’์€ ์› ๋…ผ๋ฌธ ๊ฐ’์ด๋ฉฐ โ€œโ€“โ€๋Š” ๋ฏธ๋ณด๊ณ ๋‹ค.

๋ฐ์ดํ„ฐ์…‹ยท์† ๋ฐฉ๋ฒ• \mathrm{SR}_{\mathrm{MT}} (%) \mathrm{SR}_{\mathrm{obj}} (%) ADD-AUC (%)
OakInk-v2 ์–‘์†, Inspire ManipTrans 39.5 โ€“ โ€“
SPIDER โ€“ 47.9* โ€“
DexOAK+RL 48.4 58.7 55.1
OakInk-v2 ์–‘์†, Allegro SPIDER โ€“ 45.9* โ€“
DexOAK+RL 58.6 64.1 61.5
ARCTIC, Inspire DexMachina โ€“ 67.1 62.7
SPIDER โ€“ 42.0 โ€“
DexOAK+RL 58.3 83.4 77.9
ARCTIC, Allegro DexMachina โ€“ โ€“ 83.4
DexOAK+RL 67.2 97.3 92.1

* SPIDER๋Š” OakInk-v2 ์‹œํ€€์Šค๋ฅผ ๋” ๋งŽ์ด ์จ์„œ ๋น„๊ต๊ฐ€ ์ฐธ๊ณ ์šฉ์ด๋‹ค(๋…ผ๋ฌธ ์บก์…˜). SPIDER ๋ฆฌ๋ทฐ์— ์ •๋ฆฌํ•ด ๋‘” ์› ๋…ผ๋ฌธ ๊ฐ’๊ณผ ๋Œ€์กฐํ•˜๋ฉด OakInk 47.9%(Inspire)ยท45.9%(Allegro)๋Š” SPIDER๊ฐ€ OakInk 1,022๊ฐœ ๊ถค์  ์ „์ฒด์—์„œ ๋ณด๊ณ ํ•œ ๊ฐ’์ด๊ณ , ARCTIC 42.0%์™€ DexMachina 67.1%๋Š” SPIDER์˜ RL ๋ฐฉ๋ฒ• ๋น„๊ตํ‘œ ๊ฐ’๊ณผ ๊ฐ™๋‹ค. ์ฆ‰ SPIDER ์ชฝ OakInk ์ˆ˜์น˜๋Š” ํ‰๊ฐ€ ์ง‘ํ•ฉ ์ž์ฒด๊ฐ€ ๋‹ค๋ฅด๋‹ค. ManipTrans 39.5%๋„ ManipTrans ๋ฆฌ๋ทฐ์˜ ์–‘์† ์„ฑ๊ณต๋ฅ ๊ณผ ๊ฐ™๋‹ค. ๋ฒ ์ด์Šค๋ผ์ธ ์ˆซ์ž ์ž์ฒด๋Š” ์› ๋…ผ๋ฌธ๊ณผ ๋งž๋Š”๋‹ค.


Fig. 4: ์™ผ์ชฝ์€ kinematic ๊ถค์ , ์˜ค๋ฅธ์ชฝ์€ dynamic ๊ถค์ . ์œ„ ์„ธ ํ–‰์€ Inspire ์–‘์† uncap alcohol lamp(OakInk-v2)์—์„œ ManipTrans(๋šœ๊ป‘์„ ๋ชป ์žก๊ณ  ์ผ์ฐ ์‹คํŒจ), SPIDER(๋šœ๊ป‘์„ ๋„ˆ๋ฌด ์ผ์ฐ ๋†“๊ณ  ๋ฒ„๋„ˆ๊ฐ€ ๊ธฐ์šบ), DexOAK+RL(์ „์ฒด ๋™์ž‘ ์ˆ˜ํ–‰). ๋„ท์งธ ํ–‰์€ Allegro ์–‘์† use box(ARCTIC). ๋งˆ์ง€๋ง‰ ํ–‰์€ Allegro ์˜ค๋ฅธ์† shake lab container์—์„œ ํ”๋“œ๋Š” ๋„์ค‘ ๋น„์ปค๋ฅผ ๋–จ์–ด๋œจ๋ฆฌ๋Š” ์‹คํŒจ ์‚ฌ๋ก€๋‹ค.

์ด ํ‘œ๋ฅผ ์ฝ์„ ๋•Œ ๋ฐ˜๋“œ์‹œ ๋ถ™์—ฌ์•ผ ํ•  ๋‹จ์„œ๋Š” ํ™”์‚ดํ‘œ ํ•˜๋‚˜์— ์—ฌ๋Ÿฌ ๋ณ€๊ฒฝ์ด ๋ฌถ์—ฌ ์žˆ๋‹ค๋Š” ์ ์ด๋‹ค. Inspire OakInk-v2์˜ 39.5 โ†’ 48.4(+8.9 %p)๋Š” โ‘  kinematic reference(hand-only imitator โ†’ DexOAK), โ‘ก RL backend(ManipTrans residual RL โ†’ ManipTrans ๋ณด์ƒ + DexMachina curriculum + DeepMimic tracking), โ‘ข ์‹คํ–‰ ์ฃผ์ฒด(์› ๋…ผ๋ฌธ์˜ ํ•˜๋“œ์›จ์–ดยทseed โ†’ ์ €์ž์˜ 5 seed)๊ฐ€ ๋™์‹œ์— ๋ฐ”๋€ ๊ฐ’์ด๋‹ค. ๋…ผ๋ฌธ๋„ ์ด๋ฅผ ์•Œ๊ณ  โ€œthis table compares complete systems rather than isolating the kinematic referenceโ€๋ผ๊ณ  ์บก์…˜๊ณผ ๋ณธ๋ฌธ์— ์ ์—ˆ๋‹ค. reference์˜ ์ˆœ์ˆ˜ ํšจ๊ณผ๋Š” ๊ฐ™์€ backend๋กœ ๊ณ ์ •ํ•œ Table I(+27.6 %p, Allegro ์˜ค๋ฅธ์†)๊ณผ Table III(+7.9 ADD-AUC, Allegro ARCTIC)์—์„œ ์ฝ์–ด์•ผ ํ•œ๋‹ค.

backend ํšจ๊ณผ๋ฅผ ๊ฐ€๋Š ํ•  ๋‹จ์„œ๊ฐ€ ํ•˜๋‚˜ ์žˆ๋‹ค. Allegro ARCTIC์˜ DexOAK๋Š” DexMachina RL์—์„œ 91.3(Table III), ํ†ตํ•ฉ RL์—์„œ 92.1(Table IV)์ด๋‹ค. ์ด ์กฐ๊ฑด์—์„œ๋Š” backend๋ฅผ ๋ฐ”๊ฟ”๋„ 0.8๋ฐ–์— ์›€์ง์ด์ง€ ์•Š์œผ๋ฏ€๋กœ, ์ ์–ด๋„ Allegro ARCTIC์˜ ์šฐ์œ„๋Š” backend๋ณด๋‹ค reference์—์„œ ์˜จ๋‹ค๊ณ  ์ฝ์„ ์ˆ˜ ์žˆ๋‹ค(ํ•„์ž์˜ ํ•ด์„, ๋‘ ์ˆ˜์น˜๋Š” seed ์ˆ˜๊ฐ€ 3๊ณผ 5๋กœ ๋‹ค๋ฅด๋‹ค). Inspire์™€ OakInk-v2์— ๋Œ€ํ•ด์„œ๋Š” ์ด๋Ÿฐ ํ†ต์ œ ๋น„๊ต๊ฐ€ ์—†๋‹ค.

Allegro๊ฐ€ Inspire๋ณด๋‹ค RL ์„ฑ๊ณต๋ฅ ์ด ๋†’์€ ๊ฒƒ(ARCTIC \mathrm{SR}_{\mathrm{MT}} 67.2 vs 58.3)์„ ์ €์ž๋Š” โ€œmay reflect Allegroโ€™s 16 controllable DoF versus Inspireโ€™s 6โ€๋กœ ์กฐ์‹ฌ์Šค๋Ÿฝ๊ฒŒ ํ•ด์„ํ•œ๋‹ค. ์ด๋Š” DexMachina ๋ฆฌ๋ทฐ์—์„œ ์ •๋ฆฌํ•œ hand embodiment ๋ถ„์„, ์ฆ‰ ํฌ๊ธฐ๋ณด๋‹ค DoF๊ฐ€ ์ค‘์š”ํ•˜๊ณ  ์™„์ „ ๊ตฌ๋™ ์†์ด ๋” ์ž˜ ๋ฐฐ์šด๋‹ค๋Š” ๊ด€์ฐฐ๊ณผ ๊ฐ™์€ ๋ฐฉํ–ฅ์ด๋‹ค. DexOAK๊ฐ€ ์ƒˆ๋กœ ๋ฐœ๊ฒฌํ•œ ์‚ฌ์‹ค์ด๋ผ๊ธฐ๋ณด๋‹ค ๊ธฐ์กด ๊ด€์ฐฐ์˜ ์žฌํ™•์ธ์œผ๋กœ ๋ณด๋Š” ๊ฒƒ์ด ๋งž๋‹ค.

์‹ค๊ธฐ ์‹คํ—˜ (ยงV-F, Table V)

์ ˆ์ฐจ๊ฐ€ ๊ฒฐ๊ณผ์˜ ์˜๋ฏธ๋ฅผ ์ •ํ•˜๋ฏ€๋กœ ๋จผ์ € ์ •๋ฆฌํ•œ๋‹ค.

  1. OakInk-v2 5๊ณผ์ œ๋งˆ๋‹ค DexOAK+RL๊ณผ ManipTrans๋ฅผ ๊ฐ๊ฐ 3 seed๋กœ ํ•™์Šตํ•œ๋‹ค.
  2. ๊ฐ ์ •์ฑ…์—์„œ ์ •ํ•ด์ง„ ํ‰๊ฐ€ ์ˆœ์„œ๋กœ ์ฒ˜์Œ \mathrm{SR}_{\mathrm{MT}}๋ฅผ ๋งŒ์กฑํ•œ rollout ํ•˜๋‚˜๋ฅผ ๊ณจ๋ผ ์†๋ชฉ ์ž์„ธ์™€ ์†๊ฐ€๋ฝ ๊ด€์ ˆ๊ฐ์„ ๊ธฐ๋กํ•œ๋‹ค.
  3. ๊ทธ ๊ถค์ ์„ UR5 + ์˜ค๋ฅธ์† Allegro์—์„œ open-loop๋กœ 4๋ฒˆ ์žฌ์ƒํ•œ๋‹ค. UR5 ๋ชฉํ‘œ๋Š” ๊ธฐ๋ก๋œ ์†๋ชฉ ์ž์„ธ์˜ IK๋กœ ์–ป๊ณ , 60 Hz๋กœ ๊ด€์ ˆ ๊ณต๊ฐ„์—์„œ ์žฌ์ƒํ•œ๋‹ค.
  4. ์žฌ์ƒ ์ „๋งˆ๋‹ค ๋ฌผ์ฒด๋ฅผ ํ‘œ์‹œ๋œ ์ดˆ๊ธฐ ์ž์„ธ(์‹œ๋ฎฌ๋ ˆ์ด์…˜ ์ดˆ๊ธฐํ™”์™€ 5 mm ์ด๋‚ด)๋กœ ์†์œผ๋กœ ๋˜๋Œ๋ฆฐ๋‹ค. ๋ฌผ์ฒด ์น˜์ˆ˜๋Š” ๋ชจ์บก ๋ฌผ์ฒด์™€ 2 mm ์ด๋‚ด๋กœ ๋งž์ท„๋‹ค.
๋ฐฉ๋ฒ• Take out Close gate Pour Unplug Cut
DexOAK+RL 12/12 [90%] 12/12 [41%] 10/12 [95%] 8/12 [93%] 4/8 [3%]
ManipTrans 6/12 [83%] 12/12 [12%] 8/12 [16%] 0/4 [1%] โ€“ [0%]

์นธ์€ โ€œํ•˜๋“œ์›จ์–ด ์„ฑ๊ณต/์‹œ๋„ [์‹œ๋ฎฌ๋ ˆ์ด์…˜ \mathrm{SR}_{\mathrm{MT}} seed ํ‰๊ท ]โ€์ด๋‹ค. ๋ถ„๋ชจ๊ฐ€ 12๋ณด๋‹ค ์ž‘์œผ๋ฉด 3๊ฐœ ์ •์ฑ… ์ค‘ ์ผ๋ถ€๊ฐ€ ์„ฑ๊ณต rollout์„ ํ•˜๋‚˜๋„ ๋ชป ๋งŒ๋“  ๊ฒฝ์šฐ์ด๊ณ , โ€œโ€“โ€๋Š” ์žฌ์ƒํ•  ์„ฑ๊ณต rollout์ด ์—†๋‹ค๋Š” ๋œป์ด๋‹ค. held-out ๊ณผ์ œ(retargeter ํ•™์Šต์—์„œ ์ œ์™ธ, ์ฒ˜์Œ ๋ณด๋Š” ๋ฌผ์ฒด ๋ฒ”์ฃผ์˜ non-prehensile ์ƒํ˜ธ์ž‘์šฉ) Close Laptop๊ณผ Press Button์€ ์‹œ๋ฎฌ๋ ˆ์ด์…˜ 99%ยท100%, ํ•˜๋“œ์›จ์–ด ๊ฐ 10/12๋‹ค. ์ด ๋‘˜์€ 46/56์— ํฌํ•จ๋˜์ง€ ์•Š๋Š”๋‹ค.

ํ”„๋กœ์ ํŠธ ํŽ˜์ด์ง€์˜ ์‹ค๊ธฐ ์˜์ƒ(UR5 + ์˜ค๋ฅธ์† Allegro, ์› ์†๋„, ํ•ด์ƒ๋„๋ฅผ ์ค„์—ฌ ์‹ค์—ˆ๋‹ค). ์™ผ์ชฝ ์œ„๋ถ€ํ„ฐ Take out(์„ ๋ฐ˜์˜ ๊ทธ๋ฆ‡์„ ๊บผ๋‚ด ์ ‘์‹œ ์œ„์— ๋†“๊ธฐ, 12/12), Unplug(๋ฉ€ํ‹ฐํƒญ์—์„œ ํ”Œ๋Ÿฌ๊ทธ ๋ฝ‘๊ธฐ, 8/12), Cut(์นผ์„ ์ฅ๊ณ  ์ž๋ฅด๋Š” ๋™์ž‘ ํ›„ ์Ÿ๋ฐ˜์— ๋‚ด๋ ค๋†“๊ธฐ, 4/8), Press Button(held-out ๊ณผ์ œ, 10/12). ํ”„๋กœ์ ํŠธ ํŽ˜์ด์ง€๊ฐ€ ๊ณผ์ œ๋ณ„๋กœ ํ•˜๋‚˜์”ฉ ๋ณด์—ฌ ์ฃผ๋Š” ์žฌ์ƒ ์˜์ƒ์ด๋‹ค.

์ €์ž์˜ ๊ฒฐ๋ก ์€ DexOAK+RL์ด ์•ž ์„ธ ๊ณผ์ œ์—์„œ ๊ฑฐ์˜ ์™„๋ฒฝํ•˜๊ณ , 5๊ณผ์ œ ๋ชจ๋‘์—์„œ ์‹œ๋ฎฌ๋ ˆ์ด์…˜ \mathrm{SR}_{\mathrm{MT}}๋ฅผ ์˜ฌ๋ฆฌ๋ฉฐ, ํ•˜๋“œ์›จ์–ด ์„ฑ๊ณต์—์„œ ManipTrans์™€ ๊ฐ™๊ฑฐ๋‚˜ ํฌ๊ฒŒ ์•ž์„ ๋‹ค๋Š” ๊ฒƒ์ด๋‹ค(Unplug๊ฐ€ ๊ฐ€์žฅ ๋šœ๋ ท). Unplug์™€ Cut์˜ ์‹คํŒจ๋Š” ๊ธ‰ํ•˜๊ฑฐ๋‚˜ ๊ณ ์ฃผํŒŒ์ธ ์›€์ง์ž„์—์„œ ํ”Œ๋Ÿฌ๊ทธ๋‚˜ ์นผ์ด ๋ฏธ๋„๋Ÿฌ์ง€๋Š” ๊ฒƒ์œผ๋กœ, open-loop์˜ ์ดˆ๊ธฐํ™” ๋ฏผ๊ฐ์„ฑ์ด ๊ฒน์นœ๋‹ค๊ณ  ์„ค๋ช…ํ•œ๋‹ค.

์ด ํ‘œ์—์„œ 46/56์ด๋ผ๋Š” ์ˆซ์ž๊ฐ€ ๋ฌด์—‡์„ ์„ธ๋Š”์ง€๋Š” ๋ถ„๋ช…ํžˆ ํ•ด ๋‘˜ ํ•„์š”๊ฐ€ ์žˆ๋‹ค.

  • ๋…๋ฆฝ ์‹œํ–‰์ด ์•„๋‹ˆ๋‹ค. 56ํšŒ๋Š” ์„œ๋กœ ๋‹ค๋ฅธ ๊ถค์  14๊ฐœ(3+3+3+3+2)๋ฅผ ๊ฐ๊ฐ 4๋ฒˆ ์žฌ์ƒํ•œ ๊ฒƒ์ด๋‹ค. ๊ฐ™์€ ๊ถค์ ์˜ ๋ฐ˜๋ณต ์žฌ์ƒ์€ โ€œ๊ทธ ๊ถค์ ์ด ํ•˜๋“œ์›จ์–ด์—์„œ ์žฌํ˜„ ๊ฐ€๋Šฅํ•œ๊ฐ€โ€๋ฅผ ์žฌ๋Š” ๊ฒƒ์ด๊ณ , ์ •์ฑ…์˜ ์„ฑ๊ณต๋ฅ ์ด ์•„๋‹ˆ๋‹ค.
  • ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ์„ฑ๊ณต์— ์กฐ๊ฑด๋ถ€๋‹ค. ์žฌ์ƒ ๋Œ€์ƒ์€ ์ฒ˜์Œ ์„ฑ๊ณตํ•œ rollout๋ฟ์ด๋‹ค. ๋…ผ๋ฌธ๋„ โ€œHardware success is therefore conditional on simulation successโ€๋ผ๊ณ  ์“ด๋‹ค. Close gate๋Š” ์‹œ๋ฎฌ๋ ˆ์ด์…˜ 41%์ธ๋ฐ ํ•˜๋“œ์›จ์–ด 12/12์ด๊ณ , ManipTrans๋„ ์‹œ๋ฎฌ๋ ˆ์ด์…˜ 12%์—์„œ ํ•˜๋“œ์›จ์–ด 12/12๋‹ค. ์ด ๊ณผ์ œ์—์„œ ํ•˜๋“œ์›จ์–ด ์ˆ˜์น˜๋Š” ๋‘ ๋ฐฉ๋ฒ•์„ ๊ฐ€๋ฅด์ง€ ๋ชปํ•œ๋‹ค.
  • ์†Œํ‘œ๋ณธ ํ†ต๊ณ„. ์žฌ์ƒ์„ ๋…๋ฆฝ ์‹œํ–‰์œผ๋กœ ๊ฐ€์ •ํ•ด๋„ ๊ณผ์ œ๋ณ„ Fisher exact p๋Š” Take out 0.014, Unplug 0.077, Pour 0.64, Close gate 1.0์ด๋‹ค(ํ•„์ž ๊ณ„์‚ฐ). ํ•ฉ๊ณ„ 46/56(82%, Wilson 95% ๊ตฌ๊ฐ„ 70~90%) ๋Œ€ 26/40(65%, 50~78%)์˜ p๋Š” 0.093์ด๋‹ค. ์‹ค์ œ๋กœ๋Š” ๊ถค์  ๋‹จ์œ„๋กœ ๋ฌถ์—ฌ ์žˆ์–ด ์œ ํšจ ํ‘œ๋ณธ์ด ๋” ์ž‘๋‹ค. ๋”ฐ๋ผ์„œ ํ•˜๋“œ์›จ์–ด ์‹คํ—˜์ด ๋ณด์—ฌ ์ฃผ๋Š” ๊ฒƒ์€ โ€œRL๋กœ ๋‹ค๋“ฌ์€ DexOAK ๊ถค์ ์ด ์‹ค๊ธฐ์—์„œ open-loop๋กœ ์žฌํ˜„๋œ๋‹คโ€๋Š” ์‹คํ–‰ ๊ฐ€๋Šฅ์„ฑ์ด๊ณ , ManipTrans ๋Œ€๋น„ ์šฐ์›”์„ฑ์˜ ๊ทผ๊ฑฐ๋Š” ์‹œ๋ฎฌ๋ ˆ์ด์…˜ \mathrm{SR}_{\mathrm{MT}} ์—ด์ด ๋” ๊ฐ•ํ•˜๋‹ค(Pour 95% vs 16%, Unplug 93% vs 1%).

์–ด๋А OakInk-v2 ์‹œํ€€์Šค๊ฐ€ ํ•˜๋“œ์›จ์–ด ๊ณผ์ œ์— ์“ฐ์˜€๋Š”์ง€๋Š” ๋…ผ๋ฌธ์— ๋ช…์‹œ๋ผ ์žˆ์ง€ ์•Š๋‹ค. ๋ถ€๋ก Table S6์—์„œ Take outยทClose gateยทCutยทremove plug๋Š” ์–‘์† ๋ชฉ๋ก์—๋งŒ ์žˆ๊ณ  ์˜ค๋ฅธ์† ๋ชฉ๋ก์—๋Š” Pour๋งŒ ์žˆ์œผ๋ฏ€๋กœ, ์–‘์† ๊ณผ์ œ์˜ ์˜ค๋ฅธ์†๋งŒ ์žฌ์ƒํ•œ ๊ฒƒ์ผ ๊ฐ€๋Šฅ์„ฑ์ด ์žˆ์ง€๋งŒ ํ™•์ธํ•  ์ˆ˜ ์—†๋‹ค.

์ธก์ •๋œ ๊ฒƒ๊ณผ ์„ ์–ธ๋œ ๊ฒƒ

์ฃผ์žฅ ๊ทผ๊ฑฐ์˜ ์ข…๋ฅ˜ ๊ทผ๊ฑฐ
object-aware reference๊ฐ€ downstream RL ์„ฑ๊ณต์„ ์˜ฌ๋ฆฐ๋‹ค ์ธก์ •(๊ฐ™์€ backend) Table I(20.6 โ†’ 48.2), Table III(83.4 โ†’ 91.3)
์ด๋“์˜ ์ฃผ์›์ฒœ์€ object anchor ์ธก์ • Table I ๋ถ„ํ•ด, Table II โ€œwithout object anchorsโ€
ํ•™์Šต + fine-tuning์ด ์ตœ์ ํ™”๋ณด๋‹ค ๋‚ซ๋‹ค ์ธก์ •, ๋ถ„์‚ฐ ๋ฏธ๋ณด๊ณ  Table I(+5.0 %p), Table III(+2.4)
RL์ด ์ ‘์ด‰ ํƒ์ƒ‰ ๋Œ€์‹  dynamics์— ์ง‘์ค‘ํ•˜๊ฒŒ ๋œ๋‹ค ๊ฐ€์„ค(โ€œWe hypothesizeโ€) ํ•™์Šต ๊ณก์„ ยทํƒ์ƒ‰ ๋ถ„์„ ์—†์Œ
ํ•™์Šต๋œ retargeter๊ฐ€ ์ƒˆ ๋ฐ์ดํ„ฐ์…‹์— zero-shot ์ „์ด ์ธก์ •(1๊ฐœ ์ „์ด ๋ฐฉํ–ฅ) Table III, OakInk-v2 โ†’ ARCTIC๋งŒ
๊ฐ ๊ตฌ์„ฑ์š”์†Œ๊ฐ€ ํ•„์š”ํ•˜๋‹ค ์ธก์ •(kinematic ์ง€ํ‘œ๋งŒ) Table II, RL ์„ฑ๊ณต๋ฅ  ์—†์Œ
SOTA๋ฅผ ๋„˜๋Š”๋‹ค ์‹œ์Šคํ…œ ๋น„๊ต(referenceยทbackendยท์‹คํ–‰ ์ฃผ์ฒด๊ฐ€ ํ•จ๊ป˜ ๋ฐ”๋€œ) Table IV
ํ•˜๋“œ์›จ์–ด ์‹คํ–‰ ๊ฐ€๋Šฅ ์ธก์ •(๊ถค์  14๊ฐœ ร— 4ํšŒ, open-loop) Table V
์† DoF๊ฐ€ RL ์„ฑ๊ณต ์ฐจ์ด๋ฅผ ๋งŒ๋“ ๋‹ค ์ถ”์ธก(โ€œmay reflectโ€) ํ†ต์ œ ์‹คํ—˜ ์—†์Œ
๋™์‹œ๋Œ€ Laplacian ๋ฐฉ๋ฒ•๋ณด๋‹ค ๋‚ซ๋‹ค ์„ ์–ธ(์žฌ์ตœ์ ํ™”๊ฐ€ ํ•„์š” ์—†๋‹ค๋Š” ์„ค๊ณ„ ๋…ผ์ฆ) ์ง์ ‘ ๋น„๊ต ์—†์Œ

๋น„ํŒ์ ์œผ๋กœ ๋ณด๋ฉด

๊ฐ•์ 

  • ๋ฌธ์ œ ๋ถ„ํ•ด๊ฐ€ ์ •ํ™•ํ•˜๋‹ค. โ€œkinematic ๋‹จ๊ณ„๋Š” ์–ด๋””, dynamic ๋‹จ๊ณ„๋Š” ์–ด๋–ป๊ฒŒโ€๋ผ๋Š” ๋ถ„ํ•ด์™€ โ€œ๊ธฐ์กด kinematic ๋‹จ๊ณ„๋Š” ๋ฌผ์ฒด๋ฅผ ์•ˆ ๋ณธ๋‹คโ€๋Š” ์ง„๋‹จ์ด ๋ช…๋ฃŒํ•˜๊ณ , ๊ฐ™์€ RL backend๋กœ reference๋งŒ ๋ฐ”๊พธ๋Š” ํ†ต์ œ ์‹คํ—˜(Table IยทIII)์ด ์ด ์ง„๋‹จ์„ ์ •๋Ÿ‰์œผ๋กœ ๋’ท๋ฐ›์นจํ•œ๋‹ค. hand-only 5์ข…์„ ๋ชจ๋‘ ๊ฐ™์€ backend๋กœ ๋‹ค์‹œ ๋Œ๋ฆฐ Table I์€ ์ด ๋ถ„์•ผ์—์„œ ๋“œ๋ฌธ ๊นจ๋—ํ•œ ๋น„๊ต๋‹ค.
  • amortize์˜ ์“ฐ์ž„์ƒˆ๊ฐ€ ๋ถ„๋ช…ํ•˜๋‹ค. ๋„คํŠธ์›Œํฌ๋ฅผ ๋‹จ์ง€ ๋น ๋ฅธ ๊ทผ์‚ฌ๊ธฐ๋กœ ์“ฐ๋Š” ๊ฒƒ์ด ์•„๋‹ˆ๋ผ โ€œ๋ฏธ๋ถ„ ๊ฐ€๋Šฅํ•˜๋‹ˆ ๋ฐ์ดํ„ฐ์…‹ ๋‹จ์œ„ ์†์‹ค๋กœ ํ•ด๋ฅผ ๊ฐœ์„ ํ•  ์ˆ˜ ์žˆ๋‹คโ€๋Š” ์ชฝ์œผ๋กœ ์“ด๋‹ค. supervised ์ดˆ๊ธฐํ™” ์—†์ด๋Š” fine-tuning์ด ๋ฌด๋„ˆ์ง„๋‹ค๋Š” ablation(Table II)์ด โ€œ์ตœ์ ํ™” โ†’ amortize โ†’ fine-tuneโ€ ์ˆœ์„œ๊ฐ€ ํ•„์—ฐ์ž„์„ ๋ณด์—ฌ ์ค€๋‹ค.
  • ์ ‘์ด‰ ์†์‹ค์˜ ์„ค๊ณ„๊ฐ€ ์‹คํŒจ ์œ ํ˜•๋ณ„๋กœ ๊น”๋”ํ•˜๋‹ค. surface(๋œธ), part(์—‰๋šฑํ•œ ๋ถ€์œ„), direction(๋ฐ˜๋Œ€ ๋ฉด)์˜ ์„ธ ํ•ญ๊ณผ ์ ‘๊ทผยท๋†“๊ธฐ ๊ตฌ๊ฐ„์„ ๋„๋Š” gate๋Š” ๊ฐ๊ฐ์˜ ์—ญํ• ์ด ๊ฒน์น˜์ง€ ์•Š๊ณ , leave-one-out์—์„œ ์…‹ ๋‹ค TPE๋ฅผ 2๋ฐฐ ์ด์ƒ ์•…ํ™”์‹œํ‚จ๋‹ค.
  • ์ž๊ธฐ ํ•œ๊ณ„๋ฅผ ์ˆจ๊ธฐ์ง€ ์•Š๋Š”๋‹ค. kinematic ์ง€ํ‘œ์˜ ์ง„๋‹จ์  ์„ฑ๊ฒฉ, SPIDER ๋น„๊ต์˜ ์ฐธ๊ณ ์šฉ ์„ฑ๊ฒฉ, ์‹œ์Šคํ…œ ๋น„๊ต๋ผ๋Š” Table IV์˜ ์„ฑ๊ฒฉ, ํ•˜๋“œ์›จ์–ด ์„ฑ๊ณต์˜ ์กฐ๊ฑด๋ถ€์„ฑ, ํ”๋“ค๊ธฐ ๊ฐ™์€ ๋™์  ๊ณผ์ œ์˜ ์‹คํŒจ๋ฅผ ๋ชจ๋‘ ๋ณธ๋ฌธ์— ์ ์—ˆ๋‹ค. ๋ฌผ์ฒด ์ ๊ตฐ ์ž…๋ ฅ์ด ๋ณ„๋กœ ์ค‘์š”ํ•˜์ง€ ์•Š๋‹ค๋Š” ์ž๊ธฐ ๋ถˆ๋ฆฌํ•œ ablation๋„ ๊ทธ๋Œ€๋กœ ์‹ฃ๋Š”๋‹ค.
  • ์žฌํ˜„ ๋‹จ์„œ๊ฐ€ ๋น„๊ต์  ๋งŽ๋‹ค. ๋ถ€๋ก์— IG ๋…ธ๋“œ ๋งํฌ ์ด๋ฆ„(Table S1), RL ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ(Table S3), ARCTIC ํด๋ฆฝยทOakInk-v2 ์‹œํ€€์Šค ํ•ด์‹œ ๋ชฉ๋ก(Table S4~S6)๊ณผ seed ๊ฐ’์ด ์žˆ๋‹ค.

์•ฝ์ ยทํ•œ๊ณ„

  • ํ•ต์‹ฌ ๊ธฐ์—ฌ์˜ ๋ชซ์ด ์ž‘๋‹ค. ์ด๋“์˜ 82%(Table I)ยท70%(Table III)๋Š” object anchor๋ฅผ ๋„ฃ์€ interaction-graph ์ตœ์ ํ™”์—์„œ ๋‚˜์˜ค๋Š”๋ฐ, ์ด ๋ถ€๋ถ„์€ OmniRetarget์˜ ์†๊ฐ€๋ฝ ๋‹จ์œ„ ํ™•์žฅ์ด๊ณ  ๋™์‹œ๋Œ€ RegrindยทTopoRetargetยทC2Dex๋„ ๊ฐ™์€ ๊ณ„์—ด ๋ชฉ์ ํ•จ์ˆ˜๋ฅผ ์“ด๋‹ค. ์ด ๋…ผ๋ฌธ๋งŒ์˜ ๊ธฐ์—ฌ์ธ ํ•™์Šต + ์ ‘์ด‰ fine-tuning์€ +5.0 %p, +2.4 ADD-AUC์ด๊ณ , ๋ถ„์‚ฐ์ด ๋ณด๊ณ ๋˜์ง€ ์•Š์•„ seed ์žก์Œ๊ณผ ๊ตฌ๋ถ„๋˜๋Š”์ง€ ์•Œ ์ˆ˜ ์—†๋‹ค. ๋™์‹œ๋Œ€ Laplacian ๋ฐฉ๋ฒ•๊ณผ์˜ ์ง์ ‘ ๋น„๊ต๋„ ์—†๋‹ค.
  • ํ—ค๋“œ๋ผ์ธ SOTA ์ˆ˜์น˜๋Š” ์—ฌ๋Ÿฌ ๋ณ€๊ฒฝ์˜ ํ•ฉ์ด๋‹ค. ์ดˆ๋ก์˜ 39.5 โ†’ 48.4์™€ 62.7 โ†’ 77.9๋Š” reference, RL backend(DexMachina curriculum ์ถ”๊ฐ€), ์‹คํ–‰ ์ฃผ์ฒด๊ฐ€ ํ•จ๊ป˜ ๋ฐ”๋€ ๋น„๊ต๋‹ค. ์ €์ž๊ฐ€ ์บก์…˜์— ๋ฐํ˜”์ง€๋งŒ ์ดˆ๋ก์—๋Š” ์ด ๋‹จ์„œ๊ฐ€ ์—†๋‹ค. ํ†ต์ œ ๋น„๊ต๊ฐ€ ์žˆ๋Š” ๊ฒƒ์€ Allegro ์˜ค๋ฅธ์† OakInk-v2์™€ Allegro ARCTIC๋ฟ์ด๊ณ , ์ดˆ๋ก์ด ์ธ์šฉํ•œ Inspire ์กฐ๊ฑด์—๋Š” ์—†๋‹ค.
  • โ€œ์žฌ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ retargeterโ€์˜ ๋ฒ”์œ„๊ฐ€ kinematic ๋‹จ๊ณ„์—์„œ ๋ฉˆ์ถ˜๋‹ค. held-out ๊ณผ์ œ๋„ โ€œfollowed by the same task-specific RL stageโ€์ด๊ณ , RL์€ ๊ณผ์ œ(์‹œํ€€์Šค)๋งˆ๋‹ค 4,096 env๋กœ ๋‹ค์‹œ ํ•™์Šตํ•œ๋‹ค. ํŒŒ์ดํ”„๋ผ์ธ ์ „์ฒด ๋น„์šฉ์—์„œ RL์ด ์ฐจ์ง€ํ•˜๋Š” ๋ชซ์€ ๋ณด๊ณ ๋˜์ง€ ์•Š๋Š”๋‹ค. ์ €์ž๋„ ๊ฒฐ๋ก ์—์„œ ๊ณผ์ œยทembodiment ์กฐ๊ฑด๋ถ€ residual ์ •์ฑ… ํ•˜๋‚˜๋กœ dynamics ๋ณด์ •์„ ๊ณต์œ ํ•˜๋Š” ๊ฒƒ์„ future work๋กœ ๋“ ๋‹ค. ์ด ๋ฐฉํ–ฅ์€ ์‹œ์—ฐ ์ˆ˜์‹ญ~์ˆ˜์ฒœ ๊ฐœ๋ฅผ ํ•œ ์ •์ฑ…์œผ๋กœ ์ถ”์ข…ํ•˜๋Š” FlashDexRetarget์ด ์ด๋ฏธ ๋‹ค๋ฃฌ ๋ฌธ์ œ์ด๊ณ , ๋‘ ๋ฐฉ๋ฒ•์€ ๊ฐ๊ฐ kinematic ๋‹จ๊ณ„์™€ dynamic ๋‹จ๊ณ„์˜ amortization์ด๋ผ ์ƒ๋ณด์ ์ด๋‹ค.
  • kinematic ์ง€ํ‘œ๊ฐ€ ๋ชฉ์ ํ•จ์ˆ˜์™€ ๊ฒน์นœ๋‹ค. TPE๋Š” ์†๋ ์ƒ๋Œ€ ๋ฐฐ์น˜, CC๋Š” ํ‘œ๋ฉด ๊ฑฐ๋ฆฌ๋ผ ๊ฐ๊ฐ \mathcal{L}_{\mathrm{IG}}ยท\mathcal{L}_{\mathrm{part}}์™€ \mathcal{L}_{\mathrm{surf}}๊ฐ€ ์ง์ ‘ ์ตœ์†Œํ™”ํ•˜๋Š” ์–‘์ด๋‹ค. Table II์ฒ˜๋Ÿผ kinematic ์ง€ํ‘œ๋งŒ ์žˆ๋Š” ablation์€ โ€œ๋ชฉ์ ํ•จ์ˆ˜๋ฅผ ๋บ๋”๋‹ˆ ๊ทธ ๋ชฉ์ ํ•จ์ˆ˜๊ฐ€ ๋‚˜๋น ์กŒ๋‹คโ€์— ๊ฐ€๊น๋‹ค. ๊ตฌ์„ฑ์š”์†Œ๋ณ„ downstream RL ์„ฑ๊ณต๋ฅ ์ด ์žˆ์—ˆ์–ด์•ผ ํ•œ๋‹ค.
  • ๋ณธ๋ฌธ๊ณผ ๋ถ€๋ก์˜ ๊ตฌํ˜„ ๊ฐ’์ด ์—ฌ๋Ÿฌ ๊ณณ์—์„œ ๋‹ค๋ฅด๋‹ค. ์ถฉ๋Œ margin(5 vs 2 mm), fine-tuning epoch(24 vs 25), decoder ํฌ๊ธฐ, tetrahedralization ์žฌ๊ณ„์‚ฐ ์—ฌ๋ถ€, autoregressive window(1 vs 9 ํ”„๋ ˆ์ž„), keypoint token ์ˆ˜(21 vs 42), frame step(๋ฐ์ดํ„ฐ 60 Hz vs \mathrm{d}t=1/30 s)๊ฐ€ ์—‡๊ฐˆ๋ฆฌ๊ณ , โ€œ๊ฐ€์ค‘์น˜๋Š” Table II ์ฐธ์กฐโ€๋Š” ๊ฐ€์ค‘์น˜๊ฐ€ ์—†๋Š” ํ‘œ๋ฅผ ๊ฐ€๋ฆฌํ‚จ๋‹ค. ์ฝ”๋“œ๊ฐ€ ๊ณต๊ฐœ๋˜์ง€ ์•Š์€ ์ƒํƒœ๋ผ ์–ด๋А ์ชฝ์ด ์‹ค์ œ ์‹คํ—˜ ์„ค์ •์ธ์ง€ ํ™•์ธํ•  ๋ฐฉ๋ฒ•์ด ์—†๋‹ค.
  • ํ•˜๋“œ์›จ์–ด ์‹คํ—˜์€ ์‹คํ–‰ ๊ฐ€๋Šฅ์„ฑ ์‹œ์—ฐ์ด๋‹ค. 14๊ฐœ ๊ถค์ ์˜ ๋ฐ˜๋ณต ์žฌ์ƒ์ด๊ณ , ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ์„ฑ๊ณต rollout์„ ๊ณจ๋ผ ์žฌ์ƒํ•˜๋ฏ€๋กœ ์ •์ฑ…์˜ ์‹ค๊ธฐ ์„ฑ๊ณต๋ฅ ์ด ์•„๋‹ˆ๋‹ค. ๊ณผ์ œ๋ณ„ ๋น„๊ต์—์„œ ๋…๋ฆฝ ์‹œํ–‰์„ ๊ฐ€์ •ํ•ด๋„ ํ†ต๊ณ„์ ์œผ๋กœ ๊ตฌ๋ณ„๋˜๋Š” ๊ฒƒ์€ Take out ํ•˜๋‚˜๋‹ค. ์–ด๋–ค ์‹œํ€€์Šค(์–‘์† ๊ณผ์ œ์˜ ์˜ค๋ฅธ์†์ธ์ง€)๋ฅผ ์ผ๋Š”์ง€๋„ ๋ช…์‹œ๋ผ ์žˆ์ง€ ์•Š๋‹ค.
  • ์ผ๋ฐ˜ํ™” ์ฆ๊ฑฐ๊ฐ€ ํ•œ ๋ฐฉํ–ฅ์ด๋‹ค. OakInk-v2 โ†’ ARCTIC, Allegro ํ•œ ์†, DexMachina RL ํ•œ ์กฐ๊ฑด์—์„œ๋งŒ ์ธก์ •ํ–ˆ๋‹ค. ์—ญ๋ฐฉํ–ฅ(ARCTIC 30 ์‹œํ€€์Šค๋กœ ํ•™์Šต โ†’ OakInk-v2)์ด๋‚˜ Inspire ์ „์ด๋Š” ์—†๋‹ค. ์†๋„ 108.9 Hz๋Š” source ์ชฝ ๋น„์šฉ์„ ๋บ€ ํ•œ๊ณ„ ๋น„์šฉ์ด๋‹ค.
  • ๋™์  ๊ณผ์ œ์—์„œ ๊ตฌ์กฐ์ ์œผ๋กœ ์•ฝํ•˜๋‹ค. reference๊ฐ€ kinematic์ด๋ผ ์šด๋™๋Ÿ‰ยทํž˜ ์ •๋ณด๋ฅผ ๋‹ด์ง€ ์•Š๋Š”๋‹ค. ํ”๋“ค๊ธฐยท์ž๋ฅด๊ธฐ์ฒ˜๋Ÿผ ๊ฐ€์†์ด ์ค‘์š”ํ•œ ๊ณผ์ œ์—์„œ ์‹คํŒจ๊ฐ€ ์ง‘์ค‘๋˜๊ณ (Fig. 4, Cut ์‹œ๋ฎฌ๋ ˆ์ด์…˜ 3%), ์ด๊ฒƒ์€ fine-tuning ์†์‹ค์„ ๋Š˜๋ ค์„œ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ๋Š” ์ข…๋ฅ˜์˜ ๋ฌธ์ œ๊ฐ€ ์•„๋‹ˆ๋‹ค. ์ ‘์ด‰ ํž˜์ด๋‚˜ wrench๋ฅผ reference์— ๋‹ด๋Š” CHORD ๊ฐ™์€ ๋ฐฉํ–ฅ๊ณผ ๊ฒฐํ•ฉํ•  ์—ฌ์ง€๊ฐ€ ์žˆ๋‹ค.
  • ์ถฉ๋Œ์„ soft penalty๋กœ๋งŒ ๋‹ค๋ฃฌ๋‹ค. OmniRetarget์ด ๋น„๊ด€ํ†ต์„ hard constraint๋กœ ๊ฑธ์–ด ์ •์˜์ƒ ์ œ๊ฑฐํ•œ ๊ฒƒ๊ณผ ๋‹ฌ๋ฆฌ, DexOAK๋Š” seed ์ตœ์ ํ™”์™€ fine-tuning ๋ชจ๋‘์—์„œ hinge penalty๋งŒ ์“ด๋‹ค. reference์— ๋‚จ๋Š” ๊ด€ํ†ต์˜ ์–‘์€ ๋ณด๊ณ ๋˜์ง€ ์•Š๋Š”๋‹ค.

๊ด€๋ จ ์—ฐ๊ตฌ์™€์˜ ์ž๋ฆฌ ๋งค๊น€

DexOAK๋Š” ์†์žฌ์ฃผ ๋ฆฌํƒ€๊ฒŒํŒ… ๊ณ„๋ณด์˜ kinematic ๋‹จ๊ณ„๋ฅผ ๋‹ค์‹œ ์„ค๊ณ„ํ•œ ๋…ผ๋ฌธ์ด๋‹ค. ์•ž์ชฝ์—๋Š” ๋ฌผ์ฒด๋ฅผ ๋ณด์ง€ ์•Š๋Š” keypoint ๋งค์นญ(DexPilot, AnyTeleop)๊ณผ ํ•™์Šตํ˜• ๋งคํ•‘(GeoRT)์ด ์žˆ๊ณ , ์˜†์—๋Š” interaction mesh๋กœ ๋ชธ-๋ฌผ์ฒด ๊ด€๊ณ„๋ฅผ ๋ณด์กดํ•œ ํœด๋จธ๋…ธ์ด๋“œ ์ชฝ OmniRetarget์ด ์žˆ๋‹ค. DexOAK๋Š” OmniRetarget์˜ Laplacian ๋ชฉ์ ํ•จ์ˆ˜๋ฅผ ์†๊ฐ€๋ฝ ๋‹จ์œ„ยท๋‹ค๋ฌผ์ฒดยท์‹ค์ œ ์Šค์ผ€์ผ๋กœ ํ™•์žฅํ•˜๊ณ , ๊ทธ ํ•ด๋ฅผ ๋„คํŠธ์›Œํฌ๋กœ amortizeํ•ด ์ ‘์ด‰ ์†์‹ค๋กœ ๋‹ค๋“ฌ๋Š”๋‹ค.

๊ฐ™์€ ๋ชฉ์ ํ•จ์ˆ˜๋ฅผ ์†์— ๊ฐ€์ ธ์˜จ ๋™์‹œ๋Œ€ ์—ฐ๊ตฌ๋กœ๋Š” Regrind๊ฐ€ ๊ฐ€์žฅ ๊ฐ€๊น๋‹ค. Regrind๋„ ์†-๋ฌผ์ฒด interaction mesh๋กœ reference๋ฅผ ๋งŒ๋“ค๊ณ  residual RL๋กœ ์‹ค๊ธฐ ์ „์ด๊นŒ์ง€ ๊ฐ€๋Š”๋ฐ, ์‹œ์—ฐ๋งˆ๋‹ค ์ตœ์ ํ™”๋ฅผ ๋‹ค์‹œ ํ‘ผ๋‹ค. DexOAK๋Š” ์ด per-demonstration ์ตœ์ ํ™”๋ฅผ ํ•™์Šต ๋ฐ์ดํ„ฐ ์ƒ์„ฑ์šฉ์œผ๋กœ๋งŒ ์“ฐ๊ณ  ์ดํ›„์—๋Š” forward pass๋กœ ๋Œ€์ฒดํ•œ๋‹ค๋Š” ์ ์ด ๋‹ค๋ฅด๋‹ค. ๋‘ ๋…ผ๋ฌธ ์‚ฌ์ด์˜ ์ง์ ‘ ๋น„๊ต๋Š” ์—†๋‹ค.

๋’ค์ชฝ์˜ dynamic ๋‹จ๊ณ„์—์„œ๋Š” ManipTrans(residual RL), DexMachina(virtual object control curriculum), SPIDER(์ƒ˜ํ”Œ๋ง)๊ฐ€ ๊ธฐ์ค€์„ ์ด๋‹ค. DexOAK๋Š” ์ด๋“ค์„ ๋Œ€์ฒดํ•˜์ง€ ์•Š๊ณ  ์ž…๋ ฅ์„ ๋ฐ”๊พผ๋‹ค. ์‹ค์ œ๋กœ Table IยทIII์€ ManipTrans์™€ DexMachina์˜ RL์„ ๊ทธ๋Œ€๋กœ ์“ฐ๊ณ  reference๋งŒ ๊ต์ฒดํ•œ๋‹ค. ์ด ์ ์—์„œ DexOAK๋Š” โ€œ์–ด๋–ค dynamic ๋ฐฉ๋ฒ•์—๋„ ๊ฝ‚์„ ์ˆ˜ ์žˆ๋Š” ๋” ์ข‹์€ reference ๊ณต๊ธ‰์žโ€๋กœ ์ฝ๋Š” ๊ฒƒ์ด ์ •ํ™•ํ•˜๋‹ค. human reference๋ฅผ ๋ฒ”์šฉ ์ถ”์ข… ์ •์ฑ…์œผ๋กœ ๋‹ค๋ฃจ๋Š” DexTrack, ์—ฌ๋Ÿฌ ์‹œ์—ฐ์„ ํ•œ ์ •์ฑ…์œผ๋กœ ์ถ”์ข…ํ•ด dynamic ๋‹จ๊ณ„ ๋น„์šฉ์„ amortizeํ•˜๋Š” FlashDexRetarget๊ณผ ๊ฒฐํ•ฉํ•˜๋ฉด, kinematic๊ณผ dynamic ์–‘์ชฝ์—์„œ ์‹œ์—ฐ๋ณ„ ์žฌํ•™์Šต์ด ์‚ฌ๋ผ์ง„ ํŒŒ์ดํ”„๋ผ์ธ์ด ๋œ๋‹ค. ์ €์ž๊ฐ€ future work๋กœ ๋“  RGB ์˜์ƒ ๊ธฐ๋ฐ˜ ์†-๋ฌผ์ฒด ์žฌ๊ตฌ์„ฑ๊ณผ์˜ ์—ฐ๊ฒฐ์€ ๋™์‹œ๋Œ€ C2Dex๊ฐ€ ๊ฒจ๋ƒฅํ•œ ๋ฐฉํ–ฅ์ด๊ธฐ๋„ ํ•˜๋‹ค.

์š”์•ฝ

DexOAK๋Š” kinematic retargeting์˜ ๋ชฉํ‘œ๋ฅผ ์† ๋ชจ์–‘์—์„œ ์†-๋ฌผ์ฒด ์ƒํ˜ธ์ž‘์šฉ์œผ๋กœ ์˜ฎ๊ธด๋‹ค. interaction-graph Laplacian ์ตœ์ ํ™”๋กœ ๋ฌผ์ฒด ๊ธฐ์ค€ ์†๋ ๋ฐฐ์น˜๋ฅผ ๋ณด์กดํ•œ seed๋ฅผ ๋งŒ๋“ค๊ณ , ์ด๋ฅผ autoregressive ๋„คํŠธ์›Œํฌ๋กœ amortizeํ•œ ๋’ค surfaceยทpartยทdirection ์ ‘์ด‰ ์†์‹ค๋กœ fine-tuningํ•œ๋‹ค. ๊ฐ™์€ RL backend์—์„œ reference๋งŒ ๋ฐ”๊พธ๋ฉด ์„ฑ๊ณต๋ฅ ์ด 20.6%์—์„œ 48.2%๋กœ ์˜ค๋ฅด๊ณ (Allegro ์˜ค๋ฅธ์† OakInk-v2), OakInk-v2๋กœ ํ•™์Šตํ•œ retargeter๋Š” ARCTIC์— zero-shot์œผ๋กœ ์ „์ด๋˜๋ฉฐ, ์‹ค๊ธฐ์—์„œ RL๋กœ ๋‹ค๋“ฌ์€ ๊ถค์ ์ด open-loop๋กœ ์žฌํ˜„๋œ๋‹ค.

์ •์งํ•˜๊ฒŒ ๋‚˜๋ˆ„๋ฉด, ์„ฑ๊ณต๋ฅ ์„ ๋‘ ๋ฐฐ๋กœ ๋งŒ๋“  ์ฃผ์—ญ์€ object anchor๋ฅผ ๋„ฃ์€ ์ตœ์ ํ™”์ด๊ณ  ์ด ๋…ผ๋ฌธ์˜ ๊ณ ์œ  ๊ธฐ์—ฌ์ธ ํ•™์Šต + ์ ‘์ด‰ fine-tuning์€ ๊ทธ ์œ„์— ์•ฝ 5 %p์™€ ์žฌ์‚ฌ์šฉ์„ฑ์„ ๋”ํ•œ๋‹ค. ์ดˆ๋ก์˜ SOTA ์ˆ˜์น˜๋Š” RL backend๊นŒ์ง€ ๋ฐ”๋€ ์‹œ์Šคํ…œ ๋น„๊ต์ด๊ณ , ํ•˜๋“œ์›จ์–ด 46/56์€ ๊ถค์  14๊ฐœ์˜ ๋ฐ˜๋ณต ์žฌ์ƒ์ด๋‹ค. ๋ณธ๋ฌธ๊ณผ ๋ถ€๋ก์˜ ๊ตฌํ˜„ ๊ฐ’์ด ์—ฌ๋Ÿฌ ๊ณณ์—์„œ ์—‡๊ฐˆ๋ฆฌ๋Š”๋ฐ ์ฝ”๋“œ๊ฐ€ ์•„์ง ๊ณต๊ฐœ๋˜์ง€ ์•Š์•„ ํ™•์ธํ•  ์ˆ˜ ์—†๋‹ค. ๊ทธ๋Ÿผ์—๋„ โ€œdynamic ๋‹จ๊ณ„๊ฐ€ ์ ‘์ด‰์„ ์ฐพ๊ฒŒ ๋‘์ง€ ๋ง๊ณ  reference์— ๋„ฃ์–ด ์ฃผ์žโ€๋Š” ๋…ผ์ง€๋Š” ํ†ต์ œ ์‹คํ—˜์œผ๋กœ ์ž˜ ๋’ท๋ฐ›์นจ๋˜๊ณ , ์–ด๋–ค dynamic retargeting ๋ฐฉ๋ฒ•์—๋„ ๋ถ™์ผ ์ˆ˜ ์žˆ๋Š” reference ๊ณต๊ธ‰์ž๋ผ๋Š” ์œ„์น˜๋Š” ๋ช…ํ™•ํ•˜๋‹ค. ๋‚จ์€ ๊ณผ์ œ๋Š” ์ €์ž ์Šค์Šค๋กœ ๊ผฝ์€ ๋Œ€๋กœ dynamic ๋‹จ๊ณ„์˜ ๊ณผ์ œ๋ณ„ ์žฌํ•™์Šต์„ ์—†์• ๋Š” ๊ฒƒ์ด๊ณ , kinematic reference๊ฐ€ ๋‹ด์ง€ ๋ชปํ•˜๋Š” ์šด๋™๋Ÿ‰ยทํž˜์„ ์–ด๋””์„œ ๊ฐ€์ ธ์˜ฌ์ง€๋‹ค.

์ด ๊ธ€์— ๋Œ€ํ•ด

  • ์ฝ์€ ๋ฒ”์œ„: arXiv v1(2026-10-19)์˜ HTML ๋ณธ๋ฌธ๊ณผ PDF๋ฅผ ํ•จ๊ป˜ ์ฝ์—ˆ๊ณ , ํ‘œ์˜ ์ˆ˜์น˜์™€ ๋ณธ๋ฌธยท๋ถ€๋ก ๋ถˆ์ผ์น˜ ํ•ญ๋ชฉ์€ PDF์—์„œ ๋‹ค์‹œ ํ™•์ธํ–ˆ๋‹ค. ๋ถ€๋ก(S1~S5)์„ ํฌํ•จํ•œ๋‹ค. ๊ทธ๋ฆผ 1~4๋Š” arXiv HTML์˜ ์›๋ณธ ๊ทธ๋ฆผ์ด๊ณ , ๋…ผ๋ฌธ์˜ ๊ทธ๋ฆผ์€ ์ด 4์žฅ์ด ์ „๋ถ€๋‹ค. ์‹ค๊ธฐ ์˜์ƒ 4๊ฐœ๋Š” ํ”„๋กœ์ ํŠธ ํŽ˜์ด์ง€์˜ ์›๋ณธ mp4(18~35 MB)๋ฅผ ํ•ด์ƒ๋„๋งŒ ์ค„์—ฌ ์‹ค์—ˆ๊ณ , ํ”„๋กœ์ ํŠธ ํŽ˜์ด์ง€์˜ ์ „์ฒด ์†Œ๊ฐœ ์˜์ƒ(์•ฝ 51 MB)์€ ํ—ค๋”์— ๋งํฌ๋งŒ ๊ฑธ์—ˆ๋‹ค.
  • ๊ณต๊ฐœ ๋ฒ”์œ„: ์ฝ”๋“œ๋Š” ๊ณต๊ฐœ๋˜์ง€ ์•Š์•˜๋‹ค. GitHub ์ €์žฅ์†Œ์—๋Š” README, CITATION.cff, ๋กœ๊ณ ยท๋ฐฉ๋ฒ• ๊ทธ๋ฆผ๋งŒ ์žˆ๊ณ , README๋Š” โ€œResearch code and checkpoints are not released yetโ€๋ผ๊ณ  ์ ๊ณ  ์žˆ๋‹ค. ๊ทธ๋ž˜์„œ ์ด ๊ธ€์˜ ๊ตฌํ˜„ ์„ธ๋ถ€๋Š” ๋…ผ๋ฌธ ์„œ์ˆ  ๋ฒ”์œ„๋กœ ํ•œ์ •ํ–ˆ๊ณ , ๋ณธ๋ฌธยท๋ถ€๋ก ๋ถˆ์ผ์น˜๋Š” ์–ด๋А ์ชฝ์ด ๋งž๋Š”์ง€ ํŒ์ •ํ•˜์ง€ ์•Š์•˜๋‹ค. ๋ชจ๋“  ์„ฑ๊ณต๋ฅ ยท์ง€ํ‘œ๋Š” ๋…ผ๋ฌธ์ด ๋ณด๊ณ ํ•œ ๊ฐ’์ด๋ฉฐ ์ด ๋ธ”๋กœ๊ทธ๊ฐ€ ์žฌํ˜„ํ•œ ๊ฒƒ์ด ์•„๋‹ˆ๋‹ค.
  • ํ‘œ๊ธฐ: โ€œํ•„์ž ๊ณ„์‚ฐยท๊ฒ€์‚ฐโ€์€ ๋…ผ๋ฌธ ํ‘œ์˜ ๊ฐ’์œผ๋กœ ์ง์ ‘ ๊ณ„์‚ฐํ•œ ๊ฐ’(๋‹จ๊ณ„๋ณ„ ์ฆ๋ถ„ ๋ถ„ํ•ด, ๋น„์œจ ๊ฒ€์‚ฐ, Fisher exact p์™€ Wilson ๊ตฌ๊ฐ„)์ด๋‹ค. FisherยทWilson ๊ณ„์‚ฐ์€ ์žฌ์ƒ์„ ๋…๋ฆฝ ์‹œํ–‰์œผ๋กœ ๊ฐ€์ •ํ•œ ๊ฒƒ์ด๋ผ ์‹ค์ œ ๋ถˆํ™•์‹ค์„ฑ์„ ๊ณผ์†Œํ‰๊ฐ€ํ•œ๋‹ค. โ€œํ•„์ž์˜ ํ•ด์„โ€์€ ๋…ผ๋ฌธ์ด ์ง์ ‘ ์ฃผ์žฅํ•˜์ง€ ์•Š์€ ํ•ด์„์ด๋‹ค. ๋ฒ ์ด์Šค๋ผ์ธ ์› ๋…ผ๋ฌธ ๊ฐ’๊ณผ์˜ ๋Œ€์กฐ์—๋Š” ์ด ๋ธ”๋กœ๊ทธ์˜ ๊ธฐ์กด ๋ฆฌ๋ทฐ๋ฅผ ์ผ๋‹ค.

Copyright 2026, JungYeon Lee