Skip to content
Yilong LinYilong Lin
MagicPen: Enabling Dynamic Physical Scaffolding through Haptic Feedback for 3D Content Creation in VR with a Variable-length Stylus
UIST

MagicPen: Enabling Dynamic Physical Scaffolding through Haptic Feedback for 3D Content Creation in VR with a Variable-length Stylus

Zhicheng Wang, Tianze Xie, Yilong Lin, Xuesong Zhang, Daniele Giunchi, Eyal Ofek, Seungwoo Je

In Proceedings of UIST 2026.

Abstract

Stylus interaction is widely used for 3D content creation. In VR, it is typically mid-air, offering spatial freedom while lacking the grounded haptic feedback required for both shape perception and active geometric editing. To address this, we present MagicPen, a variable-length stylus that delivers grounded haptic feedback and enables active 3D editing through on-surface interaction. MagicPen operates through dynamic physical scaffolding, an interaction approach that adjusts the length to the interaction context: compliantly rendering geometry the user is actively creating, or resistively rendering predefined virtual geometry as a rigid constraint. We conducted three user studies comparing MagicPen with mid-air interaction in terms of accuracy, task load, and user preference: (1) single-axis positioning for depth precision; (2) freeform 3D planar drawing in compliant mode; and (3) drawing on rendered geometry in rigid mode. Results show MagicPen improves depth positioning. MagicPen better supports 3D content creation by enabling grounded shape perception.

Video

Keywords

Variable-length StylusGrounded Haptic FeedbackDynamic Physical Scaffolding3D Content CreationVirtual RealityStylus Interaction

Research Story

UIST 2026 · Haptics

A pen that changes its length to give VR drawing a surface to rest on

Written by Prof. Eyal Ofek · University of Birmingham, UK · Aug 2026

Read the original post
MagicPen is a variable-length haptic stylus for 3D content creation in VR. Pulling upward extends the stylus to reach points above the table surface; pushing down shortens the pen, creating the sensation that the tip is penetrating the table and reaching a point beneath it.

Try drawing a circle in the air, with nothing to rest your hand on. Now draw the same circle on a sheet of paper. The second attempt is easier, and the reason is not that your arm suddenly got steadier — it is that the paper is doing part of the work. A surface supports the hand, absorbs tremor, and lets the small muscles of the fingers take over from the large muscles of the shoulder. Fine motor control is something we borrow from the world, not something we generate on our own.

Stylus input in VR usually gives that surface up. Mid-air interaction buys spatial freedom — you can reach anywhere in the scene — but the hand floats, and with nothing to press against there is no grounded feedback to tell you where a virtual surface begins or ends. Depth, in particular, becomes guesswork: the eye alone is a poor judge of how far away a point is along the axis pointing away from you.

Borrowing the Desk You Already Have

MagicPen's answer is to stop treating the desk as an obstacle and start treating it as infrastructure. The stylus keeps its tip on the physical tabletop and changes its own length instead — a gear-and-rack mechanism driven by a DC motor extends and retracts the body by up to 70mm. The hand stays supported on a real surface the whole time, while the virtual tip travels above and below it.

The trick is that length is read as depth. Extending the pen means the virtual tip rises above the table; shortening it means the tip sinks below. Because the physical tip never leaves the desk, the user always has something to brace against — and because the length is under motor control, the system can either follow the user or push back.

The pen follows an existing geometry, stretching and shortening so that the user feels its tip lying on the virtual surface.

Two Ways to Scaffold

In the rigid mode, the geometry already exists and the pen serves it. As the user sweeps across a virtual hill or a wall, the motor drives the length so that the tip traces the surface — lengthening over a rise, shortening into a hollow. The user is not told where the surface is; they feel it, the same way a finger finds an edge in the dark.

The compliant mode inverts the relationship. Holding the pen's button, the user grabs a point on the surface and drags it — pulling a roofline up out of a footprint, or pressing a crater down into a hilltop. Here the length follows the hand rather than the model, and the geometry is what changes. This is the dynamic physical scaffolding the paper's title refers to: a scaffold that reshapes itself according to whether the user is reading the world or writing it.

Creating a building with MagicPen. (a) The user draws a closed contour on the virtual terrain while the pen follows the shape of the ground; (b) pressing the pen's button and pulling upward assigns the building model a new height above the ground.
Terrain sculpting. (a) The user drags the pen over the terrain to draw an area defining the base of a hill. (b) Selecting a point inside that area and pulling it up deforms the ground into a hill. (c) The user draws a circle on the hilltop; (d) pushing downward carves a depression into it.

Does the Surface Pay Off?

Three studies compared MagicPen against mid-air interaction on accuracy, task load, and preference: single-axis positioning to isolate depth precision, freeform planar drawing in the compliant mode, and drawing on rendered geometry in the rigid mode. Depth positioning is where the effect shows up most clearly — precisely the axis where vision alone struggles and a grounded contact has the most to add.

That framing is worth holding onto. A stylus that can only reach into empty space asks the hand to do a job it was never built for. Giving the tip somewhere to rest, and letting the pen absorb the difference in its own body, turns 3D content creation back into something closer to drawing — where the surface, not the user, keeps track of where the line is.