A pen that follows the shape of the ground

Drawing on paper is generally easier and less tiring than drawing in mid-air. The table supports the hand, the surface provides resistance, and friction helps to stabilise fine movements. By contrast, creating three-dimensional content directly in space can introduce shaky strokes, depth errors and fatigue. Many designers therefore continue to model three-dimensional objects with two-dimensional tools such as mice and conventional styluses, despite the additional operations these tools require.

MagicPen explores whether a single device can combine the natural movement of drawing in three dimensions with the support and precision of working on a physical surface. The project brings together researchers from the Southern University of Science and Technology and the University of Birmingham. Their paper, MagicPen: Enabling Dynamic Physical Scaffolding through Haptic Feedback for 3D Content Creation in VR with a Variable-length Stylus, is due to appear at UIST 2026 in Detroit.

A stylus that changes length

MagicPen resembles an ordinary stylus, but a motor-driven gear-and-rack mechanism allows its body to extend or retract by as much as 70 mm. The tip remains in contact with a real desk or tablet while the user's hand rises and falls. The hand therefore retains physical support even when the corresponding virtual point lies above or below the tabletop.

Visual feedback in a headset or other display establishes the relationship between the physical stylus and the virtual geometry. As the pen extends, the user can perceive its virtual tip as rising above the desk. As it retracts, the tip can appear to penetrate the surface and reach a point beneath it.

Rigid and dynamic physical scaffolding

When the stylus changes length to follow an existing virtual surface, it provides what the researchers call rigid scaffolding. MagicPen can also reshape that surface. When the user holds a button, the stylus behaves as a dragging tool: pushing down retracts the pen and moves the selected point deeper, while pulling upward extends the pen and raises the point. The researchers describe this direct editing process as dynamic physical scaffolding.

Three-dimensional modelling

The team demonstrates the technique in a virtual-reality world-building application. To create a building, the user draws a footprint on rolling terrain while the stylus follows the shape of the ground. The user then selects the footprint, holds the pen's button and lifts the stylus to extrude the building to the desired height.

The same interaction supports terrain sculpting. A user can outline an area, select a point within it, and pull upward to form a hill. Drawing another contour on the hilltop and pushing downward can then create a depression. Moving directly between pulling and pushing allows the geometry to be edited through continuous hand movements without changing tools.

MagicPen extends the familiar interaction of a two-dimensional stylus into three dimensions while retaining support from the physical work surface. Its variable length connects movements above and below the tabletop with corresponding changes to virtual geometry.

Video

Adapted from Prof. Eyal Ofek's original research blog.