Metal tines, laid out like a kalimba, act as extended electrodes against a touchscreen. A tap or a
swipe across them still registers, but the hand never crosses the display. No battery, no pairing,
nothing to charge. The work became a CHI 2026 workshop paper.
SwiftCapacitive sensingMobile interaction
ProblemOcclusion is an input error, not just a view one
HardwareLinear electrodes on insulated PET, unpowered
SoftwareSwift / UIKit gesture prototype
PrincipleExtending the capacitive contact off-screen
Occlusion is an input problem
The obvious cost of a finger on glass is that it hides what is underneath. The less obvious one is that
it corrupts the input itself: a finger covering a key disturbs capacitive sensing, the detected centroid
shifts, and the system resolves to the wrong target. That is why bucket comes out as
bucked — k and l are adjacent, and the contact patch straddles both. Treating this as a display
problem leads to bigger targets. Treating it as an input problem leads to moving the contact.
Encoding direction in the electrode geometry
A capacitive screen needs a fingertip-sized conductor on its surface, so the accessory supplies one —
just not the finger. Several linear electrodes contact the glass while the hand rests off to the side,
and their spacing and width do the work of encoding a gesture: sliding across tines of one pitch reads
as a vertical swipe, across another as horizontal. The narrow segment sits under the finger, the wider
one against the screen, with an insulating layer keeping the two touch surfaces from shorting into a
single ambiguous blob.
How the geometry maps to gestures, and which segment goes where.
Disambiguating tap from drag, and the platform limit
The iOS prototype is written in Swift on UIKit. Gesture recognition classifies the incoming signal into
tap or swipe against a threshold, a cursor is drawn where the contact resolves, and a stored previous
position carries the cursor across gestures so a swipe resumes where the last one ended rather than
snapping back. The honest limit is the platform: a genuinely system-wide emulated pointer needs a
jailbroken device, so the prototype demonstrates the interaction inside its own application rather than
across iOS.
The software prototype running against the physical accessory.