A form-fitting glove that the Twiddler attaches to (f.ex. magnetically) would be much better for me.
Something like that:
(source: http://www.daniclodedesign.com/thethirdthumb)
Well, maybe not magnetically. Simple velcro on the inner palm, and velcro along the back of the Twiddler, then can reposition to hearts content and rip off when not in use.
You could really freak people out with some of those.
@dkburrows If you wanted to freak someone out, you could try this as well: