A focused carbon dioxide laser beam is used to microstructure fibers that have already been narrowed by conventional fiber tapering. We describe three new miniature devices made with this technique: a fused fiber microcoupler with an interaction length of 200 mum, a long-period grating made from a periodic chain of microtapers, and a new type of prolate whispering-gallery mode microcavity.