photo © Ken Smith 2006
Passiflora are visited by many butterflies & their larvae but there is a special relationship between them and the Heliconius butterflies. See Passiflora defences to find out more. Ken Smith's site has some beautiful pictures of Heliconius here and also advises which host Passiflora each of them prefer.
If your Passiflora are being destroyed by butterfly caterpillars eating them pick some/all off by hand wearing suitable protection. Warning! While caterpillars are quite safe if you leave them alone, many caterpillars have urticating (irritating) hairs either by touch or inhalation. To avoid problems wear protective gloves, cover your arms and ideally also wear a mask and cover the eyes. This sounds a bit extreme but some allergic reactions to caterpillars can be quite severe. Rather than kill the caterpillars try to keep one or two large Passiflora that they can eat without you worrying. Remember no caterpillars = no beautiful butterflies!