Just protect your jugular vein in case!!!! ::)
Only kidding, we had exactly the same problem and they scare the living daylights out of me too. My compost was too dry and so I keep it uncovered, wee on it (or at least in a bucket first) and turn the compost often or at least disturb it. They will move on honestly and don't worry they will NOT attack you!