Personally I add all the leftover bits of fish or game to the compost heap including heads, necks, innards, feathers, skin, fur, bones, carcasses etc... it's all organic matter & is therefore perfectly compostable. As long as it is buried in the pile not just dumped on top it does not produce unpleasant odours or attract vermin in my experience.