The problem is that wasps are the gardener's friend early in the season, when they do indeed feast on harmful pests, but our enemy later on, when the fruit ripens and they spoil so much produce.
Beekeepers don't like wasps either, as they sometimes attack and overwhelm whole hives, eating the bee larvae.