I think you just have to let them get on with it even if you don't like it. If, on the other hand they become drunk and abusive or aggressive, time to have them warned and, ultimately removed. Same applies if their plots are neglected.
We have quite a lot of social drinking going on but it never seems to get out of hand. I quite often provide the wherewithal - plum, rhubarb, blackcurrant or elderflower wines - but don't participate myself since I have to drive home.