Assuming it 's mildew on the leaves, it's strange to find it after such a dry spell.
Clear out any dead stuff & weeds, to keep things airy. Mancozeb & the like is the RHS recommendation. Yes, can affect size.
Check HDRA's advice. If you are not a member, I could mail you a print-out. = Tim