It's down to poor pollination. Two things have happened; the number of wild pollinators has declined due to pollution and loss of habitat, while the number of honeybees has declined due to the varroa mite. This is an Asian mite straight from the pits of hell, which jumped species from the Asian honeybee, which can cope with it, to ours, which can't, and spread worldwide. Since its arrival in the UK, it's virtually wiped out feral colonies (which probably made up 3/4 or so of the bee population), devastated beekeeping, and driven a lot of people out of it. It keeps developing resistance to the chemicals used to control it, which have very adverse effects on the bees anyway, and there's still no really satisfactory solution. I may (I still have to run the vital tests) have bees which are partially resistant, but not many people can even say that much.
I can't offer any simple solution to this. You could try putting a dozen beehives on your plot, but they're extremely efficient at ferreting out the best nectar and pollen sources around, and a few rows of broad beans might be completely ignored; mine usually are. If you could encourage wild pollinators which were active at the right time, that might work better.
