I love foxgloves too. I live in a terrace with a small garden, but I let them seed as much as they want. In the autumn I go round and move the small plants to somewhere more convenient or dig them up and compost them. I have some steps leading up to my greenhouse and one grew up in a crack in the bottom one, but I didn't have the heart to move it I just treated it like a newel post!
Dead-head them before they have chance to set seed and in a few years you'll be rid of them.
