Could you you let us know (warts & all) how you plant them out and what type of soil you have, plus what variety are they?
The reason I ask is; As I have mentioned before I don't firm mine in (I water them in) I think I get a better root system this way.
I also think 'variety' plays a large part in the success you get so may I suggest you buy a few varieties and grow a few of each in the same location then compare them.
I have now found that 'Maximus' is the variety best suited to my soil.
Having said that each year I try a 'new variety' i.e.
that is new to me Last season I tried 'Cascade' and found them to be much taller and the sprouts were smaller a bit like Tims.
I do this simply because seed merchants are notorious at removing the varieties,
that invariably you like, from their lists then you find yourself trying to find a replacement variety. At least this way I am partially prepared.
Plus I always buy F1 hybrids.
The two pictures below show my sprouts at Xmas and a trial I did with late Cauli & Calabrese.
I tongue in cheek sowed some summer variety seeds (Snow Prince & Samson) and planted them out in July/August to see if I would have some veg for Xmas.
Alas they succumbed to the frosts in December but I at least got a picking so if this so called 'global warming' means anything one of these years I will have home grown Cauli & Calabrese on my plate at Xmas.