I'm really sorry to hear that smg. Your first step should be to find out if your site is on Green Land. If Green Land your local council can't do it. But think it might not be green land, so you should find out from the local council under what terms the land was made over to allotments in the first place.
Where I am, we had the same trouble, but I fought the council and won. So, all is not lost but you need to take action fast, or the houses will be up before you know it.
Good luck.
Chloe.