A larger tuber will have more reserves to grow away in poor conditions or after rabbit damage in Spring, and a very small tuber may dry out more quickly if they are badly stored over winter, but being clones it's probably fine to grow from middle sized tubers, and eat the biggest. However, as you imply, tubers from adjoining plants can get mixed up in the soil, so growing from the largest tubers does have the advantage of absolutely ensuring that you are propagating from plants that make big tubers.