If it were me, and I am no expert, I would remove the fading blossom so you can at least enjoy them whilst they are looking gorgeous. Also, if it were me, and I am probably going to be told now all the reasons why I shouldn't do this, but I would leave a couple to develop, I wouldn't think leaving one or two would slow the trees progress :) and you can have the pleasure of sampling its fruits in the late summer.