If they weren't very big when this happened, I'd guess that they weren't actually viable fruits, i.e. they hadn't been pollinated properly and so they wouldn't have amounted to anything anyway. I've noticed this a few times, and just pull the fruit off and chuck it. Nothing to worry about if the rest of the plant is healthy.