If you leave them they will probably produce small asparagus plants next year, we have lots of them under the main female plants. You could then dig them up when they're bigger and replant somewhere else.
It's recommended to remove the berries, I never bother (or rather don't have the time) and it doesn't affect the yield at all.