Yes, you have to be able to feel the beans inside. Usually you can't pick all beans from each plant, if you don't feel beans inside, leave it for harvesting later. You have to check often because they mature fast once the harvesting has started.
Open a few pods you think are ready just to get the hang of feeling when they are just right. SandersJ is right, leaving them too long is such a shame.
I harvested four times from my patch and dug it over last week. Endive and lettuce are planted on it now.
