There is few possibilities and without photo impossible to say what they might bee...could be even honey bees, which would be bad news..but most likely explanation what your sitiation is, they sounds like masonry bees
They don't do any damage to your house..just using any weak points in the masonry/brick work to gain access into cavity in the wall and make their nest there.....again...they don't do any damage! For their nesting material they use what is loose, available and suitable for their purposes.
They are VERY well behaved bees....if you don't bother them, they don't bother you and if you talk nicely to them...they will let you stroke them...
Masonry bees don't make big colonies so there is no worry about increasing population.
I would just sit back...look at them in their work and enjoy the summer scenery...count yourself lucky to have such a well behaved neighbours
Come end of summer/autumn...and they will disappear again.