Stack the remaining flagstones on the border with your neighbour, exactly half over your plot and half over his.
If he still thinks they are his, then he won't need to tresspass on your plot to get them.
If he doesn't take them, after about a week paint the top one orange.
If he asks why did you paint 'my' flagstone orange, say that was the only colour you had available.
If he gets angry say sorry, and that you will paint it any colour he chooses.
If he says I didn't want it painted in the first place, say sorry and wash the colour off (use only washable poster paint).
Then ask him to move 'his' flagstone off your plot.
If he refuses, then tell him you are going to paint it orange again.
If he gets angry again, then don't paint it straight away, do it when he is not there.
If the next time you see him, he says you have painted 'my' flagstone orange again, and tells you to clean it off, say sorry and politely explain that you have used permanent paint this time by accident.
If he says that you will have to replace it, say ok, take the orange one away, and tell him to take the one underneath.
If he says it is mine anyway, ask him to take it off your plot.
If he refuses, tell him that you will paint it orange then (but do it when he is not there).
If the next time you see him........
I hope the above helps you Roxy, and you might end up with a load of orange flagstones.