That would be my first guess too, David (assuming it is around leaves and buds rather than on old bark). If you tease apart the fluff you will eventually find aphids among it.
Based on an experience with them years ago at my Mum's place, my advice would be KILL THEM NOW! >:( They can spread like wildfire, promote fungal infection, and are hard to kill even if you try to squash them all by hand.
Our experience was that they were very difficult to get rid of using anything other than repeated treatment with fairly unpleasant systemics, probably because a) they're aphids and by the time you kill one it's probably had 100s/1000s of babies and b) the fluff protects them from any treatment which has to touch them (eg. soap) and hides them from predators.
