Paprika is dried, ground peppers, the stuff we get in the UK is mild but hot variations do exist. It's most likely a blind of different sweet peppers but the hot stuff will have chilli in there too. .
Chilli powder is a blend of different dried, powdered chillis, sometimes with other spices added in.
Cayenne pepper is the dried, ground powder from the Cayenne chilli pepper.
Taste-wise, paprika I find about useful as a red colouring but I can't really notice any appreciable flavour. I have tried a smoky paprika that was quite flavoursome.
Chilli powder obviously varies in flavour depending on the blend.
Cayenne pepper is hot, clean and strong in flavour.