Why do cats lose fur on their head?

I have a cat that is almost two years that we got when he was a baby, he's a pretty short haired cat, he has gorgeous shiny black fur he looks real healthy and takes good care of grooming himself. He's naturally a tough aggressive cat I think his mother was a wild cat, so we recently started letting him outside hoping that will help him to be more mellow when he's in the house and it's seemed to work, but ever since when I hold him i'll notice some of his fur on his head looks displaced and tiny little patches will come out quite easily when i tug it and at the ends is what looks like some skin but i'm not quite sure. It's not on his body just his head. He doesn't seem stressed or in pain and when i pull out the hair he doesn't even seem to notice. I realize cats shed, and it's a possibility he's just getting into something outside but I want to make sure it's normal and there's nothing wrong with him. I'm considering taking him to the vet just to be safe but I was curious on what every ones opinion is. He has two bald spots above his eyes that have pretty much always been there and on his skin on one of the bald spots you can see a fresh scratch so I think he might be getting in fights with another cat that keeps hanging around our yard. But I want to make sure there's nothing medically wrong I thought it was weird it's just on his head.