Screeching rear brake

Isn't it the way that you think you've got the situation under control and a problem appears out of nowhere.  The rear brake on my 61' Navigator has started screeching when I press the pedal.  I've had the wheel off and I've checked the mechanism in the hub and that all seems to be working, the brake shoes seem to have plenty of lining and there is no rust in the brake drum.  I even tried roughing up the shoe linings to make the surface less glossy but the problem persists.  I have noticed that it is worse when I start riding the bike and starts to ease the longer it is in use.  It is also worse when I am coming to a halt, say at traffic lights.  The brake itself works perfectly well so it is not an issue of safety, just pure annoyance.  Does anyone have any ideas as I'm at a dead loss?

Something is loose, or needs a bit of packing possibly the brake plate.