A hammer and drift pin is many times more likely to bend the sheet metal the hinge is attached to. An air hammer is a better choice. Heat is going to do the trick getting the pin out almost every time. It's also guaranteed to fuck up the paint 100% of the time. Here's the thing to consider though, if you heat carefully and protect surrounding areas you just need to repaint the hinge. If you twerk a hinge or supporting structure you multiplied the amount of work exponentially. Hard to say what you have there, but the flange should not be on the bottom. Do not try to drive the knerls thru the hinge. If someone installed the pin bottom side up and you drive it the other way you'll really be hating life.