Damion Searls, writing for Paris Review, finds the link between language and the soon-to-be-extinct penny:
One thing we’ll lose, when the penny eventually goes the inevitable way of the half cent and the Canadian penny (extinct as of 2012), is the last possible link between our language of money and the everyday physical world.
A quarter is a fourth of a dollar, a dime a tenth (Old French dîme, Latin decima), a cent a hundredth or one percent—all math. Anyway, a cent is not a piece of money: a U.S. penny is technically a cent or one-cent coin, but in spoken language, a cent is a value and a penny is a coin. We offer someone our two cents, not two pennies; pennies can clink in your pocket, cents can’t. (When Americans adopted the British term penny in 1793, they took over the distinction, too: in England between pennies and pence.)
As for penny, its etymology is uncertain, though the ending implies a Germanic origin—the word used to be penning, with an -ing, like shilling and farthing, instead of a -y. The root may be Pfand, which turned into the English word pawn meaning “a pledge or token”: in that case, penny basically just means money. Or it may derive from the German Pfanne, “pan,” the round metal thing that you cook in. My head says it’s pawn: the pan pun sounds like classic folk etymology that somebody simply made up. But my heart belongs to Pfanne: surely the original coin goes back to some concrete reality, an object of actual use.
That said, the American penny isn't going anywhere. It's going to keep coming back like a bad...yeah.