What makes a "legendary" legendary?

3ventic's answer is the best one, but it's also worth noting that Legendaries are almost always better than an equivalent card that isn't legendary. By "equivalent" I mean class specific vs class specific, neutral vs neutral, of the same or similar mana. (Assuming there is a comparable card.)

For example: Nat Pangle may not be as effective as a Mana Tide Totem (50% vs 100%, draw at the start of your turn instead of the end), but he's better than any other neutral card-draw card. So there's less reason for a Shaman to take him, but for non-Shaman he can be a great card.

While 3ventic most definitely has the "correct" answer, there's a whole different side to this (at least, in my mind). A legendary is "legendary" because Blizzard said it was. By Blizzard telling us a card is legendary, that card now carries a certain amount of "clout" which brings the whole notion of psychology into the mix and that's immensely powerful.

For example, Nat Pagle may not be the best Legendary out there but Blizzard said he was legendary and therefore he scares your opponent. Your opponent is now going to blow something to get rid of Nat Pagle. This effectively gives Nat Pagle a pretty strong "soft" taunt.. and, again, that's super powerful. Even if your opponent ignores Nat, you still win because you will start to have the card advantage.

In short, a legendary is a legendary because they represent a unique character in the Hearthstone world. They are also legendary because Blizzard told us they were, and being legendary carries "clout" which makes them targets for spells, clears, etc which means one less spell/clear/etc for your bigger and better cards.

Legendary cards can provide the biggest single card tempo swing for their mana cost -- they can be game changing cards if played correctly:

Ragnaros can do direct (though random) damage without taking damage.

Alexstraza can set a hero's health to 15 AND has the stats of an epic giant.

Tirion Fording can deal a significant amount of damage for only 8 mana (minion + weapon) AND has taunt and divine shield.

Jaraxxus can bring the health of the hero (usually Warlock unless you're a Priest and thoughtsteal / mind vision him) back up to 15 AND you get 6/6 Infernals for only 2 mana as your hero ability AND a 3/8 weapon that you can use right away. If you get and hold on to the coin, you can summon an Infernal that turn too.

Harrison Jones can generate card draw AND destroy your opponent's weapon, and still provides a 5/4 for 5 mana if your opponent is't a weapon class hero (not ideal, but not terrible either)

Cairne Bloodhoof is worth 4/10 for 6 mana.

There aren't too many epic cards that can provide this much tempo swing in a single card, though Faceless Manipulator + Big Game Hunter are usually the most ideal answer to the larger attack legendary cards. Those with special abilities can be silenced, but your opponent may not have a silence card handy right away.

Not all legendaries are, well, all that legendary. Several provide net equality of tempo but can be played sooner rather than later and can do some damage quickly, albeit with a cost. The Beast and King Mukla come to mind as examples.

Tinkmaster and Gelbin are pretty much joke cards, and you'd probably be better off dusting them if you got them in a pack unless you wanted to make your opponent fall off his or her chair laughing.