Everyobody loves villains. They do things heroes cannot and usually manage to work in a solid one-liner in the process. If you like Die Hard, chances are Alan Rickman has something to do with it. If you enjoyed The Professional, you probably also got a kick out of Gary Oldman's use of the word "everyone". If you grew up watching classic horror movies as I did, then actors like Vincent Price and Peter Lorre likely made uncredited appearances in your greatest campaigns. Villains are wonderful. They frighten, fascinate and make us laugh all at the same time. My personal favorite is Anthony Hopkin's Hannibal Lector because he hits all three of these key points so brilliantly. Here is a simple list of principles to help bring some of that magic to the table. These are not exhaustive, but they should hold you in good stead if you consider them while designing your bad guys.
A GOOD VILLAIN IS A REAL THREAT
Villains aren't wimps. And even if they are they either have the resources or genius to be a true danger to the world. Your villain need not be unrealistic or unfair but he ought to have the power to ruin the player's day if he so chooses. That doesn't mean you give him every spell in the player's handbook or max out his core attributes. There should be some weak points to balance out his strengths. Give him one or two critical resources (wealth, connections, firepower, magic, etc).
GOOD VILLAINS DON'T NEED TO CHEAT
Don't misunderstand this heading; in-character villains cheat, lie and steal all the time. In terms of game mechanics however, don't cheat or fudge to keep your villain a challenge. If he is a solid character with well thought out resources and powers he should be able to sink or swim on his own without your intervention. Don't deprive the players of rightly earned victories because you want a villain in the game.
GOOD RECURRING VILLAINS EARN THEIR PLACE IN THE CAMPAIGN
Related to the last point, you can't force a recurring villain on players. If they kill him fair and sqaure, then he isn't a worthy adversary and needs to stay dead (barring special and unique backstories and campaign developments of course). If you have a recurring villain you like, but the players manage to put him six feet under, then retire him with dignity. Better to be remembered as the worthy foe they slayed, than as the annoying DMNPC they despise.
GOOD VILLAINS ARE PROACTIVE AND REACTIVE
Your villain shouldn't just sit in his lair or manor house waiting for PCs to arrive, he should be out and about plotting and planning. If the PCs disrupt his plans, he shouldn't just sit back and take it, he should try to get revenge or adapt a new plan of action. Good villains have life. They are dynamic forces in the campaign.
GOOD VILLAINS ARE COMPELLING AND REPUGNANT AT THE SAME TIME
Players should love to hate your villains. There needs to be things about them likeable or interesting and things that provoke dislike. Their good qualities can can make them an great source of role playing at the negotiating table while their bad qualities should make them monsters.
GOOD VILLAINS HAVE PERSONALITY
This one is important and it should come out through events and conversations in the game game, not through explanations. The old saying "show, don't tell' applies here. Really get to know your villain so you can play him effectively at the table. Normally the GM doesn't get a chance to devote much time to a single character. Well, villains are your chance to invest a little energy in background, personality, motives, etc.
GOOD VILLAINS SPEAK DIRECTLY TO THE CHARACTERS
When playing villains it is important for you, the GM, to speak in character. The moment the PCs rush into the evil mastermind's headquarters and he tries to convince them the folly of their ways is not the time for third person dialogue. Role play your villains.
GOOD VILLAINS ARE FUN
If you don't enjoy your villains, the players will notice, so design them to entertain yourself and your players. Give them qualities and flaws that intrigue you. Give them style and manners that you feel comfortable pulling off. If you are self conscious about your fake German accent, don't throw too many Nazi villains at your players. Play to your strengths as a GM.
GOOD VILLAINS HAVE WEAKNESSES AND FLAWS
A villain should have a weakness or deficiency. This could be physical, mental or spiritual in nature. Everything from drug addiction to a family curse. Whatever it is, it should be exploitable.
GOOD VILLAINS TAKE RISKS AND CROSS BUNDARIES
One of the reasons villains are compelling is they do things normal people can't or won't do. Take the movie scarface. In the restaurant scene Tony says to the diners "Who you looking at? You are all a bunch of &@&@@ $$$$$@. You know why? You don't have the guts to be what you want to be?" In his mind the thing that seperates Tony Montana from everyone else is the willingness to break a few rules and take the risks needed to get rich and powerful. This line is also an example displaying a villain's repugnance and magnetism all at once.
GOOD VILLAINS EVOKE STRONG FEELINGS
If your PCs hate the villain, they should really, truly hate him. If they distrust him, they should distruct him like nobody's business. Whether it is fear, love or any other emotion, your villain needs to generate strong feelings from the party. This gives life to a campaign and is fuel for adventure. Strong emotions toward an NPC mean the players will often go out of their way to destroy him and will help create lasting memories of your classic villain.