We tried to make magic exciting in Sertorius. The spells allow you to do all kinds of things, from splitting your body into duplicates to creating a copy of another person or calling down a massive wave of fire. When characters attain enough followers, they can even gain the power to burn an entire city or disrupt the flow of time. And the system allows you to keep casting spells. What reigns them in, is the process of becoming a Grim.
Every spell in Sertorius can be used in one of two ways, for a weak minor effect or a more potent display. When you use a spell at full potency you expose yourself to dangerous energies that can warp your body and mind over time. This is represented by Grim Points. You gain a number of Grim Points whenever you cast a spell at full potency. If you gain enough, your mind, then your body, begin to transform. Initially this is subtle, perhaps your arms turn scaly or you start having paranoid thoughts. If you persist without recovering, then things intensify and you can turn into a monster called a Grim Beast. You can also turn into something called a Grim.
Grim Beasts are powerful creatures, and treated as NPCs and monsters. When you turn into a Grim Beast, you cease to be a player character. They come in a variety of forms, but are mainly classic threats from traditional fantasy and myth. Some of the more common Grim Beasts are minotaurs, liches and banshees. But there are others like the dybuk that are less well known. While these bear the names and some of the characteristics of the source material, we also took great liberties. Minotaurs for example look like normal people most of the time and transform into giant flesh eating bull hybrids when they get angry.
Those who turn into Grims fuse with the land around them, becoming a sentient environmental presence. The specific form this takes varies from case to case, but one example is Karima head of the great library in Shahr (called The Library of Karima in her honor). Hundreds of years ago, Shahr, a halfling coastal empire, was attacked by Gru who sacked the capital and entered the library. Karima used her magic to protect their remaining scrolls from the attackers and in doing so was overwhelmed by her own magic, becoming one with the structure. When she desires, Karima appears through the stone and has the power to shape the rooms and walls of the building. Though primarily benevolent, she has been known to trap great minds in sealed off chambers, forcing them to write until they die. Karima is one of a handful of "pleasant Grims", who provide some benefit to the local population. Most of the others far more sinister, bringing harm to those who trespass into their territory.
We found in play tests that this was a highly effective mechanism for keeping magic in check, while allowing it to remain a mysterious and powerful force.
But Grims and Grim Beasts serve an additional purpose in the game: adventure. Defeating Grim Beasts is no easy task, defeating a Grim even more difficult. Taking on either is an adventure in itself. When Grims appear suddenly you can be sure the locals want to deal with the problem and the first people they will call upon are Sertori.
Every GM treats these a bit differently but I drew on my experience running Ravenloft and treated them as monster hunts or mysteries and ghost adventures in need of solving. The players were either hunting down and eliminating the threat, or they were learning about the threat in order to confront it or lay it to rest.