For those not aware, Attacks of Opportunity (AoOs) are free attacks earned when one character either disengages from, or moves around, another character in melee combat. Well, there are other times such a thing can occur, but this is the most common, and what I'm focusing on in this discussion.
AoOs have existed in D&D officially for a number of years, and somewhat more unofficially for years past that. Other games can have similar rules. Lately the debate has risen up again due to 5e, which doesn't currently have them but probably will soon. Neither proposal in 5e is effective to me, because it's either "nothing happens" or "roll extra dice that have almost no effect."
In fact, I've never really liked AoOs in most systems, because they don't solve the mechanical problem they are meant to address.
Let's do an experiment to illustrate my point.
Go get a US Marine. Give him a sword. Stick him in a hallway with 5 feet on either side of him.
Great. Now, go run past him.
After you put your innards back inside your body, let's examine what we've learned:
A trained warrior cannot be run past unless you first disable him or actively move him out of the way.
This is the point that AoOs are supposed to solve, i.e., making fighter types "sticky" so that they can't be ignored and run past. Because really, who would fight a durable, low DPS tank when a squishy, high DPS wizard is meteor swarming you?
AoOs do not solve this problem - at all. Who cares if the fighter gets a free attack on me as I move towards the wizard? I think I can take a 1d8+5 attack. What I can't take are those 20d6 meteors pelting me every round.
If I have 100 HP, why would I ever care at all about some free basic attack? The smart tactic in these games is to ignore the warrior, take the hit, and go get the squishies. As a GM I often wonder why players are so afraid of incurring AoOs when it is clearly hurting them to think that way.
But that is a disassociated line of reasoning. As we saw in our experiment, the expected and associated behavior would be that characters choosing to ignore the warrior should probably die.
The only way to make AoOs act as a solution to this, is to make them grossly fatal. In my opinion, that just sort of wastes everyone's time. Why not design rules to support associated behaviors, rather than with weird penalties or disassociated mechanics like AoOs?
In Phoenix, and I know this is a volatile subject, but one of the things I'm experimenting with is the concept of simply prohibiting fatal maneuvers. For example, a character can't fire a bow when threatened in melee because they would be cut to pieces while they tried. And yes, there is training or magic that can overcome that restriction, but breaking the rules via other rules is fun right?
One of my focuses in Phoenix is the speeding up of combat. And not via lame stuff like encouraging players to print out powers on playing cards, or having a sand timer at the table. Those are great and all, but I prefer that the flow of combat is so quick that it can't help but to feel faster. I also don't want to sacrifice fun combat decisions and tactics. I want to have my cake and eat it, too.
Along these lines, if a character in Phoenix is armed with a melee weapon, proficient in that weapon, and active (not unconscious, held, etc.), then that character exhibits a threatened zone equal to the reach of their weapon. As a general rule, enemies cannot move from one threatened hex, to another threatened hex. It's simply blocked, unless they spend their entire action to move one hex. This gives the defending warrior a choice to either move and block, or let them pass.
Aggressors can back off and try to go around, or they can try to shove the warrior out of the way, or whatever else they can dream up, but such ideas are reasonable & associated solutions and they should have a chance at working in the right situations.
Yes, there are exceptions, and yes, there are training options, spells, and more that can creatively deal with this block. The focus is on creating associated solutions, not banning tactics.
Threatened Zones aren't the only way to achieve associated results, but they are fast, fit with my design goals, and most importantly are fun in tactical combat. More playtesting will ultimately reveal their solvency.