Too often DMs complain that monsters are too weak, spells and
magic too strong, or players too clever. What is actually stated in
most such cases is that the DM is a Dungeon Milquetoast rather than
Master. Players plan and cooperate, so naturally they tend to utilize
all their strengths and abilities collectively, thus. defeating monsters
and DM alike. The classic, unthinking reaction of DMs so abused is to
create the “Monty Haul” campaign— “I really wanted my players to
walk all over everything, even me!“—or the “Killer Kampaign”
game— “It’s them or me, and none of those bastards will survive to
tell the tale!”

The game is supposed to be overseen by a disinterested judge,
referee, and moderator. The DM creates the milieu, sets the stage,
and then observes and relates information. The important part usually overlooked is the active (but still impartial) role of the DM as the
force behind each and every creature encountered not otherwise
represented by a player. Every NPC and monster encountered is a
lifeless shell until filled by the DM’s vital activity. Too often the vitality
given is cursory, stiff, unthinking, and lackluster. The NPCs and
monsters are then easy marks or else so powerful as to be impervious
to PC attacks and instant death-dealers in return. This is unfair to
players, of course, for both cheat them of the exciting suspense of
the game. It also cheats the DM out of the most enjoyable aspect of
play-active role assumption. If each DM knew his or her monster
and NPCs, then the game would begin to resemble what it is meant
to be. 

Intelligence is the key factor in determining what course a monster (whether singular or plural) will follow. Mindless creatures act
mindlessly. If they are hungry, they attack until food is obtained and

hunger is satisfied. Pain receptors are probably so inefficient as to be
immaterial. Attack until death is a likely course. Similarly, stupid
creatures will be likely to attack with total confidence. Let us consider
certain animals, pack and otherwise, as cunning. Wolves, wolverines, giant weasels, etc. will not always fight until dead. It is up to the
DM to use his or her intelligence to determine how such monsters will
react in any given situation.
The other key to monster behavior is alignment. Chaotic creatures do not follow orders well, but lawful ones do. Chaotic creatures
tend towards self far more than lawful ones do. This is not to imply
that chaotic humanoids such as gnolls, for instance, will not operate
as a unit. Training and/or self-preservation encourage such cooperation. It is to imply that a frost giant might well hurl a boulder
into a melee involving his ogre servants and attacking humans.
Monsters able to use attack forms of superior nature will tend to
do so if they have sufficient intelligence, and if opportunity permits.
Missile weapons are a good example. Not only will javelins, spears,
etc. be thrown, but bolts and arrows loosed. Even oil will be hurled
by monsters aware of the effects of fire. All this can then be applied to
more powerful monsters. Potions, scrolls, rings, wands, and so on
will certainly be employed by monsters possessing them—assuming
ability, of course. Trained, intelligent, coordinated attack and defense modes are stressed in many works, even in modules. Still, DMs
blithely ignore this. Their monsters stand for the slaughter and the
message goes for naught.
When you establish wilderness encounter areas or dungeon
complexes, place monsters with an eye towards their alignment and
organization. Thus, all goblins in an area will probably be allied and
cooperative. Gnolls need not be organized and groups will probably
be competitive, if not antagonistic. The goblins and gnolls might be
mutually hostile-unless they see a serious threat from attacking
adventurers. In both groups there will be leaders and some defense
plans. 

Consider a room with two doors. If the defenders can use the
second door to send parts of their force to strike the attacking party in
the rear, they will certainly do so if they are of even “low” intelligence. Nets, pits, rocks to hurl, and other devices are all logical parts
of a planned defense possible to all humanoid and similar monsters.
In short, as DM you should plan for the monsters according to their
mentality and then play them according to plan. 

Of course, players will still overcome monsters, but not “on the
cheap” anymore. Players who are rash will actually be defeated
fairly often. This should put more challenge in the campaign and
make the whole more fun and interesting. The DM is the one who
will benefit the most, both from the enjoyment of playing so many
roles intently and from the overall rise in the level of his or her
campaign.
Another useful and reasonable option for DMs is the combining
of monsters with guard beasts or monsters or with other intelligent
monsters. Such cooperative groups are ‘dealt with in the Monster
Manual and shown in most modules. Despite this, they are too
seldom used. When played, the DM usually gives little thought to the
actual strengths of such associations. Lowly kobolds with even one
giant weasel are far more powerful than without such a beast.
Whether the weasel is kept as a guard at the entrance to their lair or
as an unexpected force to be released upon attackers, it is a multidice monster, so attackers can strike but once each against it. Meanwhile the kobolds can organize, fire missiles, etc 

A single monster or group of like, weak monsters is/are far more
vulnerable than a combined force. In terms of medieval warfare, a
balanced force needs missile, foot, and mounted (mobile) arms
which act in concert in order to be effective on the field of battle. The
exact role of each arm might vary from situation to situation, but the
principle is the same. Terrain might negate the usefulness of one
arm—or else mitigate the lack of it. Goblins can field a balanced
force of missile troops, worgs and worg-mounted individuals, and
regular infantry to form the solid core around which to rally and
reform. They might actually be allied with nearby kobolds (thus
fielding more troops as unexpected reinforcement, either way) or
have a small group of bugbears living with them (as protectors in

return for free lodgings and food). (See Dungeon Masters Guide,
pages 104-106.) <Monsters and Organization> <Use of Non-Human Troops>

Another combination of monsters is possible where the weaker is
aware of and intelligent enough to use the stronger without the
latter’s knowledge or consent. For instance, an otyugh dwells in a
nearby garbage chamber. The monsters—orcs, let us suppose—
lead attackers to this place, setting up their main defense on the far
side. In order to get to the orcs, the attackers must contend with an
aroused otyugh, for if the quick passage of the orcs failed to make it
angry, the entry of the attackers surely will.
Whether dealing with humans, humanoids, or other monsters,
many will have active and aggressive offenses and defenses planned
and will carry them out. Powerful NPC-type monsters will have
associates and guards. Groups will act cooperatively. Other monsters and terrain will be used to advantage, for the monsters know
their own area. All this makes the whole game more meaningful for
players and DMs alike. It is time to master your campaign and stop
selling monsters—and yourself—short. You’ll not only have more
fun, but you’ll surely gain the respect of your players too! Smart
monsters are certainly one of the main ingredients of a successful
campaign.