For the defensive bonus, couldnt you script in on the action of defense to add a state that adds 5% defense and remove them when attacking?
As for the damage calculation I'm pretty sure you can accomplish what you want in Game_Battler_3 script. Line 42 starts the defining of Normal attack damage.
# Clear critical flag
self.critical = false
# First hit detection
hit_result = (rand(100) < attacker.hit)
# If hit occurs
if hit_result == true
# Calculate basic damage
atk = [attacker.atk - self.pdef / 2, 0].max
self.damage = atk * (20 + attacker.str) / 20
# Element correction
self.damage *= elements_correct(attacker.element_set)
self.damage /= 100
# If damage value is strictly positive
if self.damage > 0
# Critical correction
if rand(100) < 4 * attacker.dex / self.agi
self.damage *= 2
self.critical = true
# Guard correction
self.damage /= 2
I haven't programmed with RMXP much but the basics would be to put something like this in after if hit_result == true
if attacker.class(#) == true
elsif attack.class(#) == true
I believe monsters are given class 0 but I dont know 100%.
Once again I havent scripted in this language much at all so I'm only guessing they will help you out.