Abilities are spells you can learn using a scroll once you are high enough a level. Monsters can also know a few types of abilities as well. Using a spell costs you AP which can be recovered by standing still when not engaged in combat or using items such as a Replenish or Vitaliser.
Although having limited use based on your AP, spells can do considerably more damage to enemies than melee, this can also be used to 'safe' enemies by casting a spell then running away far enough for your gauge to fill, then rinse and repeat. This can be used to kill bosses who require a much higher combat level to kill.