Because he's mad at her for destroying Bryce's (possible) gambit? As in, not marrying Cailan (cause he died) and then not marrying Alistair to become queen (and thus help the Cousland family)... he might see her as a traitor for that. In the vein of, what Howe didn't manage to destroy, you did! Not to forget that Elissa seems to have been a prissy noble, who thought commoners beneath her notice (if I remember correctly) - and there is only one kind of family that produces those.