Adversary Aware Continual Learning