The Wrong Policy Could Drive You Crazy!