Your men have to trust you! Try to boost the army morale with any opponent by using one of the army morale tools (prophecy, strategic review, speach, propaganda).