For respectable people, social prestige is very important. The particular
guṇas
(modes of nature) of warriors make respect and honor especially important for them. For them, dishonor is worse than death. Shree Krishna reminds Arjun of this, so that if he is not inspired by superior knowledge, he may at least be inspired by inferior knowledge.
Many societies in the world observe the norm that a warrior who runs away from the battlefield out of cowardice is ostracized from society. That could be the extent of dishonor inflicted upon Arjun if he avoided his duty.