A little history:
In medieval (and early modern) times, people accused of certain faults (like adultery or other moral “trespasses”) were made to wear white garments when attending mass, as a sign of penance or public shame. The white was intended ironically: a symbol of “purity” to contrast with their moral “taint.”
The expression used to be “être dans de beaux draps blancs”, literally “to be in fine white clothes.” That phrase meant you were publicly exposed, judged, in disgrace, or in an unenviable situation.