Everypony is afraid of something. Even a spirit of chaos like Discord. For him, it is a place from his past.

Back when Discord was a tyrant, ruling Equestria in a state of chaos and unhappiness, he had built himself a palace to stay in. This palace was originally designed by him to be chaotic enough to be a dream home for somepony like him. He stayed there for five of the fourty-two years of his rule, and then abandoned it, not giving any reason for it.

I decided to uncover the reason, if there even is one. I gained the courage to go up to him and ask him about it, and when I did, all of his silliness from before faded and was replaced with total fear and dread. Because he didn't feel comfortable discussing what had happened there, I decided to go there and find out myself. He gave me a very grave warning to stay away from there. I wish I'd listened.