The source for hiddur mitzvah is Exodus/Shmot 15:2
"This is my God, and I will glorify Him."
Many mitzvot require a physical object, and hiddur mitzvah allows you to personalize the mitzvah to help invest yourself in it. By putting the extra time into choosing beautiful candlesticks, the most delicious-looking challah, or setting the Shabbat table with a pretty tablecloth, you are giving more attention and mindfulness to the mitzvah. It also helps involve all your senses in the mitzvah. In another way, you can have hiddur mitzvah by using heirloom Judaica.
Examples of items you can "beautify" to enhance the mitzvah:
Setting the Shabbat table with a pretty tablecloth and your best dishes
Dressing your best to honor Shabbat
Choosing the most beautiful/best-smelling etrog for Succot
Decorating the sukkah (Note that Chabad chassidim do not decorate the succah)
The coverings of the Torah scroll
Haggadot for Pesach
Matzah covers for Pesach
However, there is another side to the hiddur mitzvah tale: don't lose sight of the mitzvah in the quest to make it beautiful. It's possible to get carried away and forget the meaning you were trying to create. The rabbis knew this was a risk and suggested that you shouldn't spend more than 1/3 of the mitzvah to beautify it. If you want to see hiddur mitzvot gone wrong, watch the movie Ushpizin.