Of course, the cop-out, generic answer is always, "It depends on the person." It also depends on the "thing."
Let's start by talking about a couple of things, and then we can continue the discussion in the comments if you want to talk about other Jewish items!
Jewelry: Personally, I think anyone can wear Jewish jewelry at any stage. Heck, even people who have no intention of ever becoming Jewish can show an appreciation for Judaism with style! Of course, you need to be prepared for someone questioning you about it, based on the presumption that you must be Jewish if you're wearing Jewish "stuff." Therefore, don't wear the jewelry unless you're ready to deal with the conversations it can spark. And if you're not, you could still wear a necklace under your shirt or something similar that keeps it hidden from public view.
Tzitzit (specially knotted fringes usually only worn by orthodox men, though not all orthodox men): Your beit din will probably have an opinion on this. And being female, I don't know a lot about them or the laws of tzitzis. I have heard that they may be prohibited until after a conversion is complete, but that isn't verified. All that said, from an outsider perspective, it seems like tzitzit could be worn very early in the conversion process if a person finds meaning in them. However, I would suggest keeping them tucked in (a whole other topic in itself, from what I understand) because seeing the tzitzit could cause others to make assumptions about your Jewish status and rely on that for the sake of a mitzvah (particularly in making a minyan). I welcome any help in the comments section!
Kippah (Yarmulke): You can basically just re-read everything I just wrote about tzitzit. However, my understanding is that batei din have much more standardized protocols on converts wearing kippot outside of shul simply because they all require a male convert to be wearing a headcovering full-time long before a conversion is finalized. Remember to be very careful about others making assumptions about your status and relying on that assumption for the sake of a mitzvah. You should not be wearing a kippah until you are comfortable with and able to correct someone's assumption.
Of course, because of the attention it can cause, full-time kippah wearing is probably not recommended until you're relatively far along in the process. Let's not kid around; wearing a kippah can be dangerous. It's like having the bat signal for anti-Semites on your head. Therefore, I think a lot of the analysis on when to begin wearing a kippah has to do with personal comfort with the concept. However, similar to the hidden jewelry suggestion, my understanding is that many men begin by wearing a "regular" hat of some kind more often, then wearing those hats full-time, and then the hat with a kippah underneath before venturing out with only the kippah. In other words, working your way to full-time kippah wearing gradually and with attention to the place and people. Of course, a hat by itself is fine as a headcovering halachicly, from what I understand (though community standards may dictate more strictly). Remember that you're not tied to only wearing a kippah every moment of every day. (Unless that's how your community rolls.)
Tallit: My understand is that in most American communities, only married men wear a tallis except for a single man leading the davening. However, other communities begin wearing a tallis at the age of bar mitzvah. Check with your beit din, but commenters below and comments from others tell me that this is generally something that will wait until after conversion.
Tefillin: Sorry, guys, this one waits until conversion. However, like a bar mitzvah, you may be asked to practice with a tefillin set without the parchment inside for a couple of months leading up to your conversion.
Women covering their hair: Similar to the kippah conversation, this should probably be implemented gradually. Also, you will want to be prepared for the conversations that can be provoked by a full haircovering like a snood, sheitel, or tichel (as opposed your run-of-the-mill hat). Also be careful that someone doesn't make assumptions about your status that may affect the fulfillment of a mitzvah. I can't think of any examples, but maybe you can. Now when to begin wearing? If you're not married, don't. If you are married, then begin covering whenever you feel comfortable doing so. Many start with only covering inside shul and go forward from there. Most women I've spoken to/heard from about haircovering later in marriage say that they finally reached a day when they "needed" to cover their hair.
I better see some active commenting on this topic, especially from you men!