The set-up: A hot water dispenser like you see in hotels and at conferences. It is plugged in, filled up with water, and turned on before Shabbat begins.
The problem: When water is too hot, it will "cook" the tea leaves in your average tea bag. This temperature, yad soledat bo, is hot enough to induce cooking, and cooking is forbidden on Shabbat. The temperature itself is disputed, but if you can't hold your hand under it, it has definitely passed that threshold. This temperature matters, among other things, when determining whether a utensil has accidentally become meat or dairy during a mix-up while cooking or doing the dishes. When the water is hot enough, the "taste" of meat or dairy can transfer because the level of "cooking" has been reached.
I know of two answers (and I believe they're the only two), but if you know of more options, please note them in the comments!
The traditional answer: Tea essence. You brew the tea leaves (or tea bags) before Shabbat in a concentrated form. Then you mix the concentrated tea with more water to make it drinkable. According to at least some (I don't know if it's the majority opinion), the tea essence should be placed in the cup first, and then diluted. This avoids "coloring" the clear water on Shabbat.
The modern answer according to R' Moshe Feinstein: Tea bags can be used on Shabbat, so long as the water is transferred from the hot water dispenser to one cup and then poured into the drinking cup. Put another way, there is an intermediate cup between the water heater and the cup you want to drink out of. Put a third way, you need a hot water heater and two cups. You will not drink out of the intermediate cup because the water is still too hot and presumed to be at a "cooking" temperature. The water is presumed to lose enough heat via the intermediate vessel to lower it below the cooking level. If you've never tested it, it really does lower the temperature significantly. Once the water is in the second cup (the "third vessel"), you may put in the tea bag. At least technically (I don't know how followed it is), you should not remove the tea bag from the cup (like you normally would do when the tea has steeped enough). That would be removing bad from good and violate the laws of selecting. I believe there is a way to accomplish this, but I don't know. Perhaps it matters if you never stop holding the string? However, if the tea bag being in the cup really bothers you, you can avoid this issue by pouring your tea (the good) into a third cup, thus selecting the good from the bad.
The yom tov adjustment: Since you may cook on yontif for the yom tov, the intermediate cup is not necessary if you plan to drink the tea on the same day. In that case, you pour the water from the hot water heater into a cup and put the tea bag into that cup, like you normally would.
I believe this is accurate, but if there are (correct) corrections, I will update this page. So it might be in your interest to check this page again in a day or two. If corrections are coming, they tend to come within the day.