While a few nullifications have occurred, the general problem is questioned conversions.
Nullified conversion: the conversion is void. It was invalid at the time the conversion was performed. The person was never halachically a Jew. Any children born of a woman since her "conversion" are not Jews (and grandchildren and so on down the line, if applicable). You may still argue that the conversion was valid, and some communities might even still accept it. Worse, you could be told your conversion is invalid/nullified by a rabbi who doesn't have the authority to rule it invalid. (The problem is that many people won't think about whether the rabbi had the authority, they'll be so shocked and angry.) Sure, a rabbi can choose not to accept your conversion in his community, but most rabbis can't rule your conversion invalid for other rabbis/communities.
Questioned conversion: This can happen for many reasons. Most simply, there is doubt whether the conversion was valid or not. The most common is when there is a converted mother or grandmother, and all the converting rabbis are dead and unknown. If today's rabbis don't know anything about the converting rabbis, they may say, "Well, we don't know if it was a valid conversion or not." (Continued frumkeit in the family can support that it was. If the convert and family went off the derech, you may be in trouble.) This is quite common, and is even a serious problem in the frum-from-birth community.
When a conversion seems questionable, rabbis often will decline to make a ruling on it (for many reasons). It's easier (lazier? better halachically? better for the person?) to throw the person in the mikvah for a geirus l'chumrah. I say it may be better for the person because a newer, "better" conversion can remove any doubt about the person's status and gives the person new paperwork to prove his or her Jewishness. It can remove any future headaches. On the other hand, you had better hope you weren't a "born Jewish" woman trying to marry a kohen when this issue rears its ugly head (or already married to one!). If you have a geirut l'chumrah, it is almost guaranteed that you cannot marry the kohen. (It's extremely rare, but it's possible to have a geirus l'chumrah that is very explicitly emphasizing the "l'chumrah" part. If you have to get a geirus l'chumrah l'chumrah before marrying a kohen, it will be presented to you as optional and then you will never, ever speak of it again. Your sons would remain kohanim in this case, as I understand.)
Why does this distinction matter? Generally, it matters because the situation is often better than you fear. Almost all these cases are questioned conversions. There's a protocol for that, though it may cost money, time, and maybe even some of your sanity. Best case scenario, it can be "fixed" within the hour. Worst case scenario, you go through a conversion from scratch like everyone else. Most people fall in the middle. No one is saying, "Nope, you're not a Jew," even if you might want to run out for McDonald's upon hearing the news.
If, chas v'shalom, this should happen to you, feel free to ask the rabbi whether he is making a ruling on the issue and what the options are to rectify the situation. Assuming you want to "remain" Jewish, of course.