I've been speaking or performing for 30 years and guess what: I still get nervous.
Part of the problem people have with public speaking is their expectation. They expect that, once they start doing it, they will never be nervous again. When their nervousness doesn't retreat, they determine that there is something "wrong" with them, and start saying things like "I hate public speaking" or "I suck at public speaking."
Let it go. Let go of the expectation that you need to be anxiety-free. A little bit of stage fright is helpful to a speaker in the same way that an athlete uses the adrenaline rush to propel him to run faster; it's all in the way you look at it. If you convince yourself you must never feel anxious again in order to be successful as a speaker, you will be disappointed.
Check out my interview on the Sideroad about "Overcoming Public Speaking Anxiety." Of course, now you know that the article won't actually be about "overcoming" it, but rather appreciating it and managing it.
Do you have unrealistic expectations about your public speaking experience?