And as much as I remind my clients that audiences want you to succeed and are not there to critique you, there is a legitimate reality here:
We are judging and critiquing people all the time.
We walk down the street noting what people are wearing, what people are saying, what people are holding, what people are doing. We notice, we filter, and then we move on.
We do this to determine lots of things, but mostly: Is this person like me? How much is this person like me or different from me?
We are always subconsciously trying to figure out our relationship to other people. Should I smile? Should I be threatened? Is this person trustworthy? Does she want something from me? Can I connect with this person?
So yes, in a way, you are being judged. But not in the way you think. The audience is trying to determine what their relationship is with you. Can they trust you? Will you listen to them and understand where they're coming from? Is this relationship worth their time and money?
Your job on stage is to connect, relate, and deliver. You don't have to be like your audience to do so, but you do have to be compatible with what your audience expects from a speaker in the general sense (see my article on being professional, for starters), and with what the audience is looking for specific to your topic. Your words have to be consistent with your actions and emotions. You have to be sincere, authentic and focused on serving the audience.
If you make a mistake, this does not damage your relationship with the audience. They won't judge you for forgetting what you were saying or having an equipment snafu. It's not the technicalities that interfere with your relationship.
What does damage your relationship with the audience is your lack of connection, your lack of understanding of who they are and what they want, and your overly anxious concern for your own well-being.
When you get past that, nothing can scare you anymore.