Many agents say they can tell how good a novel is going to be just by how well the dialogue is written. I've even heard some say they've rejected a manuscript solely because the dialogue was sub par. You can't afford for you dialogue to be less than the very best it can be.
There's a fine line between keeping your dialogue realistic and keeping it from being so realistic it's painfully boring. Much of an actual conversation in life is full of long pauses, short phrases, and annoying words like well and uh. That doesn't mean you shouldn't use these words, it just means you need to limit them.
The dialogue you choose to put in your book should be meaningful. If the conversation doesn't move the plot or the character arc forward, chances are you don't need it in the book. If your characters are talking about something pointless that's fine as long as there's a point to the scene itself. Perhaps the reader learns something important about their mannerisms, their beliefs, or principles. In this case the conversation can be light but still meaningful.
When in doubt refer back to the books you've read. Check out the dialogue. What do they put in and more importantly, what do they leave out? You can learn a lot just by paying attention to what you're reading!
For more on the subject check out Noah Lukeman's The First Five Pages, it has a great section on dialogue: http://search.barnesandnoble.com/The-First-Five-Pages/Noah-Lukeman/e/9780743290937
And check out this guest article on dialogue on the Writer's Digest website: http://writersdigest.com/article/Tips_for_Injecting_Dialogue_With_Suspense_and_Tension