The most general criticism is that while Google's voice technology is some of the best most critics have ever used, it's not perfect. John Markoff at The New York Times says it works about 95 percent of the time, but had trouble with exotic or unusual names. He also said that it works best when speaking the way you type i.e. don't worry about speaking in full sentences. Voice search seems to be pretty fast too, according to iPhone Atlas Google's app was much faster for complex phrases than the keyboard, nothing surprising there I suppose.
Perhaps the most troubling criticism came from Michael Arrington over at TechCrunch who said that Google's app works terribly when there's too much background noise. If that's the case then I guess voice search will be pretty useless here in New York or any other major urban center.