That depends on the site, and on the computer you're using.
There's not much danger if we're talking about your own computer. The chances of someone accessing your account via an active logon are pretty thin. And in cases where it might happen, it's probably because either your PC is already infected, or an untrustworthy person has physical access to it. Either way, you're already in trouble.
But I'm a cautious person, and I recommend others be cautious, as well. That's why I recommend logging out of financial and retail sites, where someone else's illegal access can have serious consequences. I don't bother loggin out of other sites.
You really don't want to stay logged onto a financial site indefinitely. If a criminal can get access to your bank account or credit card number, your life is going to become very unpleasant. Luckily, most financial sites will log you out automatically after a certain number of minutes of no activity.
Retail sites don't do that--probably because they have a financial incentive to make your purchases as easy as possible. Amazon.com doesn't even offer a clear logout option on their site. The trick is to click the Not your name link that you'll find in parentheses near the top of the page.
If you're using a public computer--say, in a library--or even an office computer that co-workers can access, the answer is simpler: Always log off of everything.
Contributing Editor Lincoln Spector writes about technology and cinema. Email your tech questions to him at firstname.lastname@example.org, or post them to a community of helpful folks on the PCW Answer Line forum. Follow Lincoln on Twitter.