Back in the days before USB 2.0, I was tempted to start every backup article with the same dumb joke: "Backing up is hard to do."
Luckily, easy backups don't require USB If you have broadband Internet access (DSL or cable). Just back up over the Internet. I recommend Mozy Home for the best and easiest backup for the price ($5 a month).
But if you're accessing the Internet through dial-up, you'll need to supplement your old hardware with more old hardware. Buy an internal or parallel port-based external backup drive with removable media such as tape or disk cartridges. Iomega's Zip drives, which use disk-based cartridges, were quite popular ten years ago (I recently found some of the disks while cleaning out my garage). Iomega still makes and sells Zip drives, although the modern external ones use USB. But as I write this, five used parallel port Zip drives are on sale through Amazon.com.
You might also want to consider a DVD+/-RW drive to replace what I'm going to guess is your CD-ROM drive. These are still commonly made and reasonably inexpensive, and the discs are very cheap.