Very important: CPU type and speed. Multi-core CPUs have been around for a few years now.
This is the way to go if you plan to keep the unit for more than a year but there are a few of the single-core systems still out there that still run XP ok.
The important specs are a little fuzzy but let's take a shot ...

P3 or older: Forgeddaboudit! Don't bother! These systems are dogs. More importantly most newer software won't run properly if at all.

Pentium 4 or Pentium D are ok as long as the clock speed is at least 2.0ghz or higher.
I recommend that you be in the 2.4ghz and up range, and on these machines you'll definitely want to be running XP, not Vista or Windows 7.

Dual-Core ... even better.
Again, be sure clock speed is at least 2.0 ghz. Most of these machines will be Vista though so 2.0 will be a tad slow by current standards. Better to go with 2.4ghz or higher.

Triple core, Quad-core, or higher -- don't waste your time.
If you think you need more performance (gamer, graphics business, etc) get some help with your decision or invest some time reading up.
For the average PC user surfing the internet the extra performance means extra cost that isn't beneficial or justified.

Memory - on older machines running Windows XP need at least 512mb of RAM, but 1 Gigabyte of RAM is much better.
Don't bother with anything less. If someone tries to sell you a P4, XP machine with less than 512k tell 'em to keep it.
There are tons of these old dogs being offered for sale right now.
The machine will be slow and adding more ram will cost you more bucks!
Now if you can buy that machine for $50 and the CPU is at least 2.4ghz or higher then go ahead and buy it (assuming all the other specs are acceptable), then call me and spend another $25-50 to upgrade the ram to 1GB!

Quick memory (RAM) guide - Anything XP - 1 Gigabyte (no less). Vista or Windows 7 - 3 Gigabytes (no less). Lots of people will tell you that you can go with less and yes the PC will run with less RAM but you'll be disapointed by the performance.

Operating system - Windows XP is still a viable OS. Well over 50% of businesses still run it, what does that tell you? Windows 2000 - NOT! Don't waste your time with a Windows 2000 system Anything still running Windows 2000 probably has very old, slow hardware. Buyer beware!

Windows Vista and of course Windows 7 are also great.

Linux - don't go there unless you know what you are doing. Support for Linux is extremely limited and you'll get frustrated very quickly.

Hard disk - this is one area that depends a great deal on how you use your PC. If you plan to store lots of music and video files then bigger is better.
However, I suggest that you not worry too much about the size of the internal hard disk.
Generally, all systems came with at least enough hard disk space to store the operating system (windows) plus the programs (Microsoft office, etc).
If you don't plan to store a lot of large multimedia files (music / video) the factory disk is probably fine.
If you think you'll need more disk space don't even bother with opening up the case. Just buy an external hard disk.
That way your data is portable and if the PC does fail your data can easily be moved to another system either temporarily or permanently if you decide to.

CD/DVD player or burner - we're talking low-cost, used systems here. Unless you're building a home entertainment center don't worry too much about the CD/DVD player.
Of course it needs to work reliably. Beyond that there's not much to say.
Don't get stuck with a CD-only unit though, as a lot of media comes on DVD now.
And definitely don't pay a premium for lightscribe burners.
These are the CD/DVD burners that use discs with special coating to burn the label right on the surface of the disc. YUK!!!
One disc can take 30 minutes or more just to burn the label!!!
The lightscribe discs cost a lot more than regular blank discs, they are painfully slow to burn, and the quality is crap.

Ports - as long as the unit has the basic connectors - keyboard, mouse, monitor, network (RJ45 for ethernet) plus a few USB connectors, you're set. Older units still had some USB 1.1 ports though so stay away from them because the older USB ports were slooooow. Transferring pics from your digital camera will be a real pain in the backside!

Speakers - this is the least important part of your purchase. You'll be lucky if the system even comes with speakers. Unless you plan to crank up the volume most all pc speakers will get the basic job done. This shouldn't be a deal-breaker but never pay a premium price for used speakers. Buy new speakers if you want high-performance audio! Listen to 'em, and as long as they don't rattle too much they will probably be ok.

Mouse and keyboard. As long as they work and aren't covered with a decade's worth of crud.

Price -- now comes the fun part. You can buy a complete, new PC system that will blow most of the older units on Craigslist out of the water for under $500, so why would you spend anywhere close to that for a used system? Most CL systems come without a warranty. One repair can easily set you back $100. You must factor that possibility into the price. Starting point for a complete, good, used PC on CL is around $200. Never consider paying more than that. If you go much beyond that price range you might as well consider buying a new system with a warranty.

No recovery discs with the Windows operating system? If the unit is OEM (Dell, HP, etc) you might be able to buy a copy of the recovery discs from them for around $25. Problem is the OEMs stop supporting hardware within 2-3 years. If the machine is older than that, take $50 off for no recovery discs as you'll definitely be paying a premium price if you can't get them from the OEM.

Not top of the line CPU speed? (2.8 GHZ or higher) Take $25 off (from the $200) but don't even bother with a system that is slower than the clock speeds recommended above.

Not a 19inch or larger flat panel display? Take another $50 off. Do not bother with less than 17 inch and never, never take home a tube display.

Doesn't have at least 1GB RAM for XP, or 3GB for Vista/Windows 7, take $50 off.

CD-Only (no DVD) - take $50 off. The drives are cheap (internal DVD burners are available for $30) but if you're not going to replace it yourself you'll pay someone else, it will cost you. Even if you replace it yourself, hey your time is worth something isn't it?

Last but definitely not least - you must see the system run and you must test it out! Make the seller set it up for you, if it is on turn the unit off and then restart it. Plan to spend at least 15-20 minutes on this step.

• Confirm the basic operation of Windows - starts up and shuts down without errors, runs notepad, paint, calculator, etc without problems.
  
  
• confirm the hardware configuration is what the seller says it is (CPU, RAM, HDD, etc).
  
  
• Bring a USB flash drive and confirm that it reads reliably in at least two of the USB ports. Watch for the USB warning that says "this device will work faster" ... that means the USB ports are older, slower 1.1 ports. (stop here, you don't want this machine)
  
  
• Check cables and connectors. Abused cables are common and will be unreliable. Look for bent pins, forced or broken connectors, stripped or missing screws, etc.
  
  
• Bring a DVD - any movie is fine. If the unit doesn't play the DVD or even recognize it then the drive is either broken or is CD-only. If it doesn't even recognize the disc you'll probably need to replace the drive. Some XP systems did not come with DVD player software so they may not play but will at least recognize that there is a disk in the drive. Be prepared to negotiate on price if the DVD doesn't play!
  
  
• Monitor - flat panel technology is quite reliable. Most problems are the result of abuse. Look for scratches, dents, liquid spills. Check out the corners to see if the panel has been dropped. Illumination should bright and even. Here's a quick test. Open Windows paint and expand to full screen. Set Image -> attributes to 2000 by 2000. Use the little "paint bucket-control" to Fill the window with first red, then green, blue, black and finally white. You're looking for all black, all white, or off-color pixels. Practice this at home before you go so you know what you're doing. :-) If the monitor isn't perfect, negotiate!
  
  
• Noise - is the case fan running? Is it noisy? What about the hard disk or DVD drive? Ask them to turn the TV or stereo off! A little fan noise is ok, especially at startup. After the power control software is running, fans usually slow back down until the CPU is under a load. Any hard disk or DVD drive noise should be suspect. Asume it will need to be replaced. In fact, it's probably safe to assume that's why they are selling the system in the first place!

I hope this info has been helpful. If you have questions feel free to shoot me an email!