Run WordPress on your local computer
Experimenting with modifications for your blog, on a live site could be catastrophic which could kill your site and result in the dreaded error page (Which you should have customized by now).
The only solution is to set up another blog as a testing ground for all your modifications. So might as well run a local copy on your Windows machine. With the installation of XAMPP lite, and a little tweaking you can have that local copy up and running in no time!
UPDATE: Mark has also pointed out that you can run WordPress from a USB drive as well. Any ideas for am Thumbdrive version for OSX?

Or you could just install it on a USB device
Everyone who’s ever been to the wordpress support forums have known about the tutorials on tamba2 ;)