Yes, those rubber coated Flash Drives are small, safe and cheap...with some serious room on them. You can also hide them away quite easily, without too much trouble.

For a few dollars more than a basic unit, you can get one with a fingerprint reader on it so only you can open it or ones with 128bit encryption.