On one of the mailing lists I subscribe to, a fellow Oblivion player noted that he had built up just enough gold to buy a house. He wondered if it was worthwhile to go broke in the process. My reply was:
I've bought two houses in my game :-)
The main reason that I own a house is for storage. The manual says that you shouldn't put stuff into a container you don't own, or it might not be there when you come back. As an experiment, I threw some stuff into a chest one of the Fighter's Guilds. It was there when I came back a short time later. BUT. After a lengthy excursion, when I returned... gone.
I've also left stuff sitting on the floor in rooms in the guilds. That seems to work well; I don't think it has ever been cleaned up on me. I haven't run a lengthy experiment, however.
Note that bodies count as containers, too. And they'll get cleaned up pretty aggressively within dungeons, so you want to loot the body and drop it on the floor if you're encumbered. You can come back and grab the stuff from the floor, and the body will typically be gone.
Anyhow... back to the house. Once you buy a house, you also have to buy furnishings for the house. The furnishings come in about eight sets (stuff for the bedroom, stuff for a storage area, wall hangings for the bedroom, etc), and the total of the eight sets is approximately equal to the house purchase price. These furnishings are bought at one of the vendors in the city where the house is (they don't show up until you buy the house). I've bought two houses, and then one storage area furnishings (my house in Bruma has two storage areas, but I've only bought one so far; house in Bravil has just one).
I use the houses to store excess loot between excursions: really expensive stuff that I can't sell to the local vendor (since there is a cap to what they'll offer you, I'd lose out on a bunch of $$), and also all the potion components that I have gathered (after gathering huge amounts, then I'll sit down and brew a gazillion potions, selling most of the results). I've found their utility as a depository most excellent. It's hard to sell a daedric warhammer worth 5000 gold to just any vendor :-P (one day, I'll find one... until then, it's in storage)
Hope that helps.