How can I replace my paper books with digital copies?

As much as I enjoy books, I have far too many of them and am at a stage in my life that I know that I have more than I can ever read.  This gives me the urge to get rid of a good amount of them.
With the advent of electronic readers, its tempting to simply opt for digital versions of books. They don't take up any space and I can acquire even more of them than I could ever shelve in my house.
This has me thinking about how to replace a paper version of a book for an electronic one. In many cases, legitimate digital versions don't even exist, especially for older, out-of-print books which are too obscure to ever be digitized by their original publishers. Amazingly enough, though, there are plenty of pirated scans of these books.
Is it ethical to use the pirated version if there is no other way to obtain one? What if I were to make scans of my own books to digitize them for personal use? Is that different than making a backup copy of any other medium that I already own?
Of course, digitizing a book on my own is not a trivial task, but it's not an impossible one. There are DIY book scanners that can be cobbled together with a pair of digital cameras and some scrap wood. Once you have one of these devices, you can digitize a book in less than half an hour.
It is tempting to simply grab a digital file that someone's already created in order to replace the physical book I already own. The end result is the same, but is it legal, or even ethical?
To complicate the picture, when I actually look at the number of books I own, an overwhelming majority of them were purchased second-hand, which means that not a penny of the money I spent on them ever went to the original authors. My spending has mostly benefitted used-book sellers.
What would you do?