Swapping harddisk full of Windows Media Center RecordedTV shows doesn't work as I planned

Here's what I did: I got 2 USB/eSATA based harddisk craddle, one connected to the living room computer while the other one connected to my closure server computer.  I used my closure server computer to record TV shows, and use the living room computer to watch those recorded TV show.   I thought I could easily swap these 2 harddisk and preserve the drive letter (so that the Recording process in the closet computer won't be interrupted.  Well that didn't work as planned.  The letter got reset everytime I plug in the drive, so I was like, whatever.