That is a viable tradeoff. Wi-Fi download fails for me half the time.
What would be nice is a dual-mode USB solution. One is talk to the app as it sounds like you're proposing. But I don't really use the app at all because I don't have or need the subscription they are heavily pushing (but would happily subscribe to RaceChrono for more updates!)
What I do - much like what @3002tii is proposing, is pop the card out and put it into a little portable USB-microSD adapter I carry with me. Just copy the files by hand.
Whenever I have tried this with RC it wont recognize the files unless I manually re-hand link them which of course breaks the time synchronization.
It would be nice if RC was less brittle and could maintain the time sync and filename independent of the path to the file. Leave it up to us to ensure the filenames are correct.