The difference is in how we more informally say 'it is your (or her or his or my) responsibility'. It may seem strange, but BrE and AmE look like they're complete opposites on this one. In BrE, one can say It's down to you to mean 'it's your responsibility to do that', whereas AmE would say it's up to you.
One does see it's up to you in BrE to mean 'it's your responsibility', or more specifically (as in AmE) 'it's your choice'. It's common enough in BrE that the OED marks it as just 'originally' AmE. There are two examples of It's up to you in the British National Corpus (accessed through corpus.byu.edu), both with this 'choice' sense:
I've done it and er I mean it's up to you as to which date you choose. [spoken in a meeting]But all the ones that are straight responsibility meanings are it's down to you in the BNC (10 hits):
"Well, it's up to you of course, Mr Dakin, but this is the third time I've had to stitch her teats and I'm afraid it's going to keep on happening." [from James Herriot's Vets might fly]
But if they get arrested it's down to you. [conversation](Of course there may also be examples of it'll be down to you or whether you do it is down to you or it's down to her or it's down to Nigel, etc. Searching for a single set phrase made it easier to avoid senses of down to that have nothing to do with responsibility.)
Unless you're a tenant, it's down to you to make sure gas appliances receive the regular expert servicing they need. [advert]
Between now and Sunday it's down to you to decide that you definitely want to go ahead [speech]
Meanwhile, in AmE, the Corpus of Contemporary American English has 398 hits for it's up to you and only one for it's down to you--and in that (fictional) context it might have just meant 'you're the only one left' (hard to tell--the responsibility meaning or the 'only you' meaning would both fit in the context).
So, up and down. Why are two opposite words used to mean the same thing? Because figurative language is slippery stuff, that's why. The OED tells us that up to [someone] is from the game of poker (traced to 1896), and is in general use from 1913. In the poker context it means basically 'it's your turn to make a decision and act on it'. So, it's sort of 'we've got(ten) up to you in the series of people who need to act in this game'. (One could have seen it going the other way, with one going 'down' the list of people whose turn it would be next. But poker is a game of escalation, so it doesn't seem surprising to me that the turn-taking metaphor goes upward.)
The BrE down to [someone] is only traced back to 1970 in the OED. One can see how this might come about from the 'there's no one else left' reading of down to (as in we're down to one candle). It's down to you says that, for the purpose of its context, the people who could have responsibility for something are reduced to one: you.
From my non-native perspective, it seems to me that up to and down to have different connotations in BrE--up to being choice and down to being serious responsibility. AmE doesn't make any such distinction and has up to for both.
It's down/up to you to tell us what you think.