- Official Post
Not a fully fleshed theory, but here's my thought.
After Ocarina of Time in the child timeline, Ganondorf's plot if foiled and he is executed. At this point (and with Link not around to argue a different perspective), Ganondorf is not known to Hyrule as the King Of Evil, or Demise's reincarnation, or anything like that. He is purely King Of The Gerudo, a secular figure. They attach the blame to the Gerudo in general and their culture, rather than to Ganondorf individually, and they embark on a very dark campaign of cultural eradication. Come the time of Twilight Princess, and there are no Gerudo to be found anywhere. Their village is nothing more than rubble in the desert. The Spirit Temple, the most spiritually significant site in Geurdo culture, has been turned into a worryingly large prison. It suggests to be that by 'Hyrule's most ruthless criminals' what they really mean is the Gerudo wholesale, and that entire population was imprisoned there, Uighur style. Either they were executed en masse or conditions were so poor that many of them died anyway, looking at the number of undead things that wander the dungeon in TP. The lucky ones were sent to inhabit the Twilight Realm, like a Native American reserve built on the most desolate of wastes.
Sometimes Zelda gets heavier than expected.