The Guge Kingdom was founded in about the tenth century by a descendant of King Lang Darma, who fled from Lhasa after the collapse of the Tubo Kingdom. The kingdom played an important role in the second renascence in Tibet and survived for about 700 years before disappearing mysteriously in the 17th century.

Today's Tholing and neighbouring Tsaparang are the ruined former capitals oa the ancient Guge kingdom. It lie on a hilltop near a river and cover 180,000 square meters. Houses, cave dwellings, monasteries and stupas were found on the hill and surrounding area. Palaces sat at the summit, while monasteries were on the mountainside and cave dwellings for the common people were at the foot of the hill. The kingdom was enclosed by tunnels and walls which acted as fortifications. Some structures survive time and remain in good condition in this isolated region, though many structures have been reduced to dust. A two-kilometer long tunnel was built of stones, but is now in ruin, dives the summit to the river below and was used as a water supply for the Guge people.

The Guge Kingdom is famous for its murals, sculptures and stone inscriptions, which are attached to the surviving structures. Among them, murals from White Palace, Red Palace, Yamantaka Chapel, Tara Chapel and Mandala Chapel are preserved in good condition, although they are hundreds of years old. The themes of the murals include mainly stories of Buddha, Sakyamuni, Songtsen Gampo, kings of Guge and their ministers. A chapel on the summit of the hill houses a mural depicting male and female Buddhas bringing the Tantric cultivation (civilization) together, while the lower part displays purgatory with naked, enchanting Dakins flanking each side. The artistic and aesthetic value of Guge murals is deemed comparable with that of Mogao Caves.

The wall of Guge is actually a library of stone inscriptions, which are equally impressive as its murals. Mani stones are scattered around. Most sculptures of the Guge style are gold and silver Buddha.Around the ruins are weapons of the Guge people and mummies, which are probably Guge soliders, the only traces of the once glorious kingdom.