The Golems of Umnia differ rather significantly from the usual, run-of-the-mill golems usually found in Ankh-Morpork and surrounding areas in that they are not baked from clay. They were discovered by Adora Belle Dearheart on one of the missions of the Golem Trust after a working golem heard the mind of one lifted in song far from Ankh-Morpork. Initially it was thought, due to a mistranslation that the golems were made out of gold (The initial translation was "4 golden golems"); however, it was later revealed that there were actually 4,000 golems. She got them to come to Ankh-Morpork where no-one could get them to do anything until Moist von Lipwig prevailed upon the spirit of Professor Flead to translate for him, whereupon he marched them (and their marvellous Golem Horses, a far less numerous variant dedsigned to look like and do the accepted tasks allotted to equines) out of the city and got them to bury themselves again. He then proceeded to put Ankh-Morpork on the Golem Standard.
The golems, originally brought to a "life" of servitude in Umnia thousands of years ago, were so efficient at doing everything that the Umnians never even invented the wheel. However as Hubert Turvy pointed out, getting them to do everything in Ankh-Morpork would put tens of thousands out of work, leading to no-one being able to buy anything, leading to shops going out of business, leading to irretrievable economic breakdown. This presumably is why Lord Vetinari lifted nary a finger to stop Moist's burial of them all. Besides, they're always there should the need arise...(The need arose in the events of Raising Steam, when it became imperative to get a train to Überwald while a necessary bridge wasn't in place, but Moist swears the Golems never moved and the Patrician and Stoker Blake both back him up.)