A universe is a small thing by comparison, as it contains only the sum totality of any one chain of causal events.
A multiverse could perhaps best be described as "Everything", as best understood and described by Lobsang Ludd and Wen in Thief of Time.
Everything is.... well, everything. Not just the things that we perceive are, from our lowly vantage point, of only being able to see and act in three dimensions of space and what we perceive to be one-way linear time. It contains everything that is, was, will be, and more importantly all the mights and maybes. Things and events that we might distantly speculate about, or that we wonder might have happened if events had played out a bit differently... for instance, Mustrum Ridcully wonders wistfully what might have been if he'd married Esmerelda Weatherwax. Meanwhile, Granny can't think straight for all the thoughts in her head which, while they're talking in her voice, damn well aren't hers. And all those memories of different lives led by other Esmerelda Weatherwaxes that are leaking in from somewhere...
Which is because somewhere out there, it HAS happened. As Roundworld seer Robert Anton Wilson phrased it
Everything you can imagine is happening out there somewhere.
Which is the definition of a Multiverse. Even if it isn't happening here, there will be a whole sub-infinity of parallel universes where it is.
If the created Multiverse contains all possibilities and all probabilities become events somewhere, then it also includes the concept that things and people may be able to cross the boundaries between Universes. Under great stress and need, for instance, Rincewind and Twoflower are able to pass temporarily to Roundworld, where they manifest aboard an airliner in alternate personalities, as Dr Rjinswand and Jack Zweiblumen. The Elves can cross between universes - not quite at will, but whenever they are called strongly enough. Noble dragons may cross to our universe when summoned. Time-travelling bag lady Mrs Tachyon can certainly travel in time - this includes the probability she may also be able to travel in space. Things from the Dungeon Dimensions may cross from their own phase of the Multiverse whenever opportunity presents itself. The Librarian uses L-Space as a means of navigating between the possibilities and probabilities of the Multiverse; Hex trawls it to bring back fragments and echoes of books from other universes. Hex also manages the various transitions of the Wizards, between the Disc and the Roundworld project.
Vampire flashsides are such a powerful psychic event for the vampire and anyone in his/her vicinity that they are capable of wearing down the usual dividing walls, and bringing two or more realities into the same place at once.
And some concepts or qualities or even people are so powerful that they resonate across every phase of the multiverse like a very big bell... but the biggest, loudest, most resonant bell of all which is heard everywhere in the Multiverse is, of course, Azrael.