In the Nation, the Women's Place belongs to, well, the women of the island. This is where babies are born, food is cooked, beer is brewed. The culture of Nation strictly enforces gender roles. When something is of a feminine nature and men shouldn't be involved, that something happens in the Women's Place.
Herbs, spices, foods, and medicines are stored around the Women's Place. The rule is, the more poisonous or potent the item, the higher up it's kept.
Men are emphatically not welcome in the Women's Place. The youngest boys go there - and everywhere - with their mothers, yes, but eventually those boys get stared at until they take the hint and never come back. Mao reported that he was told his wingo would fall off if he ever entered (Mao does in fact enter the Women's Place during the events of Nation. No report is made of the state of his wingo thereafter, though one expects that a falling-off would merit at least a passing mention).
The voices of the Grandmothers speak to Daphne from the Women's Place in the same way the voices of the Grandfathers call to Mau from their Cave.