This may be a little lurid for some tastes, but considering the speaker is dealing with someone who clearly doesn't want to have sex, the audience needs to be reminded just how unnatural this really is. .
Further evidence that the audience is a woman can be found in the lines "Thou art thy mother's glass, and she in thee/Calls back the lovely April of her prime" (ll. 9-10). Simply paraphrasing, the speaker says she is the embodiment of her mother's youth, further comparing the audience to her mother and thus again implying that the audience is indeed at least a woman, if not the speaker's wife. Shakespeare is also once again reminding the audience that her mother had hoped to achieve a sense of immortality by giving birth to her. If the audience choose to have children, then she too will be able to look into her children's faces and see her eternal youth just as her mother sees her youth in the audience's face. "So thou through windows of thine age shalt see/Despite of wrinkles, this thy golden time" (ll 11-12). When she looks upon her self, she will be looking into a mirror that reflects her past beauty, basically a living mirror of years before. This third quatrain expanded the mirror metaphor that was introduced by Shakespeare in the first line and leads the reader to suspect that the audience suffers from narcissism. Perhaps the audience is so caught up with her self that not only does she not want the burden of children, but perhaps she doesn't want the burden of the speaker as well. The audience does not yet realize that one day she will look into her mirror and see the face of departed beauty. The one chance that she had of granting her beauty immortality by passing it down unto her children has passed. Thus wasting the best years of her life on herself. .
This revelation of the audience's self absorbency explains the anger expressed in the final two lines, "But if thou live remembered not to be/ Die single, and thine image dies with thee" (ll 13-14).