The garden in the title is a reference to the lost paradise of Genesis but also the hortus mentis of the philosophers, as well as the hortus conclusus from the Song of Songs. To sum up: all the places of solitude, where the mind can recreate itself and perform its act of contemplation.
Crown’d from some single herb or tree
While all the flowers and trees do close
The speaker places himself in a felicity which he earned by abandoning the busy companies of men, mocked for toiling in order to win a garland of recognition, woven from the leaves of either the palm of a triumphant general, the oak of the public leaders or the bays of an artist. They could have the whole tree without so much activity and nervousness. The reward for inactivity are the garlands of repose, as Marvell calls them. The tone is rather amused - the crown braided up - as a girl braids her hair. But upbraid as a reproach is also implied: fools indeed are all those hectics who prefer the scanty shade of the garland to the shade of the tree.