I am missing your point here or you're missing mine.
The spending is going on, whether we like it or not. The welfare numbers are skyrocketing as is unemployment. Now, take those fit to work and put them to work where they are needed. If we are already paying for them to live (welfare) or they are sitting idle waiting for employment, they could at least learn additional skills and complete needed work.
Edit: Unless this is more your point:
"Just think how that society would have thrived if all those poor souls assigned to monument building and similar tasks had been allowed to pursue their own individual talents and goals."
Monument building is very different than road repair, bridge and dam building, tree planting and harvesting, emergency response support, etc.