Sounds like you are losing vac. to the booster in reverse, maybe a cracked vac. hose that flexes just right in reverse? Maybe a weak vac. pump but not sure why it would only act up in reverse. Bad vac. pumps on diesels are kinda common.
Have you tried pumping the pedal while rolling forward? maybe it will act up as well and you just have not noticed it yet??