Generally only in extreme cases is it going to hurt the diff itself. Thats when its so high that the pinion bearings don't get proper oiling. People in this case often overfill the diff through the vent fitting.
There are various rules that dictate your diff placement, but the base rules are that if its a CV joint, the diff should be pointed in line with the driveshaft. If its a regular U-jiont, then it should be pointed upwards at the same angle that the t-case rear output is pointed downward.
Then, depending on your suspension type (leaves vs links, bushings vs solid joints, etc), you need to point it down a degree or two from those points due to axle wrap.