your carrier bearing should not just slide off the carrier like that. Either the bearing is very warn or the carrier is. It's usually the carrier that will need to be replaced. The bearings are harder then the carrier.
Check to see if a new bearing will slide on, or if is tight. The bearings are normally are machined more accuracy than the carrier and thus shims to account for the carrier, not the bearings. Thats why people use setup bearings to test shims and then install the new bearings and the setup is correct. The bearings can be swapped all day long as long as the shims are in the same location.
About your noise, It's hard to say if it's the pinion or not. Normal warn bearings will have slop or a sticky spot on the bearings when you turn. Is also hard to tell with the seal on because that holds the yoke also. I would pull the seal and check the bearing without a seal to see if its free and smooth. If it is, check the carrier to make sure the side that the bearing came off by hand is not warn. Press new bearings on it and put it back together to give it a shot. If it still doesn't work, then you will just need to change pinion bearings and see. You are not really out anything but time and some gear lube.