Which tractor are you working on?
Fortunately I have never had to do this, but have looked at it on the Jinma 284 and it looks like it is pretty hard to get a straight shot at with a punch. Perhaps removing the floorboard would help. I think installation would be a little easier if you do it from the top which is roughly the 2 o’clock position.
It is my understanding that you install the larger pin first, then put the smaller one inside the larger one. This adds tension to hold it in place and adds shear strength.
Perhaps start the large pin until it gets about half way through. Then put some locking pliers on it to keep it from going any deeper and then start the small pin into the large pin. It will be easier if it is no t constrained by the hole. Once started remove the pliers and tap then both in together. You may have to hold a bucking bar on the back side so the big one does not come out before the small one is seated.
I have also wondered about putting a high strength bolt in that location rather than the roll pin but have not thought all the way through the consequences of doing it that way.