Boarding point of PNR is changed not individual passenger. Since 4 will already be there i dont think TTE will ask for a head count and insist on seeing all 6 pasengers. Incase he asks you can always convince him and tell him that they will board at Patna.