those 3/7 secondaries that fail to connect – are those always the same 3 (i.e. if you power-cycle all 8 nodes, would the same 3 fail to connect?)
If yes – then I’d try to trace the issue to the hardware. It could be the HCAs, could be cables, could even the the USB-key holding the boot image. You can swap those and see if the problem follows the parts.
If no, then maybe this is a software limit or similar issue; how much memory does each node carry?
and, regardless of the answer to the question above, connectivity issues can also come from the fabric: (1) is there another vSMP instance booting on the same fabric? (2) is the fabric shared with some other cluster nodes (e.g. has another subnet manager running on it)?