depping, thanks for showing an interest and if you've got any comments I'd be grateful. It's a very little setup:
2 x Dell R620 servers
1 x Dell Powervault MD3220i with 2 controllers each with 4 NICs. It contains 12 x 300GB 10k disks. 2 disks are spare, the other 10 are a single raid 5 storage group.
There are no switches involved, we've got a pair of patch cables going direct between each host and each controller, ie 4 cables from each host. We think now that not using switches was a mistake and unhelpful advice from the Dell guy who sold it to us. It seems to be not best practice, and when I call Dell and say we're not using switches they're amazed!
On Dell's advice we've changed the subnets. The default is for port 0 on both controllers to be in one subnet, port 1 on both controllers to be in another etc, ie 4 subnets in total. Dell said each port needs its own subnet with direct connect so we now have 8.
The Dell doc (in my first post) advises setting up iscsi port binding but this is wrong. It's only appropriate if all the vmkernels and target are on the same subnet, as per VMware KB: Considerations for using software iSCSI port binding in ESX/ESXi. This isn't the case with the subnets left as the default or as we changed them. Dell agreed that the doc's wrong, and said they already knew but didn't know a date when they'd fix it (thanks Dell, nice one I didn't need those few hours).
The Dell doc doesn't mention the settings I'm posting about, iops per path, delayed ack, large receive offload. I raised a case with vmware who said they don't make recommendations, they should come from the SAN vendor, and also that they do support changing these settings. I was concerned about that because on this forum someone says vmware told him they don't support iops=1 What's the point of setting IOPS=1? , maybe they didn't when he posted but they do now. I phoned Dell who said they had no recommendations about these settings and I should do my own tests. I've been messing around with them since. We'll decide tomorrow what we think and I'll post it here but we can change it if you or anyone advise otherwise, so I'd still be grateful for opinions.