Seeing an issue attempting to connect to Dell SC storage via BCM57402 NICs using bnxtnet driver version 20.6.302.0-1OEM.650.0.0.4598673 on FW version 20.06.04.03 (boot code 20.06.77).
Dell branded BCM57402 adapters:
VID: 14e4
DID: 16d0
SVID: 14e4
SSID: 4020
Using in conjunction with software iSCSI adapter, 1 subnet, port binding enabled, 2 vmkernel adapters.
Disabled TSO and LRO, however issue remains. Originally was using driver version bnxtnet 20.2.16.0, updated to 20.6.34.0 to align with HCL for FW version 20.06.77, however issue remained. Updated driver to 20.6.302.0 and issue remains, vmkernel logs with bnxtnet debug logging enabled show:
2017-08-21T15:18:23.037Z cpu70:65725)WARNING: bnxtnet: hwrm_send_msg:201: [vmnic9 : 0x410029a96000] HWRM cmd error, cmd_type 0x90(HWRM_CFA_L2_FILTER_ALLOC) error 0x4(RESOURCE_ALLOC_ERROR) seq 2393
2017-08-21T15:18:23.037Z cpu70:65725)bnxtnet: bnxtnet_uplink_stop_rxq: 358 : [vmnic9 : 0x410029a96000] RXQ 1 stopped
2017-08-21T15:18:23.037Z cpu70:65725)bnxtnet: rxq_quiesce: 391 : [vmnic9 : 0x410029a96000] host stop rxq 1
2017-08-21T15:18:23.037Z cpu70:65725)bnxtnet: uplink_rxq_free: 639 : [vmnic9 : 0x410029a96000] uplink request to free rxq 1
2017-08-21T15:18:23.037Z cpu70:65725)bnxtnet: hwrm_send_msg: 151 : [vmnic9 : 0x410029a96000] HWRM send cmd (type: 0x41(HWRM_VNIC_FREE) seq 2394)
2017-08-21T15:18:23.038Z cpu70:65725)bnxtnet: hwrm_send_msg: 209 : [vmnic9 : 0x410029a96000] HWRM completed cmd (type: 0x41(HWRM_VNIC_FREE) seq 2394)
2017-08-21T15:18:23.038Z cpu70:65725)bnxtnet: bnxtnet_rxq_free: 1140 : [vmnic9 : 0x410029a96000] attempt to free rxq 1
2017-08-21T15:18:23.038Z cpu70:65725)bnxtnet: hwrm_send_msg: 151 : [vmnic9 : 0x410029a96000] HWRM send cmd (type: 0x51(HWRM_RING_FREE) seq 2395)
2017-08-21T15:18:23.038Z cpu54:66284)bnxtnet: bnxtnet_process_cmd_cmpl: 2126 : [vmnic9 : 0x410029a96000] HWRM cmd (type 0x20 seq 2395) completed
2017-08-21T15:18:23.038Z cpu70:65725)bnxtnet: hwrm_send_msg: 209 : [vmnic9 : 0x410029a96000] HWRM completed cmd (type: 0x51(HWRM_RING_FREE) seq 2395)
2017-08-21T15:18:23.039Z cpu70:65725)bnxtnet: hwrm_send_msg: 151 : [vmnic9 : 0x410029a96000] HWRM send cmd (type: 0x61(HWRM_RING_GRP_FREE) seq 2396)
2017-08-21T15:18:23.039Z cpu70:65725)bnxtnet: hwrm_send_msg: 209 : [vmnic9 : 0x410029a96000] HWRM completed cmd (type: 0x61(HWRM_RING_GRP_FREE) seq 2396)
2017-08-21T15:18:23.039Z cpu70:65725)bnxtnet: bnxtnet_rxq_free: 1192 : [vmnic9 : 0x410029a96000] freed rxq 1 successfully
2017-08-21T15:18:23.759Z cpu70:65725)bnxtnet: hwrm_send_msg: 151 : [vmnic8 : 0x4100299fa000] HWRM send cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 40627)
2017-08-21T15:18:23.760Z cpu70:65725)bnxtnet: hwrm_send_msg: 209 : [vmnic8 : 0x4100299fa000] HWRM completed cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 40627)
2017-08-21T15:18:23.760Z cpu70:65725)bnxtnet: bnxtnet_priv_stats_get_len: 2410 : [vmnic8 : 0x4100299fa000] driver private stats size: 40384
2017-08-21T15:18:23.760Z cpu70:65725)bnxtnet: bnxtnet_priv_stats_get: 2449 : [vmnic8 : 0x4100299fa000] requested stat buf size is 40385
2017-08-21T15:18:23.760Z cpu70:65725)bnxtnet: hwrm_send_msg: 151 : [vmnic11 : 0x41000f07a000] HWRM send cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 36420)
2017-08-21T15:18:23.760Z cpu70:65725)bnxtnet: hwrm_send_msg: 209 : [vmnic11 : 0x41000f07a000] HWRM completed cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 36420)
2017-08-21T15:18:23.761Z cpu70:65725)bnxtnet: bnxtnet_priv_stats_get_len: 2410 : [vmnic11 : 0x41000f07a000] driver private stats size: 40384
2017-08-21T15:18:23.761Z cpu70:65725)bnxtnet: bnxtnet_priv_stats_get: 2449 : [vmnic11 : 0x41000f07a000] requested stat buf size is 40385
2017-08-21T15:18:23.972Z cpu70:65725)bnxtnet: hwrm_send_msg: 151 : [vmnic8 : 0x4100299fa000] HWRM send cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 40629)
2017-08-21T15:18:23.972Z cpu65:65725)bnxtnet: hwrm_send_msg: 209 : [vmnic8 : 0x4100299fa000] HWRM completed cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 40629)
2017-08-21T15:18:23.972Z cpu65:65725)bnxtnet: bnxtnet_priv_stats_get_len: 2410 : [vmnic8 : 0x4100299fa000] driver private stats size: 40384
2017-08-21T15:18:23.972Z cpu65:65725)bnxtnet: bnxtnet_priv_stats_get: 2449 : [vmnic8 : 0x4100299fa000] requested stat buf size is 40385
2017-08-21T15:18:23.972Z cpu65:65725)bnxtnet: hwrm_send_msg: 151 : [vmnic11 : 0x41000f07a000] HWRM send cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 36422)
2017-08-21T15:18:23.973Z cpu65:65725)bnxtnet: hwrm_send_msg: 209 : [vmnic11 : 0x41000f07a000] HWRM completed cmd (type: 0x18(HWRM_FUNC_QSTATS) seq 36422)
2017-08-21T15:18:23.973Z cpu65:65725)bnxtnet: bnxtnet_priv_stats_get_len: 2410 : [vmnic11 : 0x41000f07a000] driver private stats size: 40384
2017-08-21T15:18:23.973Z cpu65:65725)bnxtnet: bnxtnet_priv_stats_get: 2449 : [vmnic11 : 0x41000f07a000] requested stat buf size is 40385
At this point just curious if anyone has run into this, particularly HWRM_CFA_L2_FILTER_ALLOC errors. Per https://reviews.freebsd.org/file/data/jhjnayad4lkbrjvsspn3/PHID-FILE-lbmmlmi2bjqh54xmpwsd/D6555.id16878.diff appears issue possibly related to NIC FW, however currently Dell only has version 20.06.04.03 (boot code 20.06.77) available. Excerpt from freebsd.org indicating HWRM is FW related:
The Hardware Resource Manager (HWRM) manages various hardware resources
+ * inside the chip. The HWRM is implemented in firmware, and runs on embedded
+ * processors inside the chip
Connectivity is established, however seeing storage related issues on the array and in guest. Any advice is welcome and than you in advance.