You can use both protocols for separate LUNs, but mixing FC and iSCSI access with the same LUN is not supported by VMware. Not on a single host and not between hosts in a cluster. Each LUN should only be accessed by a single protocol in the whole vSphere environment.
In either case, I don't think it makes much sense to mix protocols unless you have a rare use case like in-Guest clustering with iSCSI, so stick with a single protocol as long as you can.