SR-IOV is not Passthrough (Response to “The Rise of Soft Switching: Part 1″)

[Note: I've been working with networking at the virtual edge for over 3 years now, and every time SR-IOV has come up, it has been in the context of passthrough (bypassing the virtual switch).  However, Sean Varley of Intel wrote to point out that the two should not be conflated.  He further points out the potential benefits that SR-IOV can provide.  Rather than try and rehash an already great note, I figured I would just include it here in its entirety.   So first, sorry for screwing that up.  Mea culpa.  Second, thanks to Sean for providing the clarification.  And finally, passthrough still sucks ;)]

I’d like to clarify a technological fact that you may have overlooked in this recent blog post that I think has done a disservice to the SR-IOV technology and partly isn’t even relevant to the point you are trying make with this post.
The fact is essentially this – SR-IOV is NOT a kernel or soft switch bypass technology.  I agree that there is a great misperception in the industry that it is only relevant under these circumstances, however, as a specification it merely allows for multiple child functions to be instantiated under a parent function in a standard manner under the PCIe interface.  This has much more to do with identifying, associating, and managing NIC partitions than anything else.  By erroneously applying this basic functionality to a single specific use case of soft switch bypass, you are pigeon holing it to a small space and perpetuating a significant myth in the industry.
I have attached a single diagram that shows the wide variety of uses for SR-IOV in emulated, local kernel, and even non virtualized environments where SR-IOV is a significant enabling technology.  The essence of NIC partitioning is very simple:  I want a single big pipe to look like a bunch of smaller pipes.  SR-IOV doesn’t do this but it allows a standard interface to be applied to a partition that enables every OS in the industry to recognize and enumerate that partition as an interface to bind whatever service it likes to according to a standard.  All other partitioning technologies that exist today (Flex10, vNIC, etc.) are proprietary technologies that actually clutter the market place with vertical, inherently incompatible (amongst vendors) solutions to NIC partitioning.

3 Comments on “SR-IOV is not Passthrough (Response to “The Rise of Soft Switching: Part 1″)”

  1. Peter Phaal says:

    I have been following the work that Luca Deri has been doing to monitor 10G traffic using commodity network adapter cards.
    http://svn.ntop.org/nema2010.pdf

    I wonder if a similar approach might make sense for vSwitches. Instead of using the filtering hardware to create packet queues for each VM (bypassing the vSwitch), the hardware could be used to assign traffic to CPU cores running the vSwitch function. It seems like this approach would provide many of the benefits of hardware assistance, improving throughput and reducing latency in the vSwitch, while still maintaining the flexibility of a software switch.

    • Mallik Mahalingam says:

      Peter:

      Hypervisors such as vSphere, has been employing techniques to steer packets to appropriate NIC queues and these queues are then handled in appropriate cores [using MSI-X interrupts] for concurrent packet processing of different VM’s workload on a shared physical NIC. The queues are programmed based on MAC address and we have had this for a while now. You can sort of look at this as extension to what RSS does with respect to packet classification and coupling them with cores/queues. This does not bypass vSwitch as such. It gives nice scaling with respect to packet processing while giving the benefits of all solutions installed in the vSwitch layers.

      Take a look at Netqueue feature was introudced in vSphere 3.5 [http://www.vmware.com/support/vi3/doc/whatsnew_esx35_vc25.html].

      There are some potential latency benefits since each vNIC gets its own “queue” and packet interleaving is somewhat avoided but the IO processing and any associated network services are done in software, there might be potential latency trade-off. As you noted for throughput, this type of architecture is perfect.

  2. Brad Hedlund says:

    Re: anything other than SR-IOV being “proprietary” and cluttering market

    To be fair, it should be acknowledged that many OS and hypervisors still lack SR-IOV support. The solutions on the market today (Cisco UCS, Flex-10) may be vertically integrated, but so what? These solutions work _today_ in a way that’s totally transparent to the OS or hypervisor, via emulating a PCIe riser.

    BTW – passthrough doesn’t suck .. as currently implemented in some stacks such as Cisco + VMware. Totally understand the current suckage with passthrough in Linux hypervisors tho, which is where I believe your opinion on that comes from?

    Cheers,
    Brad


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 407 other followers