Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add doc for VF flags and change the namespace of the operator #4

Closed
wants to merge 4 commits into from
Closed

Add doc for VF flags and change the namespace of the operator #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8b9bdd1
Add doc for VF flags and change the namespace of the operator
pliurh Nov 14, 2019
451e11f
Fix some typos
pliurh Nov 15, 2019
9077676
Fix nits
pliurh Nov 18, 2019
7c67ef2
Remote static MTU range
pliurh Nov 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions modules/nw-sriov-configuring-device.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
name: <name> <1>
namespace: sriov-network-operator <2>
namespace: openshift-sriov-network-operator <2>
spec:
resourceName: <sriov_resource_name> <3>
nodeSelector:
Expand All @@ -55,18 +55,18 @@ spec:
<4> Specify the node selector to select which node to be configured. User can choose to label the nodes manually or with tools like Kubernetes Node Feature Discovery.
Only SR-IOV network devices on selected nodes will be configured. The SR-IOV
CNI plug-in and device plug-in will be only deployed on selected nodes.
<5> Specify an integer value between `0` and `99`. A larger number gets lower priority, so a priority of `99` is lower than a priority of `10`.
<6> Specify a value for the maximum transmission unit (MTU) of the virtual function. The value for MTU must be in the range from `1` to `9000`. If you do not need to specify the MTU, specify a value of `''`.
<5> Optional. Specify an integer value between `0` and `99`. A larger number gets lower priority, so a priority of `99` is lower than a priority of `10`. The default value is `99`.
<6> Optional. Specify a value for the maximum transmission unit (MTU) of the virtual function. The max mtu value may vary for different NIC models.
<7> Specify the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel Network Interface Card (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than `128`.
<8> The `nicSelector` mapping selects the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters. It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally.
If you specify `rootDevices`, you must also specify a value for `vendor`, `deviceID`, or `pfName`.
If you specify both `pfName` and `rootDevices` at the same time, ensure that they point to an identical device.
<9> Specify the vendor hex code of the SR-IOV network device. The only allowed values are either `8086` or `15b3`.
<10> Specify the device hex code of SR-IOV network device. The only allowed values are `158b`, `1015`, `1017`.
<11> The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.
<9> Optional. Specify the vendor hex code of the SR-IOV network device. The only allowed values are either `8086` or `15b3`.
pliurh marked this conversation as resolved.
Show resolved Hide resolved
<10> Optional. Specify the device hex code of SR-IOV network device. The only allowed values are `158b`, `1015`, `1017`.
<11> Optional. The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.
<12> The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format: `0000:02:00.1`.
<13> Specify the driver type for the virtual functions. You can specify one of the following values: `netdevice` or `vfio-pci`. The default value is `netdevice`.
<14> Specify whether to enable RDMA mode. The default value is `false`. Only RDMA over Converged Ethernet (RoCE) mode is supported on Mellanox Ethernet adapters.
<13> Optional. Specify the driver type for the virtual functions. You can specify one of the following values: `netdevice` or `vfio-pci`. The default value is `netdevice`.
<14> Optional. Specify whether to enable RDMA mode. The default value is `false`. Only RDMA over Converged Ethernet (RoCE) mode is supported on Mellanox Ethernet adapters.
+
[NOTE]
=====
Expand Down
2 changes: 1 addition & 1 deletion modules/nw-sriov-device-discovery.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodeState
metadata:
name: node-25 <1>
namespace: sriov-network-operator
namespace: openshift-sriov-network-operator
ownerReferences:
- apiVersion: sriovnetwork.openshift.io/v1
blockOwnerDeletion: true
Expand Down
22 changes: 11 additions & 11 deletions modules/nw-sriov-installing-operator.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ As a cluster administrator, you can install the Operator using the CLI.

. Create a namespace for the SR-IOV Network Operator by completing the following actions:

.. Create the following Namespace Custom Resource (CR) that defines the `sriov-network-operator` namespace, and then save the YAML in the `sriov-namespace.yaml` file:
.. Create the following Namespace Custom Resource (CR) that defines the `openshift-sriov-network-operator` namespace, and then save the YAML in the `sriov-namespace.yaml` file:
+
----
apiVersion: v1
kind: Namespace
metadata:
name: sriov-network-operator
name: openshift-sriov-network-operator
labels:
openshift.io/run-level: "1"
----
Expand All @@ -49,10 +49,10 @@ apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: sriov-network-operators
namespace: sriov-network-operator
namespace: openshift-sriov-network-operator
spec:
targetNamespaces:
- sriov-network-operator
- openshift-sriov-network-operator
----

.. Create the OperatorGroup CR by running the following command:
Expand All @@ -79,7 +79,7 @@ apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: sriov-network-operator-subsription
namespace: sriov-network-operator
namespace: openshift-sriov-network-operator
spec:
channel: <channel> <1>
name: sriov-network-operator
Expand All @@ -95,12 +95,12 @@ spec:
$ oc create -f sriov-sub.yaml
----

.. Change to the `sriov-network-operator` project:
.. Change to the `openshift-sriov-network-operator` project:
+
----
$ oc project sriov-network-operator
$ oc project openshift-sriov-network-operator

Now using project "sriov-network-operator"
Now using project "openshift-sriov-network-operator"
----

[id="install-operator-web-console"]
Expand All @@ -122,13 +122,13 @@ in the previous section.

.. Choose *SR-IOV Network Operator* from the list of available Operators, and then click *Install*.

.. On the *Create Operator Subscription* page, under *A specific namespace on the cluster* select *sriov-network-operator*. Then, click *Subscribe*.
.. On the *Create Operator Subscription* page, under *A specific namespace on the cluster* select *openshift-sriov-network-operator*. Then, click *Subscribe*.

. Optional: Verify that the SR-IOV Network Operator installed successfully:

.. Switch to the *Operators* -> *Installed Operators* page.

.. Ensure that *SR-IOV Network Operator* is listed in the *sriov-network-operator* project with a *Status* of *InstallSucceeded*.
.. Ensure that *SR-IOV Network Operator* is listed in the *openshift-sriov-network-operator* project with a *Status* of *InstallSucceeded*.
+
[NOTE]
====
Expand All @@ -144,5 +144,5 @@ If the operator does not appear as installed, to troubleshoot further:
the *Operator Subscriptions* and *Install Plans* tabs for any failure or errors
under *Status*.
* Go to the *Workloads* -> *Pods* page and check the logs for Pods in the
`sriov-network-operator` project.
`openshift-sriov-network-operator` project.

23 changes: 19 additions & 4 deletions modules/nw-sriov-network-attachment.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,35 @@ apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
name: <name> <1>
namespace: sriov-network-operator <2>
namespace: openshift-sriov-network-operator <2>
spec:
networkNamespace: <target_namespace> <3>
ipam: |- <4>
...
vlan: <vlan> <5>
resourceName: <sriov_resource_name> <6>
linkState: <link_state> <7>
maxTxRate: <max_tx_rate> <8>
minTxRate: <mix_rx_rate> <9>
vlanQoS: <vlan_qos> <10>
spoofChk: <spoof_check> <11>
trust: <trust_vf> <12>
capabilities: <capabilities> <13>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pliurh - Is this another block scalar, like the following?

capabilities: |-
  {
    ...
  }

Thanks!

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zshi Could you give some comments?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the example in zshi-redhat@73c3637

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, here is another example using block scalar: https://github.com/redhat-nfvpe/openshift-baremetal-ci/blob/master/tests/sriov/templates/sn-static-ipam.yaml
this template has been tested in CI env.


----
<1> Replace `<name>` with a name for the CR. The Operator will create a NetworkAttachmentDefinition CR with same name.
<2> Specify the namespace where the SR-IOV Operator is installed.
<3> Replace `<target_namespace>` with the namespace where the NetworkAttachmentDefinition CR will be created.
<4> Specify a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.
<5> Replace `<vlan>` with a Virtual LAN (VLAN) ID for the additional network. The integer value must be from `0` to `4095`. The default value is `0`.
<3> Optional. Replace `<target_namespace>` with the namespace where the NetworkAttachmentDefinition CR will be created. The default value is `openshift-sriov-network-operator`.
<4> Optional. Specify a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.
<5> Optional. Replace `<vlan>` with a Virtual LAN (VLAN) ID for the additional network. The integer value must be from `0` to `4095`. The default value is `0`.
<6> Replace `<sriov_resource_name>` with the value for the `.spec.resourceName` parameter from the SriovNetworkNodePolicy CR that defines the SR-IOV hardware for this additional network.
<7> Optional. Replace `<link_state>` with a the link state of VF. Allowed value are `enable`, `disable` and `auto`.
<8> Optional. Replace `<max_tx_rate>` with a Maximum tx rate, in Mbps, for the VF.
<9> Optional. Replace `<min_tx_rate>`` with a Minimum tx rate, in Mbps, for the VF. This value should always be less than or equal to Maximum tx rate. Note: Intel NICs don't support this parameter.
<10> Optional. Replace `<vlan_qos>` with a IEEE 802.1p Priority levels for the VF. The default value is `0`.
<11> Optional. Replace `<spoof_check>` with the spoof check mode of the VF. Allowed values are `"on"` and `"off"`.
<12> Optional. Replace `<trust_vf>` with the trust mode of the VF. Allowed values are `"on"` and `"off"`. If `on` or `off` is specified without quotes, it will be rejected by CRD validation, since it will be evaluated as boolean value.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pliurh -

it sounds like trust is treated differently if it's specified as trust: "on" versus trust: on, but I thought in YAML, a flow scalar is equivalent whether it's quoted or not in the simplest cases? Does Go handle YAML differently?

(http://blogs.perl.org/users/tinita/2018/03/strings-in-yaml---to-quote-or-not-to-quote.html)

Thanks!

cc @zshi-redhat

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jboxman It should be the same as spoof_check.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested to use quotes. otherwise it will report error when creating sriov network according to previous tests.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like go-yaml tend to support both 1.1 and 1.2. some relevant readings:
kubernetes/kubernetes#34146
go-yaml/yaml#214

<13> Optional. Replace `<capabilities>` with the capabilities to be configured for this network. Capabilities supported: (mac|ips), e.g. `'{"mac": true}'`, it allows SR-IOV CNI to set a static mac address to a Pod. More information can be found [TODO:[put a link to Static IP/MAC address assignment doc.]]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pliurh, due to limitations in our tooling, we aren't allow to link to other documentation modules from within a module. What's the Static IP/MAC address assignment doc?

Copy link
Owner Author

@pliurh pliurh Nov 27, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a new feature of Multus.
@dougbtv Do you know where is the doc for the Static IP/MAC address assignment?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one example for SR-IOV using runtimeConfig: zshi-redhat@73c3637
I'd also like to link to multus one.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this for now. We've got multiple PRs in flight. After I get them in I can do another pass and look for the best place to add some "Next steps" links or "Related resources" to help connect these docs.


. Create the CR by running the following command:
+
Expand Down