Forwarding NIC renaming udev rules from the initramfs

[dustymabe]

There is a dracut arg called ifname= that will allow someone to dynamically create a udev rule to change the name of a NIC to whatever they would like it to be. For example I could tell the system to name NIC with MAC 12:34:56:78:9a:bc to infra with ifname=infra:12:34:56:78:9a:bc.

This gets applied via ifname-genrules.sh from the 40network dracut module which creates a rule in the initramfs like:

# cat /etc/udev/rules.d/80-ifname.rules
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="12:34:56:78:9a:bc", ATTR{type}=="1", NAME="infra"

Since we persist other parts of the networking config from the initramfs (static IPs, hostname, etc), should we persist these udev rules as well? Or should we require a user to either write out the udev rule (or systemd link file) themselves OR add the ifname= karg so that it gets dynamically generated every time?

Just for posterity here is how to create a udev rule or systemd link file:

storage:
  files:
    - path: /etc/udev/rules.d/80-ifname.rules
      mode: 0644
      contents:
        inline: |
          SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="12:34:56:78:9a:bc", ATTR{type}=="1", NAME="infra"'

storage:
  files:
    - path: /etc/systemd/network/25-infra.link
      mode: 0644
      contents:
        inline: |
          [Match]
          MACAddress=12:34:56:78:9a:bc
          [Link]
          Name=infra

[dustymabe]

We discussed this in the meeting today.

It was a productive discussion, but we're still evaluating our current set of networking configuration hacks and if we want to bring something like this into the fold or if we want to draw a line somewhere. We'll take some time to do some more thought/investigation and revisit soon. Read full meeting log for more details!

[jlebon]

Had a chat with Dusty and Benjamin about this last week. I expressed some concerns about expanding the list of networking kargs we understand. Dusty pointed out that overall our automatic handling of these kargs is still necessary and is not obviated by the recent networking work like #460.

Specifically for ifname=, Dusty mentioned that it could be useful as a way to make it easier to configure other networking args. E.g. ifname=foo:MAC ip=$ip:...:foo so that it only turns on that specific interface.

Overall, I lean towards not special-handling it, so that users do have to explicitly also add a e.g. systemd link file. In RHCOS, there is at least one customer that did this. Though I don't feel strongly either way at this point.

[cgruver]

Good day all, I was just getting ready to test an iPXE boot & install with the karg being discussed here.

It's not clear from the conversation. Will this karg be honored and persisted through the installation process to rename the network interface?

Relevant code snippet below. The vars get filled in by a script which is creating an iPXE boot file named after the MAC of the boot NIC. I want that NIC to be named nic0.

ifname=nic0:${NET_MAC_0} ip=${IP_01}::${LAB_GATEWAY}:${LAB_NETMASK}:${HOSTNAME}.${LAB_DOMAIN}:nic0:none nameserver=${LAB_NAMESERVER}

[dustymabe]

We discussed this during the meeting today.

  * AGREED: We haven't seen a lot of users needing this functionality
    just yet so we'd prefer to wait to add that functionality. What we
    have for propagating networking karg information a bit hacky, but is
    required for now in order to not require every user who needs static
    networking to specify networking information twice. We'd prefer to
    not further the behavior for now.  (dustymabe, 17:25:22)

[dustymabe]

@cgruver - your feedback would have been useful before the meeting today 😄 - that being said if we do find it's a popular request it would need to be reconsidered.

If you (anyone) find yourself in this situation there are a few categories you fall in to:

• you need networking in the initramfs (i.e. remote sources for ignition configs (OKD fits here))
  • in this case you can provide ip= args and ifname=args but you'll also need to create a file with your ignition configs to persist the naming into the real root.

ifname=nic0:${NET_MAC_0} ip=${IP_01}::${LAB_GATEWAY}:${LAB_NETMASK}:${HOSTNAME}.${LAB_DOMAIN}:nic0:none

and also

storage:
  files:
    - path: /etc/systemd/network/25-nic0.link
      mode: 0644
      contents:
        inline: |
          [Match]
          MACAddress=${NET_MAC_0}
          [Link]
          Name=nic0

• you don't need networking in the initramfs
  • in this case you can just provide all networking configuration via Ignition by writing out files
  • so you would create the .link file and also files under /etc/NetworkManager/system-connections/

[dustymabe]

I'm going to close this issue since we made a decision last week. The conversation continues, though, and we'll re-open the discussion (this ticket) if we get a lot of new user input.

[usunnapu]

I am running into this issue during RHCOS installs using PXE. As always, an entry in grub.cfg expects user to specify appropriate nic enumeration which is becoming complicated. NIC enumeration is dependent on lot of factors like the slots where the card sits but also sometimes on the backplane the server has. Seen enumeration like eno8303, eno12399 etc.

It will be really helpful if as @dustymabe mentioned, when alternative alias enumeration is provided for a specific mac address in pxe menu entry, udev rules gets created. If not, what I am being forced to do now is, create a separate ignition file for each node in the cluster to ensure udev rules exists. With mac address being different on each server, dont see a way to eliminate separate ignition files.

[dustymabe]

While I was working with @usunnapu I also found that we have a gap here for PXE installs that use the coreos-installer kernel arguments to do the install like is documented at https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/#_installing_from_pxe

Basically the current documented workaround for this is to provide both the karg as well as some file (udev rule or systemd link file) via Ignition. The problem comes when you're doing an install as mentioned above. There is the install boot and the ignition boot (first boot). and in both cases you'd need to provide the filie via Ignition config. If we were to fix this issue you wouldn't need the extra install boot Ignition config.

[bgilbert]

Discussed in the community meeting today. @dustymabe is going to add a proposal in this ticket.

[dustymabe]

Previously we decided not to take action on this problem pending more user feedback. We also did not previously consider (as discovered recently) the PXE install use case where you'd need to provide two Ignition configs (one for the install boot and one for the Ignition boot) just to get networking working all the way through.

We now have had @cgruver and @usunnapu speak mention the desire for this behavior. In @usunnapu's case it seems like if NIC naming were more reliable they probably wouldn't need NIC renaming (@usunnapu is there an upstream issue for this?), but nonetheless, there are more reasons than just "unreliable NIC naming" to want to rename your NICs.

I propose:

• barring some other component of the OS taking care of NIC renaming for us (NM, systemd, udev) we,
• forward the udev rules created by ifname-genrules.sh from the 40network into the real root using the same "propagate initramfs networking" logic that we use for other networking configuration

Please let me know your thoughts (👍/👎) or propose alternative solutions to the problem. After one week we'll accept the above proposal unless significant objection is raised.

[cgruver]

I agree with the proposal. It will avoid some file gymnastics in certain cases.

I am currently using iPXE with a host specific iPXE file chained from the common one. So, I am able to provide a host specific ignition config with the systemd NIC renaming file plus other host specific configs.

If anyone is interested in the iPXE setup, ping me and I'll write up an example. It avoids the dual config scenario.

[PatrykKotlowski]

Hello all,

I'm also using ifname argument in iPXE boot. @cgruver could you describe the method you're using? Creating dual config doesn't seem to be the best option, so I'm looking for alternatives.

[cgruver]

@Giteq The tooling for my method is buried in my home lab tutorial: https://upstreamwithoutapaddle.com/home-lab/lab-intro/

The relevant pieces are in the network setup -

here: https://upstreamwithoutapaddle.com/home-lab/internal-router/

here: https://upstreamwithoutapaddle.com/home-lab/bastion-pi/

and in this helper script: https://github.com/cgruver/okd-home-lab/blob/main/bin/deployOkdNodes.sh

The helper script creates the host specific ignition configs and ipxe boot files that the FCOS system needs to boot up and install.

The helper script uses butane to modify a pre-existing ignition file and inject host specific configurations.

I will eventually write a blog post about how it works. Right now, it's buried in my tutorial.

[dustymabe]

Marking the proposal in #553 (comment) as ACCEPTED since no significant objection was raised. I'll reproduce it here:

• barring some other component of the OS taking care of NIC renaming for us (NM, systemd, udev) we,
• forward the udev rules created by ifname-genrules.sh from the 40network into the real root using the same "propagate initramfs networking" logic that we use for other networking configuration

[dustymabe]

Some more notes:

• systemd-network-generator also interprets ifname= and writes out .link files into /run/systemd/network. If we persist the udev rule written by ifname-genrules.sh (from the 40network dracut module will they conflict?
• we don't include the 01systemd-networkd dracut module in our initramfs because we don't use systemd-networkd so we don't get the .link files in the initramfs, but we do in the real root because systemd-network-generator is there in the real root
• systemd-network-generator also recently gained a new feature to be able to interpret a net.ifname-policy kernel arg for setting the NIC naming policy, which is useful.

More or less it seems like we're more and more in between a rock and a hard place where we want the udevish functionality of systemd-network-generator without using all of systemd-networkd. I tried softly to push the idea of a separating them out in systemd/systemd#21766 (comment).

[dustymabe]

PR to implement #553 (comment) is over in coreos/fedora-coreos-config#2411

[dustymabe]

The fix for this went into next stream release 38.20230514.1.0. Please try out the new release and report issues.

[dustymabe]

The fix for this went into testing stream release 38.20230514.2.0. Please try out the new release and report issues.

[dustymabe]

The fix for this went into stable stream release 38.20230514.3.0.