192.0.0.8 (IPv4 dummy address) considered globally reachable

[CDirkx]

BPO	42937
Nosy	@mjpieters, @CDirkx, @wjohnston99
PRs	bpo-42937: Update ipaddress.py to include additional private range of IPs #26205

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-01-15.23:00:38.626>
labels = ['type-bug', 'library', '3.11']
title = '192.0.0.8 (IPv4 dummy address) considered globally reachable'
updated_at = <Date 2021-05-24.16:57:26.598>
user = 'https://github.com/CDirkx'

bugs.python.org fields:

activity = <Date 2021-05-24.16:57:26.598>
actor = 'CosmicKid'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-01-15.23:00:38.626>
creator = 'cdirkx'
dependencies = []
files = []
hgrepos = []
issue_num = 42937
keywords = ['patch']
message_count = 6.0
messages = ['385127', '393843', '393862', '393863', '393870', '394256']
nosy_count = 3.0
nosy_names = ['mjpieters', 'cdirkx', 'CosmicKid']
pr_nums = ['26205']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42937'
versions = ['Python 3.11']

[CDirkx]

Currently the method ipaddress.is_global returns true for the IPv4 address 192.0.0.8. This was correct when is_global was initially implemented, but in 2015 192.0.0.8 got designated as the "IPv4 dummy address" and not globally reachable, see the IANA special purpose address registry: https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml.

The correct behaviour now, according to the documentation of is_global which refers to the IANA registry, would be to return false for 192.0.0.8.

[wjohnston99]

This a simple matter of adding the address to the _private_networks constant. I will add it and issue a pull request today/tomorrow. I could add some of the others too for completeness. However, these are really special use versus private. Private is a subset of special use. Should a "_special_use" constant be created. This would include multicast, link_local, private_use, and a few more.

There are many others included in https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml.

[mjpieters]

This is related to bpo-42937, the IPv4 private network list is not considering the whole of 192.0.0.0/24 to be private.

RFC 5736 / 6890 reserved 192.0.0.0/24 for special purposes (private networks) and to date a few subnets of that network have received assignments. The ipaddress modules should use that subnet for any is_private test, and not just the subnets of that network that have received specific assignments.

E.g. the list currently contains just 192.0.0.0/29 and 192.0.0.170/31, but as this bug report points out, 192.0.0.8/32 has since been added, as have 192.0.0.9/32 and 192.0.0.10/32.

The IPv6 implementation *does* cover the whole reserved subnet (although it also includes 2 specific registrations, see the aforementioned bpo-42937), it is just IPv4 that is inconsistent and incomplete here.

[mjpieters]

Oops, I got my issue numbers mixed up. This is related to bpo-44167, I meant.

[mjpieters]

Private is a subset of special use. Should a "_special_use" constant be created. This would include multicast, link_local, private_use, and a few more.

There are already dedicated tests for those other special use networks in ipaddress. 192.0.0.0/24 is the block reserved for "IETF Protocol Assignments", which really means: private use. https://datatracker.ietf.org/doc/html/rfc6890#section-2.2.2 marks the block as "Not usable unless by virtue of a more specific reservation.".

The registry at https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml lists those specific reservations, and only 2 to date are *globally reachable*, which means they are probably not private:

• 192.0.0.9/32, Port Control Protocol Anycast, RFC 7723
• 192.0.0.10/32, Traversal Using Relays around NAT Anycast, RFC 8155

I strongly feel that _any other IP address in the reserved range_ should be treated as private unless marked, by IANA, as globally reachable, at some future date.

That would require the list of networks for IPv4Address / IPv4Network is_private to include all of 192.0.0.0/24 _minus those two exceptions_; calculating the network masks for these:

>>> def exclude_all(network, *excluded):
...     try:
...         for sub in network.address_exclude(excluded[0]):
...             yield from exclude_all(sub, *excluded[1:])
...     except (IndexError, ValueError):
...         yield network
...
>>> iana_reserved = IPv4Network("192.0.0.0/24")
>>> to_remove = IPv4Network("192.0.0.9/32"), IPv4Network("192.0.0.10/32")

>>> for sub in exclude_all(iana_reserved, *to_remove):
...     print(sub)
...
192.0.0.128/25
192.0.0.64/26
192.0.0.32/27
192.0.0.16/28
192.0.0.0/29
192.0.0.12/30
192.0.0.11/32
192.0.0.8/32

The module could trivially do this on import, or we could hard-code the above list.

[wjohnston99]

I completely agree with comments about .is_global and special_use. Private IPs are just a subset of special use. For some part Private means the Private use networks reserved by IANA and specified in IETF RFC1918.

For this PR "IPv4 dummy address" is for sure not global. Technically, it is not a private use IP either. Just following what others have done. I really like the apporoach of removing the routable special IPs. Let me know if I should create a PR to do this. This would include adding specific special use constant plus a whole bunch of very specific constants. A constant like is_dummy will be added.

I pretty much have the code ready to go for IPV4.

[tsvsj]

I'm really surprised by the decisions, made within this issue and during the development, when it comes to the "grouping" of the different address spaces. Even if the arguments are correct on an academic low level, they are not correct when it comes to the practical use of those address spaces.

Private should never be considered to be outside of RFC1918. The IANA flagged those address spaces as "Private-Use". As a result, is_private should only check for 10.0.0.0/8, 192.168.0.0/16 and 172.16.0.0/12. These are the only private address spaces.
If you add all non-routable spaces as well (Globally Reachable = False), there are overlaps with the loopback spaces as well. This leads to confusion and wrong assignments from my point of view.

I'm coming from a troubleshooting, where code looked like that:

tempIp = ipaddress.ip_address("127.0.0.1")

if tempIp.is_private:
  ...
elif tempIp.is_global:
  ...
elif tempIp.is_unspecified:
  ...
elif tempIp.is_reserved:
  ...
elif tempIp.is_loopback:
  ...
elif tempIp.is_multicast:                            
  ...
elif tempIp.is_link_local:
  ...

Imagine my confusion, when it turned out to leave is_private. I'm pretty sure, that I'm not the only one, who'll get confused by this behavior.

By the checking the code

cpython/Lib/ipaddress.py

Lines 1551 to 1566 in f1f85a4

	_private_networks = [
	IPv4Network('0.0.0.0/8'),
	IPv4Network('10.0.0.0/8'),
	IPv4Network('127.0.0.0/8'),
	IPv4Network('169.254.0.0/16'),
	IPv4Network('172.16.0.0/12'),
	IPv4Network('192.0.0.0/29'),
	IPv4Network('192.0.0.170/31'),
	IPv4Network('192.0.2.0/24'),
	IPv4Network('192.168.0.0/16'),
	IPv4Network('198.18.0.0/15'),
	IPv4Network('198.51.100.0/24'),
	IPv4Network('203.0.113.0/24'),
	IPv4Network('240.0.0.0/4'),
	IPv4Network('255.255.255.255/32'),
	]

I found the issue here. In my case I will re-arrange the conditions, so that is_private is the last statement, but in general it makes sense to change this (for the most people) unintended behavior. I support the thought of @wjohnston99 in #87103 (comment) to have a separate constant _special_use, along with a separate method is_special, to summarize all those address spaces, that are mostly not in use at all, like those dummy addresses.

[jstasiak]

@tsvsj Having gone through this code recently I agree with you.

There's a ticket about documenting things so the confusion is avoided somewhat: #65056

In that ticket there's a comment #65056 (comment) that suggests the following course of action

(...) suggest the following specification:

• is_global returns False for all addresses where "Global" is "False" in the IPv4 or IPv6 Special-Purpose Address Registry
• a new method is_private_use is introduced, giving True only for the RFC1918 addresses
• a new method is_unique_local is introduced, giving True only for the RFC 4193 addresses.
• is_private is deprecated. Alternatively, it could be preserved and documented to being the union of is_privte_use and is_unique_local.

But either way I think it's out of the scope of this issue.