Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-386 IP categorization #359

Merged
merged 2 commits into from
Jan 9, 2023
Merged

NETOBSERV-386 IP categorization #359

merged 2 commits into from
Jan 9, 2023

Conversation

jotak
Copy link
Member

@jotak jotak commented Jan 3, 2023

JIRA: https://issues.redhat.com/browse/NETOBSERV-386

Allows to configure IP ranges and assign them any name
IPs falling in those ranges are flagged with that name

Allows to configure IP ranges and assign them any name
IPs falling in those ranges are flagged with that name
@jotak jotak changed the title PoC for IP categorization NETOBSERV-386 IP categorization Jan 3, 2023
@jotak jotak marked this pull request as ready for review January 3, 2023 11:05
@jotak jotak requested a review from mariomac January 3, 2023 11:05
Comment on lines 168 to 178
func (n *Network) categorizeIP(ip net.IP) string {
for _, subnetCat := range n.categories {
for _, cidr := range subnetCat.cidrs {
if cidr.Contains(ip) {
return subnetCat.name
}
}
}
return ""
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid this exhaustive search could increase noticeably the CPU usage. I'd add an lru cache implementation (like this or this) and check there the IP first.
In pseudo-golang:

func (n *Network) categorizeIP(ip net.IP) string {
	if category, ok := n.cache.Get(ip); ok {
		return category
	}
	for _, subnetCat := range n.categories {
		for _, cidr := range subnetCat.cidrs {
			if cidr.Contains(ip) {
				n.cache.Put(ip, subnetCat.name)
				return subnetCat.name
			}
		}
	}
	n.cache.Put(ip, "")
	return ""
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I have the same concern and wanted to point out in the documentation (when this is implemented on operator side) that this could be disabled to save processor's resource.
But +1 about implementing a cache right now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done
I took the opportunity to update a little bit TimedCache in order to use go's time API + a couple of small stuffs

- Cache categorized IPs to avoid too much IP parsing & matching
- Update TimedCache to use time API rather than int64
- Use lambda-style cleanup rather than an interface
@codecov-commenter
Copy link

Codecov Report

Merging #359 (4b328b7) into main (fa0662e) will increase coverage by 0.18%.
The diff coverage is 84.61%.

@@            Coverage Diff             @@
##             main     #359      +/-   ##
==========================================
+ Coverage   69.05%   69.23%   +0.18%     
==========================================
  Files          88       88              
  Lines        5157     5204      +47     
==========================================
+ Hits         3561     3603      +42     
- Misses       1378     1382       +4     
- Partials      218      219       +1     
Flag Coverage Δ
unittests 69.23% <84.61%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/api/transform_network.go 83.33% <ø> (ø)
pkg/pipeline/extract/aggregate/aggregate.go 95.86% <ø> (ø)
pkg/pipeline/utils/timed_cache.go 95.60% <80.00%> (+2.09%) ⬆️
pkg/pipeline/transform/transform_network.go 72.67% <83.33%> (+2.20%) ⬆️
pkg/pipeline/encode/encode_prom.go 77.36% <100.00%> (ø)
pkg/pipeline/extract/aggregate/aggregates.go 90.19% <100.00%> (-0.19%) ⬇️
.../pipeline/transform/transform_network_direction.go 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

svcNames: servicesDB,
svcNames: servicesDB,
categories: subnetCats,
ipCatCache: utils.NewQuietExpiringTimedCache(2 * time.Minute),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this timeout be configurable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's important to make it configurable, it won't have any impact on the results (unlike conn-tracking timeout, for instance, where timeout impacts updates granularity) - this one is really an implementation detail

@jotak jotak merged commit 6a8b986 into netobserv:main Jan 9, 2023
@jpinsonneau jpinsonneau added the breaking-change This pull request has breaking changes. They should be described in PR description. label Feb 16, 2023
@jpinsonneau
Copy link
Collaborator

NetworkTransformDirectionInfo is a breaking change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change This pull request has breaking changes. They should be described in PR description.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants