wildcard filterlists support like adblockers

[pokegamer5547]

Describe a related problem (optional)

There are some few filterlists, like https://oisd.nl, decided to drop support for the hosts file due to it not able to detect unknown sub domains. Which raises a good question.

While we can use pattern checker for asterisks when using domains, same can't be said for hosts as they only check for main domain and if we click on link which has unknown sub domain, it wont get detected by urlchecker.

Describe your suggested feature

If we can use something similar to how most adblockers do, like using a filterlists which supports wildcards, or following typical adblock syntax, it has few advantages

1. Slight or large difference in size (depending on source) of filterlists due to their syntax automatically blocking all subdomains as well and not manually including each subdomain.
2. We can use something like dynamicdns filterlist and get a warning whenever a site is using a dynamicdns in url or any malicious url. (They can be risky too)

Describe alternatives you've considered for your suggested feature

Or simply add option to treat entries in hosts file which doesn't have a subdomain as if they are having a asterisks at beginning so they would detect all subdomains as well.

Other details

Implementing similar to how adblockers do can be hard. While we can add these filterlists on browsers like brave or firefox (with extension) easily, it would be nice to have it work in urlcheck as i use hosts labeller to prevent my normie family not get into clicking weird links and wish it supported sub domains too.

Acknowledgements

• I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
• I have written a short but informative title.
• I will fill out all of the requested information in this form.

[TrianguloY]

I made it so that the host module checks subdomains too. So that a host like a.b.c will check a.b.c, then b.c then c, and report the first to match (if any).

That means you need to use a host file without subdomains, including for example b.c (but NOT "*.b.c" as that won't work)

[Murilogs1910]

That means you need to use a host file without subdomains, including for example b.c (but NOT "*.b.c" as that won't work)

Doubt that'll be a problem. Many lists, oisd.nl's included, have a version without the wildcards.

[pokegamer5547]

Actually i have a question. whats the syntax which hosts module actually uses?
Is it the typical syntax of 0.0.0.0 domain.com ? Or does it also woks even without ip address so the syntax domain.com would work directly for the module?
If not, can you please make it work directly like that?
For example this file
https://small.oisd.nl/domainswild2
I'm asking because oisd.nl doesn't provide hosts file like filterlist. And others which do often includes subdomains in host file as well.

[Murilogs1910]

I am 95% sure it also works without the local ip before the domains. I'll still check tho and, if it works, I'll react to your comment with a thumbs up.

[TrianguloY]

The current version is checking lines with exactly two entries separated by spaces, then using the second one. This was technically wrong (tabs are also valid, and some files contain more than one host on the same line) so I fixed it and added the 'single host' variation to the mix.

Now the files can be for example:

0.0.0.0 example.com
0.0.0.0          example.com
0.0.0.0	example.com # this is a tab character
0.0.0.0 example.com anotherexample.com
example.com

[pokegamer5547]

I am 95% sure it also works without the local ip before the domains. I'll still check tho and, if it works, I'll react to your comment with a thumbs up.

Thankyou so much!

The current version is checking lines with exactly two entries separated by spaces, then using the second one. This was technically wrong (tabs are also valid, and some files contain more than one host on the same line) so I fixed it and added the 'single host' variation to the mix.

Now the files can be for example:

0.0.0.0 example.com
0.0.0.0          example.com
0.0.0.0	example.com # this is a tab character
0.0.0.0 example.com anotherexample.com
example.com

Oh i see, thankyou so much for fixing so quickly! So all the syntaxs you specified in that now be used right? Including just a main domain matching all subdomains of it too?

Also, is there a reason why autoupdate for hosts module isn't a thing yet?

[TrianguloY]

Oh i see, thankyou so much for fixing so quickly! So all the syntaxs you specified in that now be used right? Including just a main domain matching all subdomains of it too?

yes, well, it will be on the next version...whenever that is published.
I just tested the file you linked and it works as expected

Also, is there a reason why autoupdate for hosts module isn't a thing yet?

because the update will take a long time, and android freaks out when you do a lot of work on a background task. I think I need to maybe use a background service, I'm not sure. I also would like to implement a way to 'cancel' the build process...but async operations are hard and there are other more fun things to do :(

[pokegamer5547]

yes, well, it will be on the next version...whenever that is published. I just tested the file you linked and it works as expected

Thankyou! :) i'm looking forward to the next release.

because the update will take a long time, and android freaks out when you do a lot of work on a background task. I think I need to maybe use a background service, I'm not sure. I also would like to implement a way to 'cancel' the build process...but async operations are hard and there are other more fun things to do :(

Ah i see, it would be cool to have but true i can relate what you mean. It's not easy also considering how many OEMs "battery optimizations" kills apps unless they use google push without going through extensive settings.

I was thinking this way

1. have setting which specities how long it should wait till it tries to autoupdate
2. Once the time is reached where previous update is considered old, when a user clicks on a link, urlchecker in background in seperate process update stuff with a timeout.

Well this conversation is getting out of scope of this issue ig sorry 😅
I tried cloning and building the app myself to see if i can implement it but i'm unable to buid for some reason. Gradle says

unsupported method: ModelBuilderParameter.getDontBuildHostTestRuntimeClasspath(). The version of Gradle you connect to does not support that method. To resolve the problem you can change/upgrade the target version of Gradle you connect to. Alternatively, you can ignore this exception and read other information from the model

[TrianguloY]

I have no idea what that error is :/

But yeah, let's continue on a discussion if you don't mind. Can you create it?

[pokegamer5547]

I have no idea what that error is :/

But yeah, let's continue on a discussion if you don't mind. Can you create it?

It's ok, i think my androidstudio is kinda broken. It doesnt seems like an issue with urlchecker repo only. i'm still new to all this development. i'll figure it out.
thankyou so much :)