Skip to content
This repository has been archived by the owner on May 9, 2022. It is now read-only.

TY-1686 Analytics #54

Merged
merged 11 commits into from
Apr 27, 2021
Merged

TY-1686 Analytics #54

merged 11 commits into from
Apr 27, 2021

Conversation

rustonaut
Copy link
Contributor

@rustonaut rustonaut commented Apr 26, 2021

References

Summary

  • implement nDCG@k
  • use it to create analytics based on various other implied "rankings":
    • LTR score
    • Context score
    • final rank
    • initial rank (different PR)

@rustonaut rustonaut marked this pull request as draft April 26, 2021 10:04
@rustonaut rustonaut marked this pull request as ready for review April 26, 2021 10:39
@rustonaut
Copy link
Contributor Author

There seem to be a mismatch between CI cargo clippy and local cargo clippy I will look into this.

@rustonaut
Copy link
Contributor Author

I ran into this rust-lang/rust-clippy#4612, which is fixed but not yet on stable. I'm more surprised that I haven't ran into this before tbh., I guess mainly because I only ran clippy before releases in previous projects, for which I always also ran cargo clean.

As a side note I noticed that currently clippy nightly warns about some committed code, we might want to fix that to prevent annoyances when that warnings land on stable.

@rustonaut
Copy link
Contributor Author

error: could not download file from 'https://static.rust-lang.org/rustup/release-stable.toml' to '/tmp/rustup-updateK6Yz3L/release-stable.toml'
error: caused by: failed to make network request
error: caused by: error sending request for url (https://static.rust-lang.org/rustup/release-stable.toml): connection error: Connection reset by peer (os error 104)
error: caused by: connection error: Connection reset by peer (os error 104)
error: caused by: Connection reset by peer (os error 104)

So the rust server has problems?

Currently this is not implemented for initial ranking as it
is currently not accessible. (Defaults to NaN.)

Some parts are currently implemented in a performance wise
suboptimal way.
Tests now use the test_ prefix and are not in
sub-modules.
@Robert-Steiner
Copy link
Contributor

error: could not download file from 'https://static.rust-lang.org/rustup/release-stable.toml' to '/tmp/rustup-updateK6Yz3L/release-stable.toml'
error: caused by: failed to make network request
error: caused by: error sending request for url (https://static.rust-lang.org/rustup/release-stable.toml): connection error: Connection reset by peer (os error 104)
error: caused by: connection error: Connection reset by peer (os error 104)
error: caused by: Connection reset by peer (os error 104)

So the rust server has problems?

Unfortunately, this happens sometimes. It can also be due to a github action issue. Try to re-run the workflow 🙂

@rustonaut
Copy link
Contributor Author

Thanks, I will. I anyway have to solve the minor merge conflict which popped up 😉

xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
xayn-ai/src/analytics.rs Show resolved Hide resolved
xayn-ai/src/analytics.rs Show resolved Hide resolved
@acrrd
Copy link
Contributor

acrrd commented Apr 26, 2021

As a side note I noticed that currently clippy nightly warns about some committed code, we might want to fix that to prevent annoyances when that warnings land on stable.

We talked about having a ci to check nightly but we didn't do it yet. In the ci we use a fixed rust version exactly to avoid that a new warning will suddenly break evreything.

This includes a test of the `compute_analytics` methods,
and a port of the test from the prev. dart implementation
(as far as applicable).
- skip if no history available for a document
- if we have no relevant historic information at all
  we do return an error
- use fold in dcg
- in pick highest skip search if we won't be able to insert
  it anyway
- use Iterator::fuse() instead of FusedIterator
The new function is called nan_safe_f32_cmp.
@rustonaut
Copy link
Contributor Author

There is one small commit missing, coming in less then 3min 😉 .

assert!(approx_eq!(f32, left, right, ulps=n)) has the
problem that it doesn't print information about the
values when it fails so assert_f32_eq adds a format
string which on failure will be used as error message.
@rustonaut
Copy link
Contributor Author

Ok that was all. I already had used that helper in some places when the assertions failed
due to the ln/log2 mismatch. So I thought it's best to properly add it.

@rustonaut rustonaut mentioned this pull request Apr 27, 2021
2 tasks
@acrrd acrrd self-requested a review April 27, 2021 11:48
xayn-ai/src/analytics.rs Outdated Show resolved Hide resolved
@rustonaut rustonaut merged commit 4e6400a into master Apr 27, 2021
@rustonaut rustonaut deleted the TY-1686-analytics branch April 27, 2021 12:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants