refactor: flat search to use datafusion top k

[eddyxu]

Closes #1324

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 76.57658% with 26 lines in your changes missing coverage. Please review.

Project coverage is 79.78%. Comparing base (2926fde) to head (2696b8c).
Report is 1 commits behind head on main.

Files	Patch %	Lines
rust/lance-index/src/vector/flat.rs	18.18%	6 Missing and 3 partials ⚠️
rust/lance/src/dataset/scanner.rs	67.85%	1 Missing and 8 partials ⚠️
rust/lance/src/io/exec/knn.rs	88.88%	3 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2535      +/-   ##
==========================================
- Coverage   79.80%   79.78%   -0.02%     
==========================================
  Files         207      207              
  Lines       59636    59569      -67     
  Branches    59636    59569      -67     
==========================================
- Hits        47591    47529      -62     
+ Misses       9261     9254       -7     
- Partials     2784     2786       +2     

Flag	Coverage Δ
unittests	79.78% <76.57%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[BubbleCal]

NIT: this method name doesn't match what it does since this PR

[eddyxu]

ok, fixed.