Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix columns duplication on MongoDB Query Runner #6640 #6641

Merged
merged 8 commits into from
Aug 1, 2024
Merged

Conversation

masayuki038
Copy link
Collaborator

What type of PR is this?

  • Bug Fix

Description

Bug fix for #6640.
And added some tests for MongoDB Query Runner.

How is this tested?

  • Unit tests (pytest, jest)
  • E2E Tests (Cypress)
  • Manually
  • N/A

Related Tickets & Documents

Closes #6640

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

without fields

image

with fields

image

Copy link

codecov bot commented Dec 3, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (939bec2) 63.42% compared to head (8ea665e) 63.72%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6641      +/-   ##
==========================================
+ Coverage   63.42%   63.72%   +0.29%     
==========================================
  Files         162      162              
  Lines       13173    13174       +1     
  Branches     1819     1820       +1     
==========================================
+ Hits         8355     8395      +40     
+ Misses       4522     4473      -49     
- Partials      296      306      +10     
Files Coverage Δ
redash/query_runner/mongodb.py 60.09% <100.00%> (+19.03%) ⬆️

konnectr
konnectr previously approved these changes Jan 27, 2024
Copy link
Contributor

@guidopetri guidopetri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@masayuki038 thanks for this fix. It looks like two of the tests aren't passing for it? Could you take a look and investigate why?

@masayuki038
Copy link
Collaborator Author

@guidopetri @konnectr Thanks for your review.

It looks like two of the tests aren't passing for it? Could you take a look and investigate why?

Sure. I missed something. I'll check it later. Please give me a little time.

"type": TYPES_MAP.get(type(value), TYPE_STRING),
}
)
if _get_column_by_name(columns, column_name) is None:
Copy link

@clearnote01 clearnote01 Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First of all thanks for the fix on this, I was having this problem recently : D

In here, to make this more efficient, can't we just get the first row, and then generate the columns array from that outside for this for loop?

In this case every cell in the table requires this comparison which is in-efficient and whether we check row1 or rowN the columns will be same.

Copy link
Collaborator Author

@masayuki038 masayuki038 Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clearnote01 I understand your comment. I also think that if it is normal RDB data, I should do that.

I thought that MongoDB can have a different column for each row, so I implemented it like this. Does such a case not exist?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@masayuki038 You are right, I don't think it's a problem when projection is specifically configured via $project, but there could be other cases that would break it.

Maybe it's safer to keep it as it is, and have a note that this could be optimized in the future if someone can rule out the edge cases.

@masayuki038
Copy link
Collaborator Author

e2e test still failed. I don't know main branch status...
I will re-run it after #6748 is fixed.

@masayuki038
Copy link
Collaborator Author

e2e test passed by merging main branch.
@guidopetri Could you check this if you have some time? Thank you always.

@MrSpark2591
Copy link

We are having a similar issue right now. Can you guys let me know if I can help in anyways?

@jagal17
Copy link

jagal17 commented May 6, 2024

Hi, any news about this? I have also the same problem all the columns of my table are duplicated since I upgraded to the last preview version.

@jagal17
Copy link

jagal17 commented Jun 21, 2024

@clearnote01 @guidopetri any news about this merge? Please we need it!

@masayuki038
Copy link
Collaborator Author

@justinclift
It looks like that @guidopetri is busy... I fixed some tests. Could you review this or assign a new reviewer? @jagal17 seems to need this update. Thanks always.

@masayuki038
Copy link
Collaborator Author

Sorry, something is wrong. I will check it. Please wait.

@masayuki038
Copy link
Collaborator Author

masayuki038 commented Jul 30, 2024

I fixed the merge issue. All checks have passed.

@justinclift
It looks like that @guidopetri is busy... I fixed some tests. Could you review this or assign a new reviewer?

@justinclift
Copy link
Member

@wlach Do you have time/interest to look over this one? 😄

@justinclift
Copy link
Member

@eradman Do you have time to look over this one? It seems a bit more urgent than some of the other PRs around, as people are emailing me directly about this one to ask. 😉

@eradman
Copy link
Collaborator

eradman commented Aug 1, 2024

@eradman Do you have time to look over this one? It seems a bit more urgent than some of the other PRs around, as people are emailing me directly about this one to ask. 😉

I don't have enough knowledge about MongoDB to provide useful input on this. Perhaps merge it and ask for feedback?

@justinclift
Copy link
Member

Good idea, lets do that. 😄

Copy link
Member

@justinclift justinclift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @eradman said, we'll merge this.

@jagal17 Are you ok to verify a build (with this merged) works for you?

@justinclift justinclift dismissed guidopetri’s stale review August 1, 2024 22:04

Things have been changed. :)

@justinclift justinclift enabled auto-merge (squash) August 1, 2024 22:04
@justinclift
Copy link
Member

@masayuki038 Thanks for taking the time to get this over the line. 😄

@justinclift justinclift merged commit ed8c05f into master Aug 1, 2024
14 checks passed
@justinclift justinclift deleted the issue-6640 branch August 1, 2024 22:34
@masayuki038
Copy link
Collaborator Author

@justinclift @eradman Thanks for your reviews! I checked it on local, but let me know if someone reported some issues around this. I am checking github every day 😄 Thank you.

@justinclift
Copy link
Member

Will do @masayuki038. 😄

@jagal17
Copy link

jagal17 commented Aug 3, 2024

@justinclift @masayuki038 it is working very well now! thank you very much guys!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MongoDB Query Runner displays duplicated columns
8 participants