Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[COST-5299] clear cache when table is dropped by migration #5221

Merged
merged 2 commits into from
Jul 18, 2024

Conversation

maskarb
Copy link
Member

@maskarb maskarb commented Jul 18, 2024

Jira Ticket

COST-5299

Description

This change will delete cache values when we drop tables via trino migration.

Testing

  1. clear db:
$ make delete-test-customer-data; make delete-testing; make delete-trino-data; make delete-redis-cache
  1. check redis cache for key:
(key this shell open)
$ docker exec -it masu_server sh
$ python koku/manage.py shell
>>> from koku.cache import build_trino_table_exists_key
>>> from koku.cache import is_key_in_cache
>>> key = build_trino_table_exists_key('org1234567', 'openshift_namespace_labels_line_items')
>>> is_key_in_cache(key)
False
  1. create customer; ingest OCP on prem:
$ make create-test-customer; make load-test-customer-data test_source=ONPREM 
  1. check redis cache for key:
>>> is_key_in_cache(key)
True
  1. run trino migration:
(in another shell)
$ docker exec -it masu_server sh
$ python koku/manage.py migrate_trino_tables --drop-tables openshift_namespace_labels_line_items
...
[2024-07-18 16:19:03,370] INFO None 40583 Running against the following schemas: org1234567
[2024-07-18 16:19:03,370] INFO None 40583 Dropping tables ['openshift_namespace_labels_line_items'] for org1234567 (1 / 1)
[2024-07-18 16:19:03,370] INFO None 40583     org1234567: Dropping table openshift_namespace_labels_line_items
[2024-07-18 16:19:03,394] INFO None 40583     org1234567: DROP TABLE result: []
[2024-07-18 16:19:03,401] INFO None 40583     org1234567: delete cache key table-exists-trino-org1234567-openshift_namespace_labels_line_items: True

if the key does not exist in cache, we log False:

[2024-07-18 16:38:04,322] INFO None 1764     org1234567: delete cache key table-exists-trino-org1234567-openshift_namespace_labels_line_items: False
  1. check cache for key:
>>> is_key_in_cache(key)
False

Release Notes

  • proposed release note
* [[COST-5299](https://issues.redhat.com/browse/COST-5299)] clear cache when table is dropped by migration

@maskarb maskarb added the smoke-tests pr_check will build the image and run minimal required smokes label Jul 18, 2024
@maskarb
Copy link
Member Author

maskarb commented Jul 18, 2024

/retest

@maskarb maskarb changed the title clear cache when table is deleted by migration [COST-5299] clear cache when table is dropped by migration Jul 18, 2024
@maskarb maskarb marked this pull request as ready for review July 18, 2024 16:03
@maskarb maskarb requested review from a team as code owners July 18, 2024 16:03
Copy link

codecov bot commented Jul 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.1%. Comparing base (ef60c15) to head (44b332a).

Additional details and impacted files
@@           Coverage Diff           @@
##            main   #5221     +/-   ##
=======================================
- Coverage   94.1%   94.1%   -0.0%     
=======================================
  Files        373     373             
  Lines      31037   31040      +3     
  Branches    4564    4564             
=======================================
+ Hits       29212   29213      +1     
- Misses      1162    1165      +3     
+ Partials     663     662      -1     

@maskarb maskarb enabled auto-merge (squash) July 18, 2024 17:04
@maskarb maskarb merged commit 0d7c445 into main Jul 18, 2024
11 checks passed
@maskarb maskarb deleted the clear-cache-when-table-deleted branch July 18, 2024 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
smoke-tests pr_check will build the image and run minimal required smokes smokes-required
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants