Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐋 Dockerfile migrations #77

Merged
merged 4 commits into from
Jul 6, 2023
Merged

🐋 Dockerfile migrations #77

merged 4 commits into from
Jul 6, 2023

Conversation

mrharpo
Copy link
Contributor

@mrharpo mrharpo commented Jul 6, 2023

Migrations

Adds migration files to Dockerfile

Test variables

Fixes config import order for pytest.

  • Tests were trying to connect to db chowda-development
  • Fixed by moving import below environ['CHOWDA_ENV'] = 'test'

CI

Adds correct DB_URL env to point to postgres service

@mrharpo mrharpo added CD 🏗️ Relating to Continuous Deployment bug 🐛 Something isn't working labels Jul 6, 2023
@mrharpo mrharpo added this to the v0.3 Pipeline 🚰 milestone Jul 6, 2023
@mrharpo mrharpo requested a review from afred July 6, 2023 18:48
@mrharpo mrharpo self-assigned this Jul 6, 2023
@mrharpo mrharpo temporarily deployed to Codecov July 6, 2023 19:08 — with GitHub Actions Inactive
@mrharpo mrharpo temporarily deployed to Codecov July 6, 2023 19:09 — with GitHub Actions Inactive
@codecov-commenter
Copy link

Codecov Report

Merging #77 (cf824b5) into main (9115917) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #77   +/-   ##
=======================================
  Coverage   82.87%   82.87%           
=======================================
  Files           9        9           
  Lines         292      292           
=======================================
  Hits          242      242           
  Misses         50       50           
Impacted Files Coverage Δ
chowda/db.py 80.00% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@mrharpo mrharpo added the CI 🦾 Relating to Continuous Integration label Jul 6, 2023
@@ -2,6 +2,9 @@ name: 🧪 Integration Tests

on: [push, pull_request, workflow_dispatch]

env:
DB_URL: postgresql://postgres:postgres@postgres:5432/chowda-test

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think DB_URL will get overridden when the tests are run, i.e.

  1. CHOWDA_ENV gets set to "test" automatically whenever running pytest
  2. the .env.test is loaded, which contains the DB_URL

at least that's how it's supposed to work. Is it erroring without this explicit env var setting?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and:

  • .env.test contains test values for testing from a local machine (db: localhost)
  • For CI, we override that with the url for the postgres service
    • Because we removed override=True, we can just set this variable, instead of:
      • Create .env.ci with DB_URL
      • Set CHOWDA_ENV=CI
      • Add a switch case in conftest.py for CI

Copy link
Contributor Author

@mrharpo mrharpo Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug

When I fixed the CI file, the tests had this bug: https://github.com/WGBH-MLA/chowda/actions/runs/5479062625/jobs/9980381904

E sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: FATAL: database "chowda-development" does not exist

Import order

Fixing the import order in conftest.py then used the correct dotfile: https://github.com/WGBH-MLA/chowda/actions/runs/5479131791/jobs/9980556927

E sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: FATAL: database "chowda-test" does not exist

Override

Setting env: DB_URL successfully connected to the db and ran the tests ✅ : https://github.com/WGBH-MLA/chowda/actions/runs/5479229000/jobs/9980783510

Copy link
Contributor

@afred afred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@afred afred merged commit 715db40 into main Jul 6, 2023
8 checks passed
@afred afred deleted the docker-migrations branch July 6, 2023 20:58
@mrharpo mrharpo linked an issue Jul 6, 2023 that may be closed by this pull request
mrharpo added a commit that referenced this pull request Jul 7, 2023
* main:
  🐋 Dockerfile migrations (#77)
  💽 `postgresql://` in all environments (#73)
mrharpo added a commit that referenced this pull request Jul 7, 2023
* docs:
  Removes old note
  Fixes DB_NAME env
  Fixes lint errors
  Changes SonyCiAsset.id to int
  🐋 Dockerfile migrations (#77)
  💽 `postgresql://` in all environments (#73)
  Removes dotenv override (#72)
  🤖 Update dependencies (#61)
  👣 Migration tool (#62)
  Adds on: workflow_dispatch to all actions (#63)
  Updates another workflow pointer (#60)
  ☝️ Updates workflow pointer (#59)
  Sets chowda db name for CI to 'chowda'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Something isn't working CD 🏗️ Relating to Continuous Deployment CI 🦾 Relating to Continuous Integration
Projects
No open projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

Add database migrations to deployment
3 participants