Skip to content

Commit

Permalink
Added a tap test case to load multiple PPG extensions in the server b…
Browse files Browse the repository at this point in the history
…efore running a test load. (#456)

1. We load and create other extensions that are distributed by Percona
   in PPG. (postgis, pg_repack, pgaudit, pgaudit_set_user, pgpool)
2. Run test data with pgbench.

To make the above test case work, updated the workflows to install above mentioned extension
where we use installed packages from PPG. On workflows where we build server or use packages
from PGDG, we are skipping this test case.
  • Loading branch information
Naeem-Akhter committed Apr 22, 2024
1 parent c2923b4 commit 5d7c424
Show file tree
Hide file tree
Showing 7 changed files with 158 additions and 10 deletions.
11 changes: 9 additions & 2 deletions .github/workflows/postgresql-12-ppg-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,19 @@ jobs:
https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo dpkg -i percona-release_latest.generic_all.deb
- name: Install Percona Distribution Postgresql 12
- name: Install Percona Distribution Postgresql 12 & Extensions
run: |
sudo percona-release setup ppg-12
sudo apt-get update -y
sudo apt-get install -y percona-postgresql-12 \
percona-postgresql-contrib percona-postgresql-server-dev-all
percona-postgresql-contrib percona-postgresql-server-dev-all \
percona-pgpool2 libpgpool2 percona-postgresql-12-pgaudit \
percona-postgresql-12-pgaudit-dbgsym percona-postgresql-12-repack \
percona-postgresql-12-repack-dbgsym percona-pgaudit12-set-user \
percona-pgaudit12-set-user-dbgsym percona-postgresql-12-postgis-3 \
percona-postgresql-12-postgis-3-scripts \
percona-postgresql-postgis-scripts percona-postgresql-postgis \
percona-postgis
- name: Change src owner to postgres
run: |
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/postgresql-13-ppg-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,19 @@ jobs:
https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo dpkg -i percona-release_latest.generic_all.deb
- name: Install Percona Distribution Postgresql 13
- name: Install Percona Distribution Postgresql 13 & Extensions
run: |
sudo percona-release setup ppg-13
sudo apt-get update -y
sudo apt-get install -y percona-postgresql-13 \
percona-postgresql-contrib percona-postgresql-server-dev-all
percona-postgresql-contrib percona-postgresql-server-dev-all \
percona-pgpool2 libpgpool2 percona-postgresql-13-pgaudit \
percona-postgresql-13-pgaudit-dbgsym percona-postgresql-13-repack \
percona-postgresql-13-repack-dbgsym percona-pgaudit13-set-user \
percona-pgaudit13-set-user-dbgsym percona-postgresql-13-postgis-3 \
percona-postgresql-13-postgis-3-scripts \
percona-postgresql-postgis-scripts percona-postgresql-postgis \
percona-postgis
- name: Change src owner to postgres
run: |
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/postgresql-14-ppg-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,19 @@ jobs:
https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo dpkg -i percona-release_latest.generic_all.deb
- name: Install Percona Distribution Postgresql 14
- name: Install Percona Distribution Postgresql 14 & Extensions
run: |
sudo percona-release setup ppg-14
sudo apt-get update -y
sudo apt-get install -y percona-postgresql-14 \
percona-postgresql-contrib percona-postgresql-server-dev-all
percona-postgresql-contrib percona-postgresql-server-dev-all \
percona-pgpool2 libpgpool2 percona-postgresql-14-pgaudit \
percona-postgresql-14-pgaudit-dbgsym percona-postgresql-14-repack \
percona-postgresql-14-repack-dbgsym percona-pgaudit14-set-user \
percona-pgaudit14-set-user-dbgsym percona-postgresql-14-postgis-3 \
percona-postgresql-14-postgis-3-scripts \
percona-postgresql-postgis-scripts percona-postgresql-postgis \
percona-postgis
- name: Change src owner to postgres
run: |
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/postgresql-15-ppg-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,19 @@ jobs:
https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo dpkg -i percona-release_latest.generic_all.deb
- name: Install Percona Distribution Postgresql 15
- name: Install Percona Distribution Postgresql 15 & Extensions
run: |
sudo percona-release setup ppg-15
sudo apt-get update -y
sudo apt-get install -y percona-postgresql-15 \
percona-postgresql-contrib percona-postgresql-server-dev-all
percona-postgresql-contrib percona-postgresql-server-dev-all \
percona-pgpool2 libpgpool2 percona-postgresql-15-pgaudit \
percona-postgresql-15-pgaudit-dbgsym percona-postgresql-15-repack \
percona-postgresql-15-repack-dbgsym percona-pgaudit15-set-user \
percona-pgaudit15-set-user-dbgsym percona-postgresql-15-postgis-3 \
percona-postgresql-15-postgis-3-scripts \
percona-postgresql-postgis-scripts percona-postgresql-postgis \
percona-postgis
- name: Change src owner to postgres
run: |
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/postgresql-16-ppg-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,19 @@ jobs:
https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo dpkg -i percona-release_latest.generic_all.deb
- name: Install Percona Distribution Postgresql 16
- name: Install Percona Distribution Postgresql 16 & Extensions
run: |
sudo percona-release setup ppg-16
sudo apt-get update -y
sudo apt-get install -y percona-postgresql-16 \
percona-postgresql-contrib percona-postgresql-server-dev-all
percona-postgresql-contrib percona-postgresql-server-dev-all \
percona-pgpool2 libpgpool2 percona-postgresql-16-pgaudit \
percona-postgresql-16-pgaudit-dbgsym percona-postgresql-16-repack \
percona-postgresql-16-repack-dbgsym percona-pgaudit16-set-user \
percona-pgaudit16-set-user-dbgsym percona-postgresql-16-postgis-3 \
percona-postgresql-16-postgis-3-scripts \
percona-postgresql-postgis-scripts percona-postgresql-postgis \
percona-postgis
- name: Change src owner to postgres
run: |
Expand Down
113 changes: 113 additions & 0 deletions t/032_multiple_extensions.pl
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
#!/usr/bin/perl

use strict;
use warnings;
use File::Basename;
use File::Compare;
use File::Copy;
use Test::More;
use lib 't';
use pgsm;

# Get file name and CREATE out file name and dirs WHERE requried
PGSM::setup_files_dir(basename($0));

my $PG_VERSION_STRING = `pg_config --version`;

if (index(lc($PG_VERSION_STRING), lc("percona")) == -1)
{
plan skip_all => "pg_stat_monitor test case only for PPG server package install with extensions.";
}

# CREATE new PostgreSQL node and do initdb
my $node = PGSM->pgsm_init_pg();
my $pgdata = $node->data_dir;

# UPDATE postgresql.conf to include/load pg_stat_monitor library
open my $conf, '>>', "$pgdata/postgresql.conf";
print $conf "shared_preload_libraries = 'pg_stat_monitor, pgaudit, set_user, pg_repack'\n";
print $conf "pg_stat_monitor.pgsm_bucket_time = 360000\n";
print $conf "pg_stat_monitor.pgsm_normalized_query = 'yes'\n";
close $conf;

# Start server
my $rt_value = $node->start;
ok($rt_value == 1, "Start Server");

# Create PGSM extension
my ($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION pg_stat_monitor;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE PGSM EXTENSION");
PGSM::append_to_debug_file($stdout);

($cmdret, $stdout, $stderr) = $node->psql('postgres', 'SELECT pg_stat_monitor_reset();', extra_params => ['-a', '-Pformat=aligned','-Ptuples_only=off']);
ok($cmdret == 0, "Reset PGSM EXTENSION");
PGSM::append_to_debug_file($stdout);

# Create Other extensions
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS pgaudit;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE pgaudit EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS set_user;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE set_user EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS pg_repack;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE pg_repack EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', "SET pgaudit.log = 'none'; CREATE EXTENSION IF NOT EXISTS postgis; SET pgaudit.log = 'all';", extra_params => ['-a']);
ok($cmdret == 0, "CREATE postgis EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS postgis_raster;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE postgis_raster EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS postgis_sfcgal;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE postgis_sfcgal EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS fuzzystrmatch;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE fuzzystrmatch EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS address_standardizer;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE address_standardizer EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS address_standardizer_data_us;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE address_standardizer_data_us EXTENSION");
PGSM::append_to_debug_file($stdout);
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE EXTENSION IF NOT EXISTS postgis_tiger_geocoder;', extra_params => ['-a']);
ok($cmdret == 0, "CREATE postgis_tiger_geocoder EXTENSION");
PGSM::append_to_debug_file($stdout);

# Print PGSM settings
($cmdret, $stdout, $stderr) = $node->psql('postgres', "SELECT name, setting, unit, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, pending_restart FROM pg_settings WHERE name='pg_stat_monitor.pgsm_query_shared_buffer';", extra_params => ['-a', '-Pformat=aligned','-Ptuples_only=off']);
ok($cmdret == 0, "Print PGSM EXTENSION Settings");
PGSM::append_to_debug_file($stdout);

# Create example database and run pgbench init
($cmdret, $stdout, $stderr) = $node->psql('postgres', 'CREATE database example;', extra_params => ['-a']);
print "cmdret $cmdret\n";
ok($cmdret == 0, "CREATE Database example");
PGSM::append_to_debug_file($stdout);

my $port = $node->port;
print "port $port \n";

my $out = system ("pgbench -i -s 20 -p $port example");
print " out: $out \n";
ok($cmdret == 0, "Perform pgbench init");

$out = system ("pgbench -c 10 -j 2 -t 5000 -p $port example");
print " out: $out \n";
ok($cmdret == 0, "Run pgbench");

($cmdret, $stdout, $stderr) = $node->psql('postgres', 'SELECT datname, substr(query,0,150) AS query, SUM(calls) AS calls FROM pg_stat_monitor GROUP BY datname, query ORDER BY datname, query, calls;', extra_params => ['-a', '-Pformat=aligned','-Ptuples_only=off']);
ok($cmdret == 0, "SELECT XXX FROM pg_stat_monitor");
PGSM::append_to_debug_file($stdout);

# DROP EXTENSION
$stdout = $node->safe_psql('postgres', 'DROP EXTENSION pg_stat_monitor;', extra_params => ['-a']);
ok($cmdret == 0, "DROP PGSM EXTENSION");
PGSM::append_to_debug_file($stdout);

# Stop the server
$node->stop;

# Done testing for this testcase file.
done_testing();
Empty file modified t/expected/007_settings_pgsm_query_shared_buffer.out.15
100755 → 100644
Empty file.

0 comments on commit 5d7c424

Please sign in to comment.