Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PG-810] PG-17 Support #463

Merged
merged 47 commits into from
Jul 18, 2024
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
12e75be
Temporary disable all workflows
artemgavrilov May 7, 2024
5094fb4
Add build workflow with PG17
artemgavrilov May 7, 2024
c85c03d
Fix incompatibilities
artemgavrilov May 7, 2024
adc19d5
Fix 007_settings_pgsm_query_shared_buffer.pl test
artemgavrilov May 7, 2024
4a76938
Fix 018_column_names.pl
artemgavrilov May 8, 2024
c6c394a
Fix 025_compare_pgss.pl
artemgavrilov May 8, 2024
8c79f5d
Remove tuplestore_donestoring usage at all
artemgavrilov May 9, 2024
3c68a60
Merge remote-tracking branch 'origin/main' into pg-17-support
artemgavrilov May 13, 2024
9d8b072
Rename I/O timing statistics columns to shared_blk_{read|write}_time
artemgavrilov May 22, 2024
f5059a8
Fix comments with fileds numbers
artemgavrilov May 22, 2024
28a0696
Fix format
artemgavrilov May 22, 2024
3dafa30
Revert "Temporary disable all workflows"
artemgavrilov May 22, 2024
73febf3
Disable all workflows except check and build for PG 15, 16 and 17
artemgavrilov May 23, 2024
00ea6af
Fix
artemgavrilov May 23, 2024
f5eec50
Fix comments
artemgavrilov May 23, 2024
2441445
Fix migration
artemgavrilov May 23, 2024
bf31663
Use REL_17_BETA1 in CI
artemgavrilov May 23, 2024
cd6d836
Add timers tests to 028_temp_block.pl
artemgavrilov May 24, 2024
60a03d1
Add local blocks timing statistics columns local_blk_{write|read}_time
artemgavrilov May 24, 2024
ffd9685
Fix t/027_local_blocks.pl test for older PG versions
artemgavrilov May 27, 2024
14e61c3
Fix
artemgavrilov May 27, 2024
6d741b8
Add jit_deform_{count|time} metrics
artemgavrilov May 27, 2024
fdbb072
Fix
artemgavrilov May 27, 2024
c78d1af
Add stats_since and minmax_stats_since fields
artemgavrilov Jun 5, 2024
e8a8603
Revert "Disable all workflows except check and build for PG 15, 16 an…
artemgavrilov Jun 5, 2024
1855bc9
Fix t/028_temp_block.pl for PG14 and below
artemgavrilov Jun 6, 2024
bbdc2d1
Fix build for PG12
artemgavrilov Jun 6, 2024
fd1e3a9
Add pgdg workflow for PG17
artemgavrilov Jun 6, 2024
fd8864d
Try to fix PG pgdg workflow
artemgavrilov Jun 6, 2024
122a30c
Fixes and formatting
artemgavrilov Jun 6, 2024
2af9ae7
Format code
artemgavrilov Jun 6, 2024
7c36562
Add level tracking regression test
artemgavrilov Jun 7, 2024
91c5683
Fix nesting level tracking
artemgavrilov Jun 7, 2024
ef378f5
Format code
artemgavrilov Jun 7, 2024
90e53e6
Add level tracking test expected result for PG13
artemgavrilov Jun 8, 2024
63a0679
Fix for PG12
artemgavrilov Jun 8, 2024
2675916
Skip level tracking regression test for PG version less than 14
artemgavrilov Jun 8, 2024
b0aa23f
Fix toplevel calculation for older PG version
artemgavrilov Jun 11, 2024
0b3feb7
Fix level tracking test results
artemgavrilov Jun 11, 2024
3e91da8
Fix nesting level counting for older PG version
artemgavrilov Jun 11, 2024
7a5add3
Revert "Fix nesting level counting for older PG version"
artemgavrilov Jun 11, 2024
8eddd50
Fix level tracking for older PG versions once again
artemgavrilov Jun 13, 2024
eb493e0
Set REL_17_BETA2 tag for PG
artemgavrilov Jul 2, 2024
45b0bfe
Merge remote-tracking branch 'origin/main' into pg-17-support
artemgavrilov Jul 11, 2024
4dd6e90
Add CI badge for PG17
artemgavrilov Jul 11, 2024
4f59dc7
Merge remote-tracking branch 'origin/main' into pg-17-support
artemgavrilov Jul 12, 2024
bc90dca
Use PG17 for examples in readme
artemgavrilov Jul 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 147 additions & 0 deletions .github/workflows/postgresql-17-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
name: postgresql-17-build
on:
pull_request:
push:
branches:
- main
tags:
- '[0-9]+.[0-9]+.[0-9]+*'

jobs:
build:
name: pg-17-build-test
runs-on: ubuntu-22.04
steps:
- name: Clone postgres repository
uses: actions/checkout@v4
with:
repository: 'postgres/postgres'
ref: 'REL_17_BETA1' # TODO: Replace with REL_17_STABLE
artemgavrilov marked this conversation as resolved.
Show resolved Hide resolved

- name: Install dependencies
run: |
sudo apt-get update
sudo apt purge postgresql-client-common postgresql-common \
postgresql postgresql*
sudo apt-get install -y libreadline6-dev systemtap-sdt-dev \
zlib1g-dev libssl-dev libpam0g-dev bison flex \
libipc-run-perl -y docbook-xsl docbook-xsl libxml2 libxml2-utils \
libxml2-dev libxslt-dev xsltproc libkrb5-dev libldap2-dev \
libsystemd-dev gettext tcl-dev libperl-dev pkg-config clang-11 \
llvm-11 llvm-11-dev libselinux1-dev python3-dev \
uuid-dev liblz4-dev
sudo rm -rf /var/lib/postgresql /var/log/postgresql /etc/postgresql \
/usr/lib/postgresql /usr/include/postgresql /usr/share/postgresql \
/etc/postgresql
sudo rm -f /usr/bin/pg_config
sudo /usr/bin/perl -MCPAN -e 'install IPC::Run'
sudo /usr/bin/perl -MCPAN -e 'install Text::Trim'

- name: Create pgsql dir
run: mkdir -p /opt/pgsql

- name: Build postgres
run: |
export PATH="/opt/pgsql/bin:$PATH"
./configure '--build=x86_64-linux-gnu' '--prefix=/usr' \
'--includedir=${prefix}/include' '--mandir=${prefix}/share/man' \
'--infodir=${prefix}/share/info' '--sysconfdir=/etc' \
'--localstatedir=/var' '--libdir=${prefix}/lib/x86_64-linux-gnu' \
'--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--with-icu' \
'--with-tcl' '--with-perl' '--with-python' '--with-pam' \
'--with-openssl' '--with-libxml' '--with-libxslt' '--with-ldap' \
'PYTHON=/usr/bin/python3' '--mandir=/usr/share/postgresql/17/man' \
'--docdir=/usr/share/doc/postgresql-doc-17' '--with-pgport=5432' \
'--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share' \
'--datadir=/usr/share/postgresql/17' '--with-uuid=e2fs' \
'--bindir=/usr/lib/postgresql/17/bin' '--enable-tap-tests' \
'--libdir=/usr/lib/x86_64-linux-gnu' '--enable-debug' \
'--libexecdir=/usr/lib/postgresql' '--with-gnu-ld' \
'--includedir=/usr/include/postgresql' '--enable-dtrace' \
'--enable-nls' '--enable-thread-safety' '--disable-rpath' \
'--with-system-tzdata=/usr/share/zoneinfo' '--with-llvm' \
'LLVM_CONFIG=/usr/bin/llvm-config-11' 'CLANG=/usr/bin/clang-11' \
'--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' \
'PROVE=/usr/bin/prove' 'TAR=/bin/tar' 'XSLTPROC=xsltproc --nonet' \
'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' \
'build_alias=x86_64-linux-gnu' '--with-gssapi' \
'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' \
'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fno-omit-frame-pointer' \
'CXXFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security'
make world
sudo make install-world

- name: Start postgresql cluster
run: |
export PATH="/usr/lib/postgresql/17/bin:$PATH"
sudo cp /usr/lib/postgresql/17/bin/pg_config /usr/bin
initdb -D /opt/pgsql/data
pg_ctl -D /opt/pgsql/data -l logfile start

- name: Clone pg_stat_monitor repository
uses: actions/checkout@v4
with:
path: 'src/pg_stat_monitor'

- name: Build pg_stat_monitor
run: |
make USE_PGXS=1
sudo make USE_PGXS=1 install
working-directory: src/pg_stat_monitor

- name: Configure and Restart Server
run: |
export PATH="/usr/lib/postgresql/17/bin:$PATH"
pg_ctl -D /opt/pgsql/data -l logfile stop
echo "shared_preload_libraries = 'pg_stat_monitor'" >> \
/opt/pgsql/data/postgresql.conf
echo "compute_query_id = regress" >> /opt/pgsql/data/postgresql.conf
pg_ctl -D /opt/pgsql/data -l logfile start
working-directory: src/pg_stat_monitor

- name: Start pg_stat_monitor_tests
run: |
make installcheck
working-directory: src/pg_stat_monitor/

- name: Change dir permissions on fail
if: ${{ failure() }}
run: |
sudo chmod -R ugo+rwx t
sudo chmod -R ugo+rwx tmp_check
exit 2 # regenerate error so that we can upload files in next step
working-directory: src/pg_stat_monitor

- name: Upload logs on fail
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: Regressions diff and postgresql log
path: |
src/pg_stat_monitor/regression.diffs
src/pg_stat_monitor/regression.out
src/pg_stat_monitor/logfile
src/pg_stat_monitor/t/results/
src/pg_stat_monitor/tmp_check/log/
!src/pg_stat_monitor/tmp_check/**/archives/*
!src/pg_stat_monitor/tmp_check/**/backup/*
!src/pg_stat_monitor/tmp_check/**/pgdata/*
!src/pg_stat_monitor/tmp_check/**/archives/
!src/pg_stat_monitor/tmp_check/**/backup/
!src/pg_stat_monitor/tmp_check/**/pgdata/
if-no-files-found: warn
retention-days: 3

- name: Start Server installcheck-world tests
run: make installcheck-world

- name: Report on installcheck-world test suites fail
uses: actions/upload-artifact@v4
if: ${{ failure() }}
with:
name: Regressions output files of failed testsuite, and pg log
path: |
**/regression.diffs
**/regression.out
src/pg_stat_monitor/logfile
retention-days: 3
93 changes: 93 additions & 0 deletions .github/workflows/postgresql-17-pgdg-package.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: postgresql-17-pgdg-package
on:
pull_request:
push:
branches:
- main
tags:
- '[0-9]+.[0-9]+.[0-9]+*'

jobs:
build:
name: pg-17-pgdg-package-test
runs-on: ubuntu-22.04
steps:
- name: Clone pg_stat_monitor repository
uses: actions/checkout@v4
with:
path: 'src/pg_stat_monitor'

- name: Delete old postgresql files
run: |
sudo apt-get update
sudo apt purge postgresql-client-common postgresql-common \
postgresql postgresql*
sudo apt-get install -y libreadline6-dev systemtap-sdt-dev wget \
zlib1g-dev libssl-dev libpam0g-dev bison flex libipc-run-perl
sudo rm -rf /var/lib/postgresql /var/log/postgresql /etc/postgresql \
/usr/lib/postgresql /usr/include/postgresql /usr/share/postgresql \
/etc/postgresql
sudo rm -f /usr/bin/pg_config
sudo /usr/bin/perl -MCPAN -e 'install IPC::Run'
sudo /usr/bin/perl -MCPAN -e 'install Text::Trim'

- name: Install PG Distribution Postgresql 17
run: |
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt \
$(lsb_release -cs)-pgdg main 17" > /etc/apt/sources.list.d/pgdg.list'
sudo wget --quiet -O - \
https://www.postgresql.org/media/keys/ACCC4CF8.asc |
sudo apt-key add -
sudo apt update
sudo apt -y install postgresql-17 postgresql-server-dev-17

- name: Change src owner to postgres
run: |
sudo chmod o+rx ~
sudo chown -R postgres:postgres src

- name: Build pg_stat_monitor
run: |
sudo -u postgres bash -c 'make USE_PGXS=1'
sudo make USE_PGXS=1 install
working-directory: src/pg_stat_monitor

- name: Start pg_stat_monitor_tests
run: |
sudo service postgresql stop
echo "shared_preload_libraries = 'pg_stat_monitor'" |
sudo tee -a /etc/postgresql/17/main/postgresql.conf
sudo service postgresql start
sudo psql -V
export PG_TEST_PORT_DIR=${GITHUB_WORKSPACE}/src/pg_stat_monitor
echo $PG_TEST_PORT_DIR
sudo -E -u postgres bash -c 'make installcheck USE_PGXS=1'
working-directory: src/pg_stat_monitor

- name: Change dir permissions on fail
if: ${{ failure() }}
run: |
sudo chmod -R ugo+rwx t
sudo chmod -R ugo+rwx tmp_check
exit 2 # regenerate error so that we can upload files in next step
working-directory: src/pg_stat_monitor

- name: Upload logs on fail
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: Regressions diff and postgresql log
path: |
src/pg_stat_monitor/regression.diffs
src/pg_stat_monitor/regression.out
src/pg_stat_monitor/logfile
src/pg_stat_monitor/t/results/
src/pg_stat_monitor/tmp_check/log/
!src/pg_stat_monitor/tmp_check/**/archives/*
!src/pg_stat_monitor/tmp_check/**/backup/*
!src/pg_stat_monitor/tmp_check/**/pgdata/*
!src/pg_stat_monitor/tmp_check/**/archives/
!src/pg_stat_monitor/tmp_check/**/backup/
!src/pg_stat_monitor/tmp_check/**/pgdata/
if-no-files-found: warn
retention-days: 3
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ MODULE_big = pg_stat_monitor
OBJS = hash_query.o guc.o pg_stat_monitor.o $(WIN32RES)

EXTENSION = pg_stat_monitor
DATA = pg_stat_monitor--2.0.sql pg_stat_monitor--1.0--2.0.sql
DATA = pg_stat_monitor--2.0.sql pg_stat_monitor--1.0--2.0.sql pg_stat_monitor--2.0--2.1.sql

PGFILEDESC = "pg_stat_monitor - execution statistics of SQL statements"

LDFLAGS_SL += $(filter -lm, $(LIBS))

TAP_TESTS = 1
REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_monitor/pg_stat_monitor.conf --inputdir=regression
REGRESS = basic version guc pgsm_query_id functions counters relations database error_insert application_name application_name_unique top_query cmd_type error rows tags user
REGRESS = basic version guc pgsm_query_id functions counters relations database error_insert application_name application_name_unique top_query cmd_type error rows tags user level_tracking

# Disabled because these tests require "shared_preload_libraries=pg_stat_statements",
# which typical installcheck users do not have (e.g. buildfarm clients).
Expand Down
2 changes: 2 additions & 0 deletions hash_query.c
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,8 @@ hash_entry_alloc(pgsmSharedState * pgsm, pgsmHashKey * key, int encoding)
memset(&entry->counters, 0, sizeof(Counters));
entry->query_text.query_pos = InvalidDsaPointer;
entry->counters.info.parent_query = InvalidDsaPointer;
entry->stats_since = GetCurrentTimestamp();
entry->minmax_stats_since = entry->stats_since;

/* set the appropriate initial usage count */
/* re-initialize the mutex each time ... we assume no one using it */
Expand Down
Loading
Loading