Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use prometheus v2.9.2, common v0.4.0 & tsdb v0.8.0 #1133

Merged
merged 3 commits into from
May 31, 2019

Conversation

sylr
Copy link
Contributor

@sylr sylr commented May 10, 2019

No description provided.

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me (:

@sylr sylr marked this pull request as ready for review May 10, 2019 14:57
Copy link
Member

@povilasv povilasv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit, other than that 💯

pkg/compact/downsample/streamed_block_writer.go Outdated Show resolved Hide resolved
pkg/rule/rule.go Outdated Show resolved Hide resolved
pkg/compact/downsample/streamed_block_writer.go Outdated Show resolved Hide resolved
pkg/compact/downsample/streamed_block_writer.go Outdated Show resolved Hide resolved
pkg/rule/rule.go Outdated Show resolved Hide resolved
pkg/runutil/runutil.go Outdated Show resolved Hide resolved
pkg/runutil/runutil.go Outdated Show resolved Hide resolved
@brancz
Copy link
Member

brancz commented May 11, 2019

This needs a rebase. I have no strong opinion on the tsdb_errors vs tsdberrors thing, while tsdb may be doing something wrong here I generally prefer to stay as close as possible to goimports as named imports usually just end up confusing people. In that sense i think i would prefer not renaming and sticking to the upstream and try to continue that consistently. (And fix it in tsdb)

In any case once rebased this is good to go.

@sylr sylr force-pushed the prometheus-2.9.2 branch 4 times, most recently from 81886b5 to dfa6499 Compare May 13, 2019 12:45
@GiedriusS
Copy link
Member

@povilasv is it okay with you that we leave it like tsdb_errors?

@povilasv
Copy link
Member

@GiedriusS yes @brancz makes a good point

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but let's fix this package prefix. Also this is not the most important part - the important part is to grab this and deploy on clusters ): It's always critical change.

Can we walk through the changelog and grab what changes and map to what we use? I was always doing it for last Prom/TSDB upgrades

https://github.com/improbable-eng/thanos/blob/master/CHANGELOG.md
For example as we had in previous one:

Changes that affects Thanos:

query:
[ENHANCEMENT] In histogram_quantile merge buckets with equivalent le values. #5158.
[ENHANCEMENT] Show list of offending labels in the error message in many-to-many scenarios. #5189
[BUGFIX] Fix panic when aggregator param is not a literal. #5290
ruler:
[ENHANCEMENT] Reduce time that Alertmanagers are in flux when reloaded. #5126
[BUGFIX] prometheus_rule_group_last_evaluation_timestamp_seconds is now a unix timestamp. #5186
[BUGFIX] prometheus_rule_group_last_duration_seconds now reports seconds instead of nanoseconds. Fixes our issue #1027
[BUGFIX] Fix sorting of rule groups. #5260
store: [ENHANCEMENT] Fast path for EmptyPostings cases in Merge, Intersect and Without.
tooling: [FEATURE] New dump command to tsdb tool to dump all samples.
compactor:
[ENHANCEMENT] When closing the db any running compaction will be cancelled so it doesn't block.
[CHANGE] Renamed flag --sync-delay to --consistency-delay #1053

pkg/compact/downsample/downsample.go Outdated Show resolved Hide resolved
pkg/compact/downsample/streamed_block_writer.go Outdated Show resolved Hide resolved
@bwplotka
Copy link
Member

Thanks for the work @sylr !

Also please rebase, this failing preview might be worrying (:

@sylr
Copy link
Contributor Author

sylr commented May 15, 2019

All checks are green now.

@sylr
Copy link
Contributor Author

sylr commented May 16, 2019

It's conflicting again, will you merge it if I rebase ?

@brancz
Copy link
Member

brancz commented May 16, 2019

I think @bwplotka was primarily waiting on something doing:

the important part is to grab this and deploy on clusters

If we can validate this, I'm happy with moving forward.

@GiedriusS
Copy link
Member

GiedriusS commented May 18, 2019

LGTM, but let's fix this package prefix. Also this is not the most important part - the important part is to grab this and deploy on clusters ): It's always critical change.

Can we walk through the changelog and grab what changes and map to what we use? I was always doing it for last Prom/TSDB upgrades

https://github.com/improbable-eng/thanos/blob/master/CHANGELOG.md
For example as we had in previous one:

Changes that affects Thanos:

query:
[ENHANCEMENT] In histogram_quantile merge buckets with equivalent le values. #5158.
[ENHANCEMENT] Show list of offending labels in the error message in many-to-many scenarios. #5189
[BUGFIX] Fix panic when aggregator param is not a literal. #5290
ruler:
[ENHANCEMENT] Reduce time that Alertmanagers are in flux when reloaded. #5126
[BUGFIX] prometheus_rule_group_last_evaluation_timestamp_seconds is now a unix timestamp. #5186
[BUGFIX] prometheus_rule_group_last_duration_seconds now reports seconds instead of nanoseconds. Fixes our issue #1027
[BUGFIX] Fix sorting of rule groups. #5260
store: [ENHANCEMENT] Fast path for EmptyPostings cases in Merge, Intersect and Without.
tooling: [FEATURE] New dump command to tsdb tool to dump all samples.
compactor:
[ENHANCEMENT] When closing the db any running compaction will be cancelled so it doesn't block.
[CHANGE] Renamed flag --sync-delay to --consistency-delay #1053

I believe this comment by @bwplotka is still not addressed. It would be nice to have these things in the CHANGELOG.md 😄 Could you do that, @sylr? Really awesome job that you've done here 👍

@bwplotka
Copy link
Member

Yea, the changelog is quite manual job ):

@bwplotka
Copy link
Member

Seems like a rebase is not difficult as well @sylr

Sylvain Rabot added 2 commits May 31, 2019 13:37
- prometheus/prometheus@v2.9.2+incompatible
- prometheus/common@v0.4.0
- prometheus/tsdb@v0.8.0

Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
@sylr sylr changed the title Use prometheus v2.9.2 and tsdb master Use prometheus v2.9.2, common v0.4.0 & tsdb v0.8.0 May 31, 2019
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from my side.

@bwplotka
Copy link
Member

However we need to still have a changelog for this @sylr ): This adds a lot, if you can grab what Prometheus and TSDB Changelog covers from our previous versions to the thing we update in this PR offline we can work out what is interesting/impacting Thanos users. What do you think?

@sylr
Copy link
Contributor Author

sylr commented May 31, 2019

TSDB changelog:

d48a5e2 (tag: v0.8.0, tag: 0.8.0) Release 0.8.0 (#607) Krasi Georgiev 2 weeks ago
bce663e Export the current segment index as a metic. (#601) Callum Styan 2 weeks ago
96a8784 fix wal panic when page flush fails. (#582) Krasi Georgiev 2 weeks ago
30d0ea5 Don't crash on an unknown tombstone ref. (#604) Brian Brazil 2 weeks ago
63d9c2a makefile: update Makefile.common with newer version (#603) PrometheusBot 2 weeks ago
89a90fe Simplify mergedPostings.Seek (#595) Brian Brazil 3 weeks ago
6ac81cc Correctly handle empty labels. (#594) Brian Brazil 3 weeks ago
2ae0281 Add FromData function (again)! (#600) Goutham Veeramachaneni 4 weeks ago
4da65c4 makefile: update Makefile.common with newer version (#597) PrometheusBot 4 weeks ago
3ccab17 docs: add label offset table and postings offset table (#593) zhulongcheng 4 weeks ago
5512826 make Close methods for the querier safe to call more than once. (#581) Krasi Georgiev 4 weeks ago
19d402d cleanup tmp files (#570) zhulongcheng 5 weeks ago
288f67e fix some unchecked errors and remove unused vars. (#592) Krasi Georgiev 5 weeks ago
a10c001 Synchronize Makefile.common from prometheus/prometheus (#591) PrometheusBot 5 weeks ago
c204505 (tag: v0.7.1) Release 0.7.1 (#587) Brian Brazil 5 weeks ago
f9bc081 Avoid allocation in mergedPostings.Seek. (#586) Brian Brazil 5 weeks ago
7a12d45 Avoid potential collision of createIdxChkReaders in querier_test.go (#585) naivewong 6 weeks ago
e2c6eb2 Synchronize Makefile.common from prometheus/prometheus (#584) PrometheusBot 6 weeks ago
4b3a5ac (tag: v0.7.0) release 0.7.0 (#580) Krasi Georgiev 7 weeks ago
dfed85e Keep series that are still in WAL in checkpoints (#577) Brian Brazil 7 weeks ago
259847a  Be smarter in how we look at matchers. (#572) Brian Brazil 7 weeks ago
7ab060c selectOverlappingDirs selects wrong dirs where there are 2 disjoint sets of overlapping dirs naivewong 8 weeks ago
8b33ee9 force persisting the tombstone file (#578) zhulongcheng 8 weeks ago
520b1d8 index: add a test to trigger fd leak on corrupted index (#576) Pavel Borzenkov 8 weeks ago
8eeb70f remove Fsync workaround for macos. (#574) Krasi Georgiev 8 weeks ago
4f204dc chunks: close all opened MmapFiles on NewDirReader error path (#575) Pavel Borzenkov 8 weeks ago
e46ec89 use fsync to force persisting the meta file. (#573) Krasi Georgiev 9 weeks ago
aed1662 Add Head.compactable method (#542) zhulongcheng 9 weeks ago
fd188be Fixed default benchmark test path (#571) Vladimir Masarik 9 weeks ago
7757fe6 Added ability to create db with NopWal (#519) Vishnunarayan K I 10 weeks ago
db9177d docs: add multi chunks layout in chunks readme file (#567) zhulongcheng 10 weeks ago
e11e01f Fix exported function comments (#566) zhulongcheng 10 weeks ago
4d03c70 Verify chunk format version (#544) zhulongcheng 10 weeks ago
3ab5f4e index: reduce empty postings trees (#509) Bartek Płotka 2 months ago
1c392d0 Update wal.md that the first wal entry includes only base id and timestamp without a value (#562) naivewong 2 months ago
837ae9a Update comment for ErrOutOfOrderSample (#563) zhulongcheng 2 months ago
62cfe44 Make Head.symbols map with size hint (#552) zhulongcheng 2 months ago
921da4d Remove unused statement from index test (#558) zhulongcheng 2 months ago
c3ffdf1 Test createBlock and check all os.RemoveAll in the tests for errors. (#549) Krasi Georgiev 2 months ago
dd0d3c6 Several small fixes (#550) naivewong 2 months ago
99703b3 README: move the build badge to the badge list (#555) zhulongcheng 2 months ago
95648b3 Fix a typo in head.go (#553) zhulongcheng 2 months ago
8feb67e Update Appender.AddFast method comment (#551) zhulongcheng 3 months ago
b7cac50 Remove unused NewReader function (#547) zhulongcheng 3 months ago
84e51f7 Update Makefile.common (#548) Simon Pasquier 3 months ago
719b57d Remove obsolete FromData function (#540) zhulongcheng 3 months ago

Prometheus changelog:

sylvain@ubuntu-1604-dev:~/gopath/src/github.com/prometheus/prometheus[go-mod-v2]$ PAGER=cat git log --oneline  4d60eb36dcbe...v2.9.2 --pretty="format:%C(yellow)%h%C(reset)%C(auto)%d %C(reset)%s %C(green)%an %C(cyan)%cr %C(reset)"
d3245f15 (tag: v2.9.2, origin/release-2.9) Merge pull request #5503 from prometheus/release292 Brian Brazil 5 weeks ago
cdc2ad05 Cut 2.9.2 Brian Brazil 5 weeks ago
7f9ff08c Upgrade tsdb to 0.7.1 (#5502) Brian Brazil 5 weeks ago
23c382e2 Remote Storage: string interner should not panic in release (#5487) Callum Styan 5 weeks ago
31a2d14e cmd/promtool: return errors from rule evaluations (#5483) Simon Pasquier 5 weeks ago
f2ca36e5 Exhaust every request body before closing it (#5166) (#5479) Romain Baugue 5 weeks ago
3da718c2 Make sure subquery range is taken into account for selection (#5467) Goutham Veeramachaneni 5 weeks ago
ad71f278 (tag: v2.9.1) Merge pull request #5464 from prometheus/rel291 Brian Brazil 6 weeks ago
a9309a2e Prepare 2.9.1 Brian Brazil 6 weeks ago
b7169df0 discovery/kubernetes: fix missing label sanitization (#5462) Simon Pasquier 6 weeks ago
ef19ede1 Prevent reshard concurrent with calling stop (#5460) Vasily Sliouniaev 6 weeks ago
1a9cdbd0 (tag: v2.9.0) Release 2.9.0 (#5458) Brian Brazil 7 weeks ago
4d7417f5 *: add CI job to synchronize Makefile.common across all repos (#5444) Paweł Krupa 7 weeks ago
bc1c7f18 Fix scalar-vector comparisons (#5454) Julius Volz 7 weeks ago
46660a07 (tag: v2.9.0-rc.0) Prepare 2.9.0-rc.0 release (#5450) Brian Brazil 7 weeks ago
8ff6938f Update dependencies. (#5449) Brian Brazil 7 weeks ago
69dd0908 Check for cancellation on every step of a range evaluation Bryan Boreham 7 weeks ago
e4a37d09 Replace select with simpler error check Bryan Boreham 7 weeks ago
cc75c275 Fix multiple response.WriteHeader calls error in remote read adapter (#5159) qinng 7 weeks ago
e5dbac79 cmd/prometheus: group flags properly (#5419) Simon Pasquier 7 weeks ago
dafd1632 discovery/kubernetes: add present labels for labels/annotations (#5443) Simon Pasquier 7 weeks ago
c1682adb Bump prometheus/common to v0.3.0 (#5344) Simon Pasquier 7 weeks ago
0e42449b docs/querying: improve wording (#5442) Lucas Servén Marín 7 weeks ago
141a165e Merge pull request #5441 from prometheus/29docs Frederic Branczyk 7 weeks ago
c8ee34d0 Put myself down for 2.9 release. Brian Brazil 7 weeks ago
c2b88992 Remote Write: fix checkpoint reading (#5429) Callum Styan 7 weeks ago
f7184978 Protect against memory exhaustion when scraping. Brian Brazil 8 weeks ago
dd307361 Don't lose the scrape cache on a failed scrape. Brian Brazil 8 weeks ago
4f47806a discovery/dns: fix slice with wrong length (#5432) Simon Pasquier 8 weeks ago
81c42480 *: bump gRPC and protobuf dependencies (#5367) Simon Pasquier 8 weeks ago
813b5836 [OpenStack SD] Add ProjectID and UserID meta labels (#5431) Kien Nguyen-Tuan 8 weeks ago
f8932dde Improve docs for query POST requests (#5428) Bob Shannon 8 weeks ago
8c8bb82d Add support for POSTing to /series endpoint (#5422) Bob Shannon 8 weeks ago
e5a427f2 Merge pull request #5418 from tariq1890/update_promclient Björn Rabenstein 9 weeks ago
0afd1722 Merge pull request #5416 from prometheus/release-2.8 Ganesh Vernekar 9 weeks ago
1ef6aaee (origin/release-2.8) Update CHANGELOG.md Tariq Ibrahim 9 weeks ago
be394823 import the latest patch release of client_golang in prometheus tariqibrahim 9 weeks ago
7a60e22c cmd/promtool: resolve relative paths in alert test files (#5336) David Symonds 9 weeks ago
8fdfa8ab refine error handling in prometheus (#5388) Tariq Ibrahim 10 weeks ago
0a87dcd4 cmd: Warn rather than Info when retention time wraps (#5403) Brian Brazil 10 weeks ago
19152a45 lockbot: don't comment when locking issues (#5402) Goutham Veeramachaneni 10 weeks ago
782d0005 discovery: factorize for SD based on refresh (#5381) Simon Pasquier 10 weeks ago
b95f4337 Add config for GitHub Lock app (#5392) Ben Kochie 2 months ago
91306bdf Support non POST methods for Lifecycle and Admin APIs (#5376) Bharath 2 months ago
2833ad49 vendor: bump github.com/hashicorp/serf (#5378) Simon Pasquier 2 months ago
0d7104b7 discovery/azure:optimize iteration logic for VMScalesets, VMScalesetVMs, and VMs (#5363) Tariq Ibrahim 2 months ago
b7e978d2 web/ui: Fix padding in navbar container (#5371) Matthias Loibl 2 months ago
5603b857 Check if label value is valid when unmarhsaling external labels from YAML, add a test to config_tests for valid/invalid external label value. Callum Styan 2 months ago
807fd33e Review feedback. Tom Wilkie 2 months ago
1a7923dd Add ref counting to string interning so we can remove a string when there are no longer any refs. Add tests for interning. Callum Styan 2 months ago
cbf5f132 Naive string iterning for labes & values in the remote_write path. Tom Wilkie 2 months ago
c7b35359 Use pkg/relabelling in remote write. Tom Wilkie 2 months ago
6c72cdb1 Add GODEBUG to the runtime page (#5324) Julien Pivotto 2 months ago
10ec2552 promparse: sort all labels when parsing (#5372) Giedrius Statkevičius 3 months ago
5f933e99 discovery/azure: make local virtualMachine struct more generic by removing the go sdk field reference (#5350) Tariq Ibrahim 3 months ago
e15d8c58 reload: copy state on both name and labels (#5368) James Ravn 3 months ago
23069b87 scrape: fallback to hostname if lookup fails (#5366) Simon Pasquier 3 months ago
4397916c Add honor_timestamps (#5304) Julien Pivotto 3 months ago
8155cc49 Expose lexer item types (#5358) Julius Volz 3 months ago
985eaa81 add go 1.12.x to the travis, circleci, and promu YAMLs (#5305) Tariq Ibrahim 3 months ago
5354ffff Fix some spelling issues (#5361) Mario Trangoni 3 months ago
9f8e1624 Makefile.common: run staticcheck for linux, freebsd and darwin only (#5331) Simon Pasquier 3 months ago
b0be2709 Makefile.common: add deps target (#5339) Simon Pasquier 3 months ago
d6f0e14a fix redundant err!=nil check in web.go (#5346) Tariq Ibrahim 3 months ago
67385f35 discovery/openstack: pass context to the OpenStack client (#5231) Simon Pasquier 3 months ago
7c8f0b08 Update CHANGELOG (#5341) Ganesh Vernekar 3 months ago
0d1a6935 scrape: Add global jitter for HA server (#5181) xjewer 3 months ago
83c46fd5 update Consul vendor code so that catalog.ServiceMultipleTags can be (#5151) Callum Styan 3 months ago
04ce817c scrape: Rewrite scrape loop options as a struct (#5314) Julien Pivotto 3 months ago
027d2ece config: resolve more file paths (#5284) Simon Pasquier 3 months ago
683fbc59 exponentation operator to drop metric name in result of op operation (#5329) Daisy T 3 months ago

Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome thanks @sylr for the offline work we did with parsing changelog. Lots of manual work. LGTM

@bwplotka bwplotka merged commit d5441f2 into thanos-io:master May 31, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants