Ocean archive indexer and api server #2724

Jouzo · 2023-11-27T19:28:55Z

Summary

Introduces ocean indexes and API

Flags :

"-oceanarchive", Enable ocean archive indexer
"-oceanarchiveserver", Enable ocean archive server
"-oceanarchiveport=<port>", Listen for ocean archive connections on <port>
"-oceanarchivebind=<addr>[:port]", Bind to given address to listen for Ocean connections. Do not expose the Ocean server to untrusted networks such as the public internet! This option is ignored unless -rpcallowip is also passed. Port is optional and overrides -oceanarchiveport. This option can be specified multiple times (default: 127.0.0.1 i.e., localhost)"

Jouzo · 2023-12-19T11:18:10Z

List of RPC queries done by Ocean that should be exposed directly by FFI:

account.getAccount
account.getAccountHistory
account.getBurnInfo
account.listAccountHistory
blockchain.getBlockchainInfo
loan.getLoanInfo
loan.getLoanScheme
masternode.getGov
net.getNetworkInfo
poolpair.getPoolPair
poolpair.listPoolPairs
token.getToken
token.listTokens

* add ApiPageResponse + impl json on list_blocks * refine * body -> query

* Added `rockdb` to ocean api. * Implementation of ocean-api database method all module. * Added data_access layer * Added test case for module.

* wip * fail-fast false

* pagination query * poolpair path api path * mild refactor on error.rs and block.rs * masternode typing * rm debug_handler * extra line * default masternodestate * fix return type * typo * pub enum state * missing import * clone * simplified mn

* added block indexing * added block indexing and query method * added query methods and extra table * added query methods to modules * added oracle price active and prive feed module * added oracle price active and prive feed module * rebased indexing ocean-archive * rebased indexing ocean-archive and updated query methods all modules * rebased indexing ocean-archive and updated query methods all modules * fixed white space * fixed white space * fixed query method for masternode and masterstats

* OceanStore building * With RepositoryOps trait * Cleanup columns * Implement put/get/list masternode * Fix MasternodeByHeight get_key * Cleanup MasternodeData * Invalidate masternode * Remove comments * cfg(test_off) --------- Co-authored-by: canonbrother <w.canonbrother@gmail.com>

* target branch feature/ocean-archive * docker-build * on pr master + note

* model cleanup dup fields * fix apis * fmt_rs * cleanup PriceTicker 2 * rm oracle_history * model cleanup * fix api * clippy * fmt_rs * better naming * fmt * fix get_oracle_by_address api * update rounding strategy update rounding strategy * aggregated amt and weigtage typing * fmt_rs

* cache pool utils * get_network_info_cached * guard

* ignore if any zero * fmt

This reverts commit 04716fa.

prasannavl · 2024-09-28T05:00:27Z

lib/ain-grpc/src/lib.rs

@@ -135,7 +135,7 @@ pub async fn init_ocean_server(addr: String) -> Result<()> {
    let (user, pass) = ain_cpp_imports::get_rpc_auth().map_err(|e| format_err!("{e}"))?;
    let client = Arc::new(
        Client::new(
-            &format!("localhost:{}", ain_cpp_imports::get_rpc_port()),
+            &format!("http://127.0.0.1:{}", ain_cpp_imports::get_rpc_port()),


Note, we might need to let this be configured from the flags similar to the rest as if the IP binding is different, it might not work

prasannavl · 2024-09-28T05:06:39Z

Going ahead and merging this in, as even though it doesn't affect consensus and the rest is also completed isolated and flagged - it's good to have the core part of it in for the testnet release before further refinements and fixes that target the optional ocean archive.

Bushstar · 2024-09-28T05:47:23Z

src/dfi/validation.cpp

@@ -4666,6 +4685,15 @@ Res ProcessDeFiEventFallible(const CBlock &block,
    // Construct undo
    FlushCacheCreateUndo(pindex, mnview, cache, uint256S(std::string(64, '1')));

+    // Ocean archive


Maybe this should be after evm_try_unsafe_commit_block in ConnectBlock, that's the very last point of failure when connecting new blocks.

Jouzo marked this pull request as draft November 27, 2023 19:36

Jouzo force-pushed the feature/ocean-archive branch 3 times, most recently from f12bdf8 to 96ff191 Compare November 27, 2023 20:19

Jouzo mentioned this pull request Dec 27, 2023

Ocean tracker #2763

Open

21 tasks

canonbrother force-pushed the feature/ocean-archive branch from cf793b1 to 039219b Compare February 7, 2024 07:15

Jouzo and others added 23 commits February 7, 2024 22:13

Ocean endpoints scaffold

91282a7

Model scaffold

8956bbb

Start ocean server

fc0f48c

Remove storage changes

42b01a1

Ocean list pagination + json req/res (#2737)

cd00f58

* add ApiPageResponse + impl json on list_blocks * refine * body -> query

Ocean indexing scaffold

d0cd25d

Ocean API: rocks db implementation for ocean API (#2728)

641ed94

* Added `rockdb` to ocean api. * Implementation of ocean-api database method all module. * Added data_access layer * Added test case for module.

WIP: Ocean test suite CI (#2734)

1865191

* wip * fail-fast false

Fix manger typo

35362b2

Add -oceanarchive flag

b3aee67

Ocean API MN typing (#2744)

5a067b0

* pagination query * poolpair path api path * mild refactor on error.rs and block.rs * masternode typing * rm debug_handler * extra line * default masternodestate * fix return type * typo * pub enum state * missing import * clone * simplified mn

Adds cached crate

c2885f3

Fix format

491362b

Fix ocean ci docker not found (#2759)

6937cd1

* target branch feature/ocean-archive * docker-build * on pr master + note

BlockService

a73860a

Finish Masternode indexing impl

9a7b078

Use serde as default Column trait implementation (#2765)

0acc887

Fix build

1d56342

Add 404 handler

274beca

Cleanup db iter method

9f27d77

Remove need for nightly

8cad7f9

Jouzo force-pushed the feature/ocean-archive branch from e33fc67 to d422a1b Compare September 24, 2024 08:00

Jouzo and others added 9 commits September 24, 2024 12:48

Merge branch 'master' into feature/ocean-archive

12985df

Merge branch 'master' into feature/ocean-archive

3748eb9

Ocean: cache pool utils (#3069)

9a6ce3a

* cache pool utils * get_network_info_cached * guard

Rename LoanCount to camelCase

b73ec36

handle diff usdt pool naming (#3070)

8c9ec8e

Merge branch 'master' into feature/ocean-archive

939cd54

Debug ocean endpoint error

420c38c

Terser logs

3648669

Jouzo changed the title ~~WIP: Ocean archive and Ocean API~~ Ocean Sep 27, 2024

prasannavl previously approved these changes Sep 27, 2024

View reviewed changes

Ocean: fix get apr default if any zero (#3075)

19fea19

* ignore if any zero * fmt

Jouzo dismissed prasannavl’s stale review via 19fea19 September 27, 2024 16:56

Jouzo added 10 commits September 27, 2024 17:58

Rename ocean rest* to server

995116b

Update gArgs access

5d47699

Grammar

c573c31

Fix guard

02f7f95

Cleanup -oceanarchive default

04716fa

Fix client host

34145e3

Cleanup logs

5b3525d

Cache with sync_writes

9224e8a

Revert "Cleanup -oceanarchive default"

c5ea970

This reverts commit 04716fa.

Sort poolpairs by id

e853878

prasannavl reviewed Sep 28, 2024

View reviewed changes

prasannavl changed the title ~~Ocean~~ Ocean archive indexer and api server Sep 28, 2024

prasannavl merged commit b309f1d into master Sep 28, 2024
17 of 28 checks passed

prasannavl deleted the feature/ocean-archive branch September 28, 2024 05:11

Bushstar reviewed Sep 28, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ocean archive indexer and api server #2724

Ocean archive indexer and api server #2724

Jouzo commented Nov 27, 2023 •

edited

Loading

Jouzo commented Dec 19, 2023

prasannavl Sep 28, 2024 •

edited

Loading

prasannavl commented Sep 28, 2024

Bushstar Sep 28, 2024

Ocean archive indexer and api server #2724

Ocean archive indexer and api server #2724

Conversation

Jouzo commented Nov 27, 2023 • edited Loading

Summary

Flags :

Jouzo commented Dec 19, 2023

prasannavl Sep 28, 2024 • edited Loading

Choose a reason for hiding this comment

prasannavl commented Sep 28, 2024

Bushstar Sep 28, 2024

Choose a reason for hiding this comment

Jouzo commented Nov 27, 2023 •

edited

Loading

prasannavl Sep 28, 2024 •

edited

Loading