Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

local-tx-submission extension: evaluation of transaction execution units #176

Merged
merged 16 commits into from
Feb 13, 2022

Conversation

KtorZ
Copy link
Member

@KtorZ KtorZ commented Feb 12, 2022

Fixes #172 & #174


  • 📍 Rename 'SubmitTxPayload' into 'SerializedTx'
    It'll now be used also for evaluating transactions, so the name would've been misleading.

  • 📍 Draft tx-submission protocol extension.
    Thinking through this a bit, I concluded that the existing local-tx-submission was the most suitable place for adding this functionality. Users are already submitting serialized transactions through that protocol, and in essence, evaluating execution units shouldn't be much different.

    So, this extends existing Ogmios' capabilities and protocol and deviates now from the the core local-tx-submission protocol. From a client's perspective, this remains quite transparent and would look like just another method usable through Ogmios' websocket. Behind the scene, it'll actually be mostly implemented using the state-query protocol as we need to (a) resolve the transaction inputs into a UTXO set and (b) lookup current protocol parameters to correctly evaluate the transaction.

  • 📍 Write (& document) JSON encoders for ScriptFailure.

  • 📍 Add 'Array' utilities to Ogmios' Prelude.

  • 📍 Draft execution units evaluation.
    Remains now to run the needed state-query queries to get the necessary protocol parameters, utxo and time interpreter needed to calculate execution units.

  • 📍 Implement effectful execution units evaluator.

  • 📍 Wire the execution units evaluator through websocket connection stack.
    This effectively now starts two connections PER websocket connections... I might want to revisit that later if it turns out to cause too much overhead; There's already another connection started for the health check (although, one per Ogmios' server only) and it may be possible to wire into that one instead.
    However, since all protocols on that connection are pretty much idling all the time, it should cause any problem.

  • 📍 Add JSON-schema for the new 'EvaluateTx' local-tx-submission message.

  • 📍 Improve on error reporting for the SubmitTx and EvaluateTx errors.
    Still not perfect on CBOR deserialisation failures. Somehow, the decoder always returns a byteoffset of 0 and a 'expected tag' error. This is most probably a problem with the upstream CBOR decoder in the ledger... I will investigate one day to see if I can get anything better there.

  • 📍 Add missing matcher for Tx-Monitor in top-level error handler.
    Without this, sending invalid Tx-Monitor errors which simply return a generic client fault without much information.

  • 📍 Add support for 'EvaluateTx' in the TypeScript client & repl
    This enables e2e tests on real Testnet, which adds confidence that
    things are working properly. So far, only basic contracts and
    executino flow but I'll generate some slightly more complex test
    scenarios in upcoming commits.

  • 📍 Add submitted transaction's id to the success response of 'SubmitTx'
    This is so-to-speak a breaking change BUT, the API still supports the old way of submitting transactions which will not yield the transaction id. Indeed, the request has also been changed from

    >>> {"type":"jsonwsp/request","version":"1.0","servicename":"ogmios","methodname":"SubmitTx","args":{"bytes":"84a50081[...]c40ef5f6"}}
    {
      "type": "jsonwsp/response",
      "version": "1.0",
      "servicename": "ogmios",
      "methodname": "SubmitTx",
      "result": "SubmitSuccess",
      "reflection": null
    }

    to

    >>> {"type":"jsonwsp/request","version":"1.0","servicename":"ogmios","methodname":"SubmitTx","args":{"submit":"84a50081[..]900ff5f6"}}
    {
      "type": "jsonwsp/response",
      "version": "1.0",
      "servicename": "ogmios",
      "methodname": "SubmitTx",
      "result": {
        "SubmitSuccess": {
          "txId": "182ce8ad170adf9ad040b94053822f92172a518f442f17a9f0fbe32972b7f8d7"
        }
      },
      "reflection": null
    }

    The former will yield a response as before whereas the later will yields the new SubmitTxResponse which also contains a transaction id.

  • 📍 Add JSON-schema property tests (and fixes) for EvaluateTx

  • 📍 Replace e2e golden transaction in TypeScript evaluateTx tests
    Whoops. I did actually spent the collateral during a smoke test... so the transaction no longer valid 😬
    It'd be nice to allow Ogmios to also take an extra UTXO as input during evaluation to allow a more development-friendly mode where transaction may refer to inputs which do not exist (yet) on the network. This would make it a lot easier to construct test cases and to toy around.

  • 📍 Reword tx-submission user-guide and add section about EvaluateTx.

  • 📍 Fill-in CHANGELOG for 5.2.0


ws> {"type":"jsonwsp/request","version":"1.0","servicename":"ogmios","methodname":"EvaluateTx","args":{"evaluate":"84a6008282582078e963207a3fa50f5db363439a246d9c5631d398c7b7397435b6ec133432a6470182582097b2af6dfc6a4825e934146f424cdd6ede43ff98c355d2ae3aa95b0f70b63949040d8182582097b2af6dfc6a4825e934146f424cdd6ede43ff98c355d2ae3aa95b0f70b63949040181825839004a294f1ef53b30cdbf7caf17798422a90227224f9fbf037fcf6c47a5bc2ec1952d1189886fe018214eed45f83ab04171c41f373d530ca7a61a3bb94e8002000e800b58206df8859ec92c3ff6bc0e2964793789e44e4c5abbcc9ff6f2387b94f4c2020e6ea303814e4d010000332222200512001200110481800581840000182a820000f5f6"}}
{
  "type": "jsonwsp/response",
  "version": "1.0",
  "servicename": "ogmios",
  "methodname": "EvaluateTx",
  "result": {
    "EvaluationResult": {
      "spend:0": {
        "memory": 1700,
        "steps": 476468
      }
    }
  },
  "reflection": null
}
ws> {"type":"jsonwsp/request","version":"1.0","servicename":"ogmios","methodname":"EvaluateTx","args":{"evaluate":"84a6008182582097b2af6dfc6a4825e934146f424cdd6ede43ff98c355d2ae3aa95b0f70b63949030d800182825839004a294f1ef53b30cdbf7caf17798422a90227224f9fbf037fcf6c47a5bc2ec1952d1189886fe018214eed45f83ab04171c41f373d530ca7a61a3b79acf783581d7067f33146617a5e61936081db3b2117cbf59bd2123748f58ac96786561a001e8480582045b0cfc220ceec5b7c1c62c4d4193d38e4eba48e8815729ce75f9c0ab0e4c1c0021a000298890e800b582028aaa7a1cdf12d0070820ab5f5d8f1acba787535fed0729e4d11fbb3b068d17ea200818258209c3b2886f7b196ee20ce39c46abda9a76278534678b3e74288055f8b73f8ba3a58409ab3692df6560be6e4d831f69c6c6762b8892f905a37434b9b8aba6b355b761a12ea26289c9be6467fbd4a3a45c30131dc00ac836ecb1224b7e50d79d2e9c80c048180f5f6"}}
{
  "type": "jsonwsp/response",
  "version": "1.0",
  "servicename": "ogmios",
  "methodname": "EvaluateTx",
  "result": {
    "EvaluationFailure": {
      "UnknownInputs": [
        {
          "txId": "97b2af6dfc6a4825e934146f424cdd6ede43ff98c355d2ae3aa95b0f70b63949",
          "index": 3
        }
      ]
    }
  },
  "reflection": null
}
ws> {"type":"jsonwsp/request","version":"1.0","servicename":"ogmios","methodname":"EvaluateTx","args":{"evaluate":"83a4008182582039786f186d94d8dd0b4fcf05d1458b18cd5fd8c6823364612f4a3c11b77e7cc700018282581d60f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d1a05f5e10082581d60f8a68cd18e59a6ace848155a0e967af64f4d00cf8acee8adc95a6b0d1b000000d18635a3cf021a0002a331031878a10081825820eb94e8236e2099357fa499bfbc415968691573f25ec77435b7949f5fdfaa5da05840c8c0c016b714adb318a9495849c8ec647bc9742ef2b4cd03b9bc8694b65a42dbe3a2275ebcfe482c246fc8fbc34aa8dcebf18a4c3836b3ce8473e990d61c1506f6"}}
{
  "type": "jsonwsp/response",
  "version": "1.0",
  "servicename": "ogmios",
  "methodname": "EvaluateTx",
  "result": {
    "EvaluationFailure": {
      "IncompatibleEra": "Mary"
    }
  },
  "reflection": null
}

  It'll now be used also for evaluating transactions, so the name would've been misleading.
  Thinking through this a bit, I concluded that the existing local-tx-submission was the most suitable place for adding this functionality. Users are already submitting serialized transactions
  through that protocol, and in essence, evaluating execution units shouldn't be much different.

  So, this _extends_ existing Ogmios' capabilities and protocol and deviates now from the the core local-tx-submission protocol. From a client's perspective, this remains quite transparent and would look like just another method usable through Ogmios' websocket. Behind the scene, it'll actually be mostly implemented using the state-query protocol as we need to (a) resolve the transaction inputs into a UTXO set and (b) lookup current protocol parameters to correctly evaluate the transaction.
@KtorZ KtorZ self-assigned this Feb 12, 2022
  Remains now to run the needed state-query queries to get the necessary protocol parameters, utxo and time interpreter needed to calculate execution units.
  This effectively now starts two connections PER websocket connections... I might want to revisit that later if it turns out to cause too much overhead; There's already another connection started for the health check (although, one per Ogmios' server only) and it may be possible to wire into that one instead.
  However, since all protocols on that connection are pretty much idling all the time, it should cause any problem.
@KtorZ KtorZ force-pushed the KtorZ/172/evaluate-execution-units branch from fdcf28b to 4bd5b5e Compare February 12, 2022 18:13
@KtorZ KtorZ force-pushed the KtorZ/172/evaluate-execution-units branch from 4bd5b5e to ba00e78 Compare February 12, 2022 18:31
  Still not perfect on CBOR deserialisation failures. Somehow, the decoder always returns a byteoffset of 0 and a 'expected tag' error. This is most probably a problem with the upstream CBOR decoder in the ledger... I will investigate one day to see if I can get anything better there.
  Without this, sending invalid Tx-Monitor errors which simply return a generic client fault without much information.
  This enables e2e tests on real Testnet, which adds confidence that things are working properly. So far, only basic contracts and executino flow but I'll generate some slightly more complex test scenarios in upcoming commits.
@KtorZ KtorZ force-pushed the KtorZ/172/evaluate-execution-units branch from f8908e7 to 1939ae5 Compare February 13, 2022 12:08
  This is so-to-speak a breaking change BUT, the API still supports the old way of submitting transactions which will not yield the transaction id. Indeed, the request has also been changed from

  ```json
  >>> {"type":"jsonwsp/request","version":"1.0","servicename":"ogmios","methodname":"SubmitTx","args":{"bytes":"84a50081[...]c40ef5f6"}}
  {
    "type": "jsonwsp/response",
    "version": "1.0",
    "servicename": "ogmios",
    "methodname": "SubmitTx",
    "result": "SubmitSuccess",
    "reflection": null
  }
  ```

  to

  ```json
  >>> {"type":"jsonwsp/request","version":"1.0","servicename":"ogmios","methodname":"SubmitTx","args":{"submit":"84a50081[..]900ff5f6"}}
  {
    "type": "jsonwsp/response",
    "version": "1.0",
    "servicename": "ogmios",
    "methodname": "SubmitTx",
    "result": {
      "SubmitSuccess": {
        "txId": "182ce8ad170adf9ad040b94053822f92172a518f442f17a9f0fbe32972b7f8d7"
      }
    },
    "reflection": null
  }
  ```

  The former will yield a response as before whereas the later will yields the new SubmitTxResponse which also contains a transaction id.
@KtorZ KtorZ force-pushed the KtorZ/172/evaluate-execution-units branch from d87679d to 21590f5 Compare February 13, 2022 13:39
  Whoops. I did actually spent the collateral during a smoke test... so the transaction no longer valid 😬
  It'd be nice to allow Ogmios to also take an extra UTXO as input during evaluation to allow a more development-friendly mode where transaction may refer to inputs which do not exist (yet) on the network. This would make it a lot easier to construct test cases and to toy around.
@KtorZ KtorZ force-pushed the KtorZ/172/evaluate-execution-units branch from 4b5f3b9 to b5ff588 Compare February 13, 2022 14:27
@KtorZ KtorZ marked this pull request as ready for review February 13, 2022 15:05
@KtorZ KtorZ merged commit c113136 into master Feb 13, 2022
@KtorZ KtorZ deleted the KtorZ/172/evaluate-execution-units branch February 13, 2022 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for the estimating execution units for a transaction
1 participant