Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: perform randomness hashing in the kernel #1842

Merged
merged 3 commits into from
Aug 16, 2023
Merged

feat: perform randomness hashing in the kernel #1842

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9c22c04
feat: perform randomness hashing in the kernel
arajasek Aug 11, 2023
d710e06
address review
arajasek Aug 16, 2023
d002d48
bump test vectors
arajasek Aug 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions fvm/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ rand = "0.8.5"
quickcheck = { version = "1", optional = true }
once_cell = "1.18"
minstant = "0.1.2"
blake2b_simd = "1.0.0"
byteorder = "1.4.3"

[dev-dependencies]
pretty_assertions = "1.3.0"
Expand Down
14 changes: 2 additions & 12 deletions fvm/src/externs/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,11 @@ pub trait Consensus {
pub trait Rand {
/// Gets 32 bytes of randomness for ChainRand paramaterized by the DomainSeparationTag,
/// ChainEpoch, Entropy from the ticket chain.
fn get_chain_randomness(
&self,
pers: i64,
round: ChainEpoch,
entropy: &[u8],
) -> anyhow::Result<[u8; 32]>;
fn get_chain_randomness(&self, round: ChainEpoch) -> anyhow::Result<[u8; 32]>;

/// Gets 32 bytes of randomness for ChainRand paramaterized by the DomainSeparationTag,
/// ChainEpoch, Entropy from the latest beacon entry.
fn get_beacon_randomness(
&self,
pers: i64,
round: ChainEpoch,
entropy: &[u8],
) -> anyhow::Result<[u8; 32]>;
fn get_beacon_randomness(&self, round: ChainEpoch) -> anyhow::Result<[u8; 32]>;
}

/// Chain information provider.
Expand Down
42 changes: 36 additions & 6 deletions fvm/src/kernel/default.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ use std::panic::{self, UnwindSafe};
use std::path::PathBuf;

use anyhow::{anyhow, Context as _};
use blake2b_simd::Params;
use byteorder::WriteBytesExt;
use cid::Cid;
use filecoin_proofs_api::{self as proofs, ProverId, PublicReplicaInfo, SectorId};
use fvm_ipld_blockstore::Blockstore;
Expand All @@ -27,6 +29,7 @@ use lazy_static::lazy_static;
use multihash::MultihashDigest;
use rayon::iter::{IndexedParallelIterator, IntoParallelRefIterator, ParallelIterator};
use rayon::prelude::ParallelDrainRange;
use std::io::Write;

use super::blocks::{Block, BlockRegistry};
use super::error::Result;
Expand Down Expand Up @@ -745,6 +748,24 @@ where
}
}

fn draw_randomness(
rbase: &[u8; RANDOMNESS_LENGTH],
pers: i64,
round: ChainEpoch,
entropy: &[u8],
) -> anyhow::Result<[u8; RANDOMNESS_LENGTH]> {
let mut state = Params::new().hash_length(32).to_state();
state.write_i64::<byteorder::BigEndian>(pers)?;
state.write_all(rbase)?;
state.write_i64::<byteorder::BigEndian>(round)?;
state.write_all(entropy)?;
state
.finalize()
.as_bytes()
.try_into()
.map_err(anyhow::Error::from)
}

impl<C> RandomnessOps for DefaultKernel<C>
where
C: CallManager,
Expand All @@ -763,12 +784,17 @@ where

// TODO(M2): Check error code
// Specifically, lookback length?
t.record(

let digest = t.record(
self.call_manager
.externs()
.get_chain_randomness(personalization, rand_epoch, entropy)
.get_chain_randomness(rand_epoch)
.or_illegal_argument(),
)
)?;

draw_randomness(&digest, personalization, rand_epoch, entropy).map_err(|e| {
syscall_error!(IllegalArgument; "failed to draw chain randmness {}", e).into()
})
}

fn get_randomness_from_beacon(
Expand All @@ -785,12 +811,16 @@ where

// TODO(M2): Check error code
// Specifically, lookback length?
t.record(
let digest = t.record(
self.call_manager
.externs()
.get_beacon_randomness(personalization, rand_epoch, entropy)
.get_beacon_randomness(rand_epoch)
.or_illegal_argument(),
)
)?;

draw_randomness(&digest, personalization, rand_epoch, entropy).map_err(|e| {
syscall_error!(IllegalArgument; "failed to draw beacon randmness {}", e).into()
})
}
}

Expand Down
4 changes: 0 additions & 4 deletions fvm/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,7 @@ mod test {
impl Rand for DummyExterns {
fn get_chain_randomness(
&self,
_pers: i64,
_round: fvm_shared::clock::ChainEpoch,
_entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
let msg = "mel was here".as_bytes();
let mut out = [0u8; 32];
Expand All @@ -89,9 +87,7 @@ mod test {

fn get_beacon_randomness(
&self,
_pers: i64,
_round: fvm_shared::clock::ChainEpoch,
_entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
todo!()
}
Expand Down
4 changes: 0 additions & 4 deletions fvm/tests/dummy.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,18 +35,14 @@ impl Externs for DummyExterns {}
impl Rand for DummyExterns {
fn get_chain_randomness(
&self,
_pers: i64,
_round: fvm_shared::clock::ChainEpoch,
_entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
todo!()
}

fn get_beacon_randomness(
&self,
_pers: i64,
_round: fvm_shared::clock::ChainEpoch,
_entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
todo!()
}
Expand Down
18 changes: 4 additions & 14 deletions testing/conformance/src/externs.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,22 +25,12 @@ impl TestExterns {
impl Externs for TestExterns {}

impl Rand for TestExterns {
fn get_chain_randomness(
&self,
pers: i64,
round: ChainEpoch,
entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
self.rand.get_chain_randomness(pers, round, entropy)
fn get_chain_randomness(&self, round: ChainEpoch) -> anyhow::Result<[u8; 32]> {
self.rand.get_chain_randomness(round)
}

fn get_beacon_randomness(
&self,
pers: i64,
round: ChainEpoch,
entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
self.rand.get_beacon_randomness(pers, round, entropy)
fn get_beacon_randomness(&self, round: ChainEpoch) -> anyhow::Result<[u8; 32]> {
self.rand.get_beacon_randomness(round)
}
}

Expand Down
26 changes: 6 additions & 20 deletions testing/conformance/src/rand.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ pub struct ReplayingRand {
pub struct TestFallbackRand;

impl Rand for TestFallbackRand {
fn get_chain_randomness(&self, _: i64, _: ChainEpoch, _: &[u8]) -> anyhow::Result<[u8; 32]> {
fn get_chain_randomness(&self, _: ChainEpoch) -> anyhow::Result<[u8; 32]> {
Ok(*b"i_am_random_____i_am_random_____")
}

fn get_beacon_randomness(&self, _: i64, _: ChainEpoch, _: &[u8]) -> anyhow::Result<[u8; 32]> {
fn get_beacon_randomness(&self, _: ChainEpoch) -> anyhow::Result<[u8; 32]> {
Ok(*b"i_am_random_____i_am_random_____")
}
}
Expand All @@ -50,40 +50,26 @@ impl ReplayingRand {
}

impl Rand for ReplayingRand {
fn get_chain_randomness(
&self,
dst: i64,
epoch: ChainEpoch,
entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
fn get_chain_randomness(&self, epoch: ChainEpoch) -> anyhow::Result<[u8; 32]> {
let rule = RandomnessRule {
kind: RandomnessKind::Chain,
dst,
epoch,
entropy: entropy.to_vec(),
};
if let Some(bz) = self.matches(rule) {
Ok(bz)
} else {
self.fallback.get_chain_randomness(dst, epoch, entropy)
self.fallback.get_chain_randomness(epoch)
}
}
fn get_beacon_randomness(
&self,
dst: i64,
epoch: ChainEpoch,
entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
fn get_beacon_randomness(&self, epoch: ChainEpoch) -> anyhow::Result<[u8; 32]> {
let rule = RandomnessRule {
kind: RandomnessKind::Beacon,
dst,
epoch,
entropy: entropy.to_vec(),
};
if let Some(bz) = self.matches(rule) {
Ok(bz)
} else {
self.fallback.get_beacon_randomness(dst, epoch, entropy)
self.fallback.get_beacon_randomness(epoch)
}
}
}
3 changes: 0 additions & 3 deletions testing/conformance/src/vector.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,10 +117,7 @@ pub enum RandomnessKind {
#[derive(Debug, Deserialize_tuple, PartialEq, Eq, Clone)]
pub struct RandomnessRule {
pub kind: RandomnessKind,
pub dst: i64,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a somewhat unfortunate decrease in test vector granularity that I'm guessing will also break the existing vectors we have? Any clever ideas about how we can prevent this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we probably just need to recreate them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, in theory, preserve the current behavior by mocking in the kernel. But then we wouldn't be testing the randomness function.

pub epoch: ChainEpoch,
#[serde(with = "base64_bytes")]
pub entropy: Vec<u8>,
}

#[derive(Debug, Deserialize, Clone)]
Expand Down
4 changes: 0 additions & 4 deletions testing/integration/src/dummy.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,7 @@ impl Externs for DummyExterns {}
impl Rand for DummyExterns {
fn get_chain_randomness(
&self,
_pers: i64,
_round: fvm_shared::clock::ChainEpoch,
_entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
let rng: String = thread_rng()
.sample_iter(&Alphanumeric)
Expand All @@ -29,9 +27,7 @@ impl Rand for DummyExterns {

fn get_beacon_randomness(
&self,
_pers: i64,
_round: fvm_shared::clock::ChainEpoch,
_entropy: &[u8],
) -> anyhow::Result<[u8; 32]> {
let rng: String = thread_rng()
.sample_iter(&Alphanumeric)
Expand Down
Loading