Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite gateway emitter for significant performance improvement #832

Merged
merged 6 commits into from
Jun 2, 2021
Merged

Rewrite gateway emitter for significant performance improvement #832

merged 6 commits into from
Jun 2, 2021

Conversation

zeylahellyer
Copy link
Member

@zeylahellyer zeylahellyer commented May 22, 2021

Rewrite the Emitter for performance. The emitter is what's called when bytes, an event, or a JSON payload is being emitted by the ShardProcessor.

Emitters have internally held a DashMap of listeners; shards and clusters could create multiple streams of events via Cluster::events, Cluster::some_events, Shard::events, and Shard::some_events. This was an intentional design decision, but over time we've seen that this capability hasn't seen much purpose.

Instead, we can create a stream of events when clusters or shards are created, and simply return that from Cluster::new, Shard::new, and their builder equivalents. This means that instead of only returning the cluster or shard, a two element tuple is returned of the cluster or shard and the stream of events. Events can then be sent by the ShardProcessor directly over a channel that the stream polls. This results in removing DashMap as a dependency and a significant performance improvement, as seen in the benchmark below.

The first pair of tests is emitting the raw bytes of a GUILD_ROLE_DELETE dispatch payload. The first test has a listener that doesn't want to receive the EventTypeFlags::SHARD_PAYLOAD variant, while the second does. While the first test sees no change - the change tends to go up and down due to how small of a scale of time it is - the second sees a significant improvement.

The second pair of tests is similar to the first, but instead of emitting the raw bytes of the payload it is attempting to emit a received payload as a deserialized event. If a listener does not want an event type, then the payload is discarded. If the user does want the event type, then the payload is deserialized and then sent to the user. Much like the first pair, the first test receives little improvement, but the second receives a comparable performance improvement.

emit bytes: doesn't want
                        time:   [3.1906 ns 3.1919 ns 3.1936 ns]
                        change: [+0.1098% +0.3127% +0.5112%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

emit bytes: wants       time:   [77.733 ns 78.493 ns 79.427 ns]
                        change: [-79.574% -79.312% -79.025%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

emit json: doesn't want time:   [7.2541 ns 7.2587 ns 7.2656 ns]
                        change: [-2.2163% -2.1091% -1.9993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

emit json: wants        time:   [274.46 ns 276.97 ns 279.86 ns]
                        change: [-60.786% -59.956% -58.741%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
Criterion benchmark used:
use criterion::{criterion_group, criterion_main, Criterion};

use twilight_gateway::{shard::emitter::Emitter, EventTypeFlags};

const INPUT: &str = r##"{
    "op": 0,
    "s": 2,
    "d": {
        "guild_id": "1",
        "role_id": "2"
    },
    "t": "GUILD_ROLE_DELETE"
}"##;
const INPUT_BYTES: &[u8] = INPUT.as_bytes();

fn emit_bytes(emitter: &Emitter) {
    emitter.bytes(INPUT_BYTES);
}

fn emit_json(emitter: &Emitter, input: &mut str) {
    emitter.json(0, Some(12), Some("GUILD_ROLE_DELETE"), input).unwrap();
}

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("emit bytes: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit bytes: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::SHARD_PAYLOAD);

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit json: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });

    c.bench_function("emit json: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::ROLE_DELETE);
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);

@zeylahellyer zeylahellyer added t-docs A change to the documentation c-gateway Affects the gateway crate m-breaking change Breaks the public API. enhancement t-refactor Refactors APIs or code. labels May 22, 2021
AEnterprise
AEnterprise previously approved these changes May 22, 2021
Gelbpunkt
Gelbpunkt previously approved these changes May 22, 2021
Copy link
Member

@Erk- Erk- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good and solid pr just a really minor nit that I spotted

gateway/src/shard/emitter.rs Outdated Show resolved Hide resolved
@zeylahellyer zeylahellyer dismissed stale reviews from Gelbpunkt and AEnterprise via 30687af May 22, 2021 12:59
@zeylahellyer
Copy link
Member Author

Addressed comments

Erk-
Erk- previously approved these changes May 22, 2021
AEnterprise
AEnterprise previously approved these changes May 22, 2021
tbnritzdoge
tbnritzdoge previously approved these changes May 22, 2021
Rewrite the `Emitter` for performance. The emitter is what's called when
bytes, an event, or a JSON payload is being emitted by the
`ShardProcessor`.

Emitters have internally held a `DashMap` of listeners; shards and
clusters could create multiple streams of events via `Cluster::events`,
`Cluster::some_events`, `Shard::events`, and `Shard::some_events`. This
was an intentional design decision, but over time we've seen that this
capability hasn't seen much purpose.

Instead, we can create a stream of events when clusters or shards are
created, and simply return that from `Cluster::new`, `Shard::new`, and
their builder equivalents. This means that instead of only returning the
cluster or shard, a two element tuple is returned of the cluster or
shard and the stream of events. Events can then be sent by the
`ShardProcessor` directly over a channel that the stream polls. This
results in removing `DashMap` as a dependency and a significant
performance improvement, as seen in the benchmark below.

The first pair of tests is emitting the raw bytes of a
`GUILD_ROLE_DELETE` dispatch payload. The first test has a listener that
doesn't want to receive the `EventTypeFlags::SHARD_PAYLOAD` variant,
while the second does. While the first test sees no change - the change
tends to go up and down due to how small of a scale of time it is - the
second sees a significant improvement.

The second pair of tests is similar to the first, but instead of
emitting the raw bytes of the payload it is attempting to emit a
received payload as a deserialized event. If a listener does not want an
event type, then the payload is discarded. If the user *does* want the
event type, then the payload is deserialized and then sent to the user.
Much like the first pair, the first test receives little improvement,
but the second receives a comparable performance improvement.

```
emit bytes: doesn't want
                        time:   [3.1906 ns 3.1919 ns 3.1936 ns]
                        change: [+0.1098% +0.3127% +0.5112%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

emit bytes: wants       time:   [77.733 ns 78.493 ns 79.427 ns]
                        change: [-79.574% -79.312% -79.025%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

emit json: doesn't want time:   [7.2541 ns 7.2587 ns 7.2656 ns]
                        change: [-2.2163% -2.1091% -1.9993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

emit json: wants        time:   [274.46 ns 276.97 ns 279.86 ns]
                        change: [-60.786% -59.956% -58.741%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
```

Criterion benchmark used:

```rust
use criterion::{criterion_group, criterion_main, Criterion};

use twilight_gateway::{shard::emitter::Emitter, EventTypeFlags};

const INPUT: &str = r##"{
    "op": 0,
    "s": 2,
    "d": {
        "guild_id": "1",
        "role_id": "2"
    },
    "t": "GUILD_ROLE_DELETE"
}"##;
const INPUT_BYTES: &[u8] = INPUT.as_bytes();

fn emit_bytes(emitter: &Emitter) {
    emitter.bytes(INPUT_BYTES);
}

fn emit_json(emitter: &Emitter, input: &mut str) {
    emitter.json(0, Some(12), Some("GUILD_ROLE_DELETE"), input).unwrap();
}

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("emit bytes: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit bytes: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::SHARD_PAYLOAD);

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit json: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });

    c.bench_function("emit json: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::ROLE_DELETE);
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
```

BREAKING CHANGES: `Cluster::new`, `ClusterBuilder::build`, `Shard::new`,
and `ShardBuilder::build` now return a tuple with two elements: the
cluster or shard itself and a stream of events.

Signed-off-by: Vivian Hellyer <vivian@hellyer.dev>
Signed-off-by: Vivian Hellyer <vivian@hellyer.dev>
Signed-off-by: Vivian Hellyer <vivian@hellyer.dev>
Signed-off-by: Vivian Hellyer <vivian@hellyer.dev>
Signed-off-by: Vivian Hellyer <vivian@hellyer.dev>
Signed-off-by: Vivian Hellyer <vivian@hellyer.dev>
@zeylahellyer zeylahellyer dismissed stale reviews from tbnritzdoge, AEnterprise, and Erk- via fcb7459 May 29, 2021 01:04
@zeylahellyer zeylahellyer merged commit fbed24a into twilight-rs:next Jun 2, 2021
@zeylahellyer zeylahellyer deleted the enhancement/gateway/faster-emitter branch June 2, 2021 01:30
7596ff pushed a commit to 7596ff/twilight that referenced this pull request Jun 7, 2021
Rewrite the `Emitter` for performance. The emitter is what's called when
bytes, an event, or a JSON payload is being emitted by the
`ShardProcessor`.

Emitters have internally held a `DashMap` of listeners; shards and
clusters could create multiple streams of events via `Cluster::events`,
`Cluster::some_events`, `Shard::events`, and `Shard::some_events`. This
was an intentional design decision, but over time we've seen that this
capability hasn't seen much purpose.

Instead, we can create a stream of events when clusters or shards are
created, and simply return that from `Cluster::new`, `Shard::new`, and
their builder equivalents. This means that instead of only returning the
cluster or shard, a two element tuple is returned of the cluster or
shard and the stream of events. Events can then be sent by the
`ShardProcessor` directly over a channel that the stream polls. This
results in removing `DashMap` as a dependency and a significant
performance improvement, as seen in the benchmark below.

The first pair of tests is emitting the raw bytes of a
`GUILD_ROLE_DELETE` dispatch payload. The first test has a listener that
doesn't want to receive the `EventTypeFlags::SHARD_PAYLOAD` variant,
while the second does. While the first test sees no change - the change
tends to go up and down due to how small of a scale of time it is - the
second sees a significant improvement.

The second pair of tests is similar to the first, but instead of
emitting the raw bytes of the payload it is attempting to emit a
received payload as a deserialized event. If a listener does not want an
event type, then the payload is discarded. If the user *does* want the
event type, then the payload is deserialized and then sent to the user.
Much like the first pair, the first test receives little improvement,
but the second receives a comparable performance improvement.

```
emit bytes: doesn't want
                        time:   [3.1906 ns 3.1919 ns 3.1936 ns]
                        change: [+0.1098% +0.3127% +0.5112%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

emit bytes: wants       time:   [77.733 ns 78.493 ns 79.427 ns]
                        change: [-79.574% -79.312% -79.025%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

emit json: doesn't want time:   [7.2541 ns 7.2587 ns 7.2656 ns]
                        change: [-2.2163% -2.1091% -1.9993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

emit json: wants        time:   [274.46 ns 276.97 ns 279.86 ns]
                        change: [-60.786% -59.956% -58.741%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
```

Criterion benchmark used:

```rust
use criterion::{criterion_group, criterion_main, Criterion};

use twilight_gateway::{shard::emitter::Emitter, EventTypeFlags};

const INPUT: &str = r##"{
    "op": 0,
    "s": 2,
    "d": {
        "guild_id": "1",
        "role_id": "2"
    },
    "t": "GUILD_ROLE_DELETE"
}"##;
const INPUT_BYTES: &[u8] = INPUT.as_bytes();

fn emit_bytes(emitter: &Emitter) {
    emitter.bytes(INPUT_BYTES);
}

fn emit_json(emitter: &Emitter, input: &mut str) {
    emitter.json(0, Some(12), Some("GUILD_ROLE_DELETE"), input).unwrap();
}

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("emit bytes: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit bytes: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::SHARD_PAYLOAD);

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit json: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });

    c.bench_function("emit json: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::ROLE_DELETE);
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
```

BREAKING CHANGES: `Cluster::new`, `ClusterBuilder::build`, `Shard::new`,
and `ShardBuilder::build` now return a tuple with two elements: the
cluster or shard itself and a stream of events.
7596ff pushed a commit to 7596ff/twilight that referenced this pull request Jun 13, 2021
Rewrite the `Emitter` for performance. The emitter is what's called when
bytes, an event, or a JSON payload is being emitted by the
`ShardProcessor`.

Emitters have internally held a `DashMap` of listeners; shards and
clusters could create multiple streams of events via `Cluster::events`,
`Cluster::some_events`, `Shard::events`, and `Shard::some_events`. This
was an intentional design decision, but over time we've seen that this
capability hasn't seen much purpose.

Instead, we can create a stream of events when clusters or shards are
created, and simply return that from `Cluster::new`, `Shard::new`, and
their builder equivalents. This means that instead of only returning the
cluster or shard, a two element tuple is returned of the cluster or
shard and the stream of events. Events can then be sent by the
`ShardProcessor` directly over a channel that the stream polls. This
results in removing `DashMap` as a dependency and a significant
performance improvement, as seen in the benchmark below.

The first pair of tests is emitting the raw bytes of a
`GUILD_ROLE_DELETE` dispatch payload. The first test has a listener that
doesn't want to receive the `EventTypeFlags::SHARD_PAYLOAD` variant,
while the second does. While the first test sees no change - the change
tends to go up and down due to how small of a scale of time it is - the
second sees a significant improvement.

The second pair of tests is similar to the first, but instead of
emitting the raw bytes of the payload it is attempting to emit a
received payload as a deserialized event. If a listener does not want an
event type, then the payload is discarded. If the user *does* want the
event type, then the payload is deserialized and then sent to the user.
Much like the first pair, the first test receives little improvement,
but the second receives a comparable performance improvement.

```
emit bytes: doesn't want
                        time:   [3.1906 ns 3.1919 ns 3.1936 ns]
                        change: [+0.1098% +0.3127% +0.5112%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

emit bytes: wants       time:   [77.733 ns 78.493 ns 79.427 ns]
                        change: [-79.574% -79.312% -79.025%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

emit json: doesn't want time:   [7.2541 ns 7.2587 ns 7.2656 ns]
                        change: [-2.2163% -2.1091% -1.9993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

emit json: wants        time:   [274.46 ns 276.97 ns 279.86 ns]
                        change: [-60.786% -59.956% -58.741%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
```

Criterion benchmark used:

```rust
use criterion::{criterion_group, criterion_main, Criterion};

use twilight_gateway::{shard::emitter::Emitter, EventTypeFlags};

const INPUT: &str = r##"{
    "op": 0,
    "s": 2,
    "d": {
        "guild_id": "1",
        "role_id": "2"
    },
    "t": "GUILD_ROLE_DELETE"
}"##;
const INPUT_BYTES: &[u8] = INPUT.as_bytes();

fn emit_bytes(emitter: &Emitter) {
    emitter.bytes(INPUT_BYTES);
}

fn emit_json(emitter: &Emitter, input: &mut str) {
    emitter.json(0, Some(12), Some("GUILD_ROLE_DELETE"), input).unwrap();
}

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("emit bytes: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit bytes: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::SHARD_PAYLOAD);

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit json: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });

    c.bench_function("emit json: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::ROLE_DELETE);
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
```

BREAKING CHANGES: `Cluster::new`, `ClusterBuilder::build`, `Shard::new`,
and `ShardBuilder::build` now return a tuple with two elements: the
cluster or shard itself and a stream of events.
7596ff pushed a commit to 7596ff/twilight that referenced this pull request Jun 13, 2021
Rewrite the `Emitter` for performance. The emitter is what's called when
bytes, an event, or a JSON payload is being emitted by the
`ShardProcessor`.

Emitters have internally held a `DashMap` of listeners; shards and
clusters could create multiple streams of events via `Cluster::events`,
`Cluster::some_events`, `Shard::events`, and `Shard::some_events`. This
was an intentional design decision, but over time we've seen that this
capability hasn't seen much purpose.

Instead, we can create a stream of events when clusters or shards are
created, and simply return that from `Cluster::new`, `Shard::new`, and
their builder equivalents. This means that instead of only returning the
cluster or shard, a two element tuple is returned of the cluster or
shard and the stream of events. Events can then be sent by the
`ShardProcessor` directly over a channel that the stream polls. This
results in removing `DashMap` as a dependency and a significant
performance improvement, as seen in the benchmark below.

The first pair of tests is emitting the raw bytes of a
`GUILD_ROLE_DELETE` dispatch payload. The first test has a listener that
doesn't want to receive the `EventTypeFlags::SHARD_PAYLOAD` variant,
while the second does. While the first test sees no change - the change
tends to go up and down due to how small of a scale of time it is - the
second sees a significant improvement.

The second pair of tests is similar to the first, but instead of
emitting the raw bytes of the payload it is attempting to emit a
received payload as a deserialized event. If a listener does not want an
event type, then the payload is discarded. If the user *does* want the
event type, then the payload is deserialized and then sent to the user.
Much like the first pair, the first test receives little improvement,
but the second receives a comparable performance improvement.

```
emit bytes: doesn't want
                        time:   [3.1906 ns 3.1919 ns 3.1936 ns]
                        change: [+0.1098% +0.3127% +0.5112%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

emit bytes: wants       time:   [77.733 ns 78.493 ns 79.427 ns]
                        change: [-79.574% -79.312% -79.025%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

emit json: doesn't want time:   [7.2541 ns 7.2587 ns 7.2656 ns]
                        change: [-2.2163% -2.1091% -1.9993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

emit json: wants        time:   [274.46 ns 276.97 ns 279.86 ns]
                        change: [-60.786% -59.956% -58.741%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
```

Criterion benchmark used:

```rust
use criterion::{criterion_group, criterion_main, Criterion};

use twilight_gateway::{shard::emitter::Emitter, EventTypeFlags};

const INPUT: &str = r##"{
    "op": 0,
    "s": 2,
    "d": {
        "guild_id": "1",
        "role_id": "2"
    },
    "t": "GUILD_ROLE_DELETE"
}"##;
const INPUT_BYTES: &[u8] = INPUT.as_bytes();

fn emit_bytes(emitter: &Emitter) {
    emitter.bytes(INPUT_BYTES);
}

fn emit_json(emitter: &Emitter, input: &mut str) {
    emitter.json(0, Some(12), Some("GUILD_ROLE_DELETE"), input).unwrap();
}

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("emit bytes: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit bytes: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::SHARD_PAYLOAD);

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit json: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });

    c.bench_function("emit json: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::ROLE_DELETE);
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
```

BREAKING CHANGES: `Cluster::new`, `ClusterBuilder::build`, `Shard::new`,
and `ShardBuilder::build` now return a tuple with two elements: the
cluster or shard itself and a stream of events.
7596ff pushed a commit to 7596ff/twilight that referenced this pull request Jun 13, 2021
Rewrite the `Emitter` for performance. The emitter is what's called when
bytes, an event, or a JSON payload is being emitted by the
`ShardProcessor`.

Emitters have internally held a `DashMap` of listeners; shards and
clusters could create multiple streams of events via `Cluster::events`,
`Cluster::some_events`, `Shard::events`, and `Shard::some_events`. This
was an intentional design decision, but over time we've seen that this
capability hasn't seen much purpose.

Instead, we can create a stream of events when clusters or shards are
created, and simply return that from `Cluster::new`, `Shard::new`, and
their builder equivalents. This means that instead of only returning the
cluster or shard, a two element tuple is returned of the cluster or
shard and the stream of events. Events can then be sent by the
`ShardProcessor` directly over a channel that the stream polls. This
results in removing `DashMap` as a dependency and a significant
performance improvement, as seen in the benchmark below.

The first pair of tests is emitting the raw bytes of a
`GUILD_ROLE_DELETE` dispatch payload. The first test has a listener that
doesn't want to receive the `EventTypeFlags::SHARD_PAYLOAD` variant,
while the second does. While the first test sees no change - the change
tends to go up and down due to how small of a scale of time it is - the
second sees a significant improvement.

The second pair of tests is similar to the first, but instead of
emitting the raw bytes of the payload it is attempting to emit a
received payload as a deserialized event. If a listener does not want an
event type, then the payload is discarded. If the user *does* want the
event type, then the payload is deserialized and then sent to the user.
Much like the first pair, the first test receives little improvement,
but the second receives a comparable performance improvement.

```
emit bytes: doesn't want
                        time:   [3.1906 ns 3.1919 ns 3.1936 ns]
                        change: [+0.1098% +0.3127% +0.5112%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

emit bytes: wants       time:   [77.733 ns 78.493 ns 79.427 ns]
                        change: [-79.574% -79.312% -79.025%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

emit json: doesn't want time:   [7.2541 ns 7.2587 ns 7.2656 ns]
                        change: [-2.2163% -2.1091% -1.9993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

emit json: wants        time:   [274.46 ns 276.97 ns 279.86 ns]
                        change: [-60.786% -59.956% -58.741%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
```

Criterion benchmark used:

```rust
use criterion::{criterion_group, criterion_main, Criterion};

use twilight_gateway::{shard::emitter::Emitter, EventTypeFlags};

const INPUT: &str = r##"{
    "op": 0,
    "s": 2,
    "d": {
        "guild_id": "1",
        "role_id": "2"
    },
    "t": "GUILD_ROLE_DELETE"
}"##;
const INPUT_BYTES: &[u8] = INPUT.as_bytes();

fn emit_bytes(emitter: &Emitter) {
    emitter.bytes(INPUT_BYTES);
}

fn emit_json(emitter: &Emitter, input: &mut str) {
    emitter.json(0, Some(12), Some("GUILD_ROLE_DELETE"), input).unwrap();
}

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("emit bytes: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit bytes: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::SHARD_PAYLOAD);

        b.iter(|| emit_bytes(&emitter))
    });

    c.bench_function("emit json: doesn't want", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::empty());
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });

    c.bench_function("emit json: wants", |b| {
        let (emitter, _rx) = Emitter::new(EventTypeFlags::ROLE_DELETE);
        let mut input = INPUT.to_owned();

        b.iter(|| emit_json(&emitter, &mut input));
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
```

BREAKING CHANGES: `Cluster::new`, `ClusterBuilder::build`, `Shard::new`,
and `ShardBuilder::build` now return a tuple with two elements: the
cluster or shard itself and a stream of events.
7596ff added a commit that referenced this pull request Jun 13, 2021
Upgrade Path

Replace `zlib` features with their new names.

Create a `Cluster` or `Shard` like this:

```diff
-let cluster = Cluster::new(token, intents).await?;
-let mut events = cluster.events();
+let (cluster, mut events) = Cluster::new(token, intents).await?;
```

Replace references to `UpdateStatus` and `UpdateStatusInfo` with
`UpdatePresence` and `UpdatePresencePayload` respectively.

Ensure at least one `Activity` is present in `UpdatePresence`.

Changes

The `zlib` feature choices have been renamed from `'stock-zlib` and
`simd-zlib` to `zlib-stock` and `zlib-simd` respectively ([#829] -
[@vivian]).

`Cluster::new`, `ClusterBuilder::build`, `Shard::new`, and
`ShardBuilder::build` now return a tuple with two elements: the cluster
or shard itself and a stream of events. See the PR for more details
([#832] - [@vivian]).

`UpdateStatus` and `UpdateStatusInfo` have been renamed to
`UpdatePresence` and `UpdatePresencePayload` respectively ([#902] -
[@7596ff]).

At least one `Activity` is required when building an `UpdatePresence`
payload.  `UpdatePresenceError` and `UpdatePresenceErrorType` have been
created to validate this ([#891] - [@7596ff]).

[#829]: #829
[#832]: #832
[#891]: #891
[#902]: #902

[@7596ff]: https://github.com/7596ff
[@vivian]: https://github.com/vivian
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c-gateway Affects the gateway crate m-breaking change Breaks the public API. t-docs A change to the documentation t-refactor Refactors APIs or code.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants