Skip to content

Commit

Permalink
Fix alignment in chunkedContentCoder (#147)
Browse files Browse the repository at this point in the history
An unaligned atomic bug was unfortunately introduced in #119 because the
`bytesWritten` field was placed at the end of the `chunkedContentCoder`
struct.

This places this after the bytes.Buffers and the bool causing it to be
misaligned.

The ideal placement of this variable is not entirely clear but placing
it before the progresiveWrite bool should help.

An alternative would be to just place this atomic field at the top of
the struct then there would be no risk of it becoming misaligned in
future.

I moved a few things around to reduce the size of the struct too but it
could be possible to adjust things a little more to make the struct a
little smaller.

Signed-off-by: Andrew Thornton <art27@cantab.net>
  • Loading branch information
zeripath committed Nov 30, 2022
1 parent 96a016b commit e34dce0
Showing 1 changed file with 10 additions and 11 deletions.
21 changes: 10 additions & 11 deletions contentcoder.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,27 +31,26 @@ func init() {
reflectStaticSizeMetaData = int(reflect.TypeOf(md).Size())
}

var termSeparator byte = 0xff
var termSeparatorSplitSlice = []byte{termSeparator}
var (
termSeparator byte = 0xff
termSeparatorSplitSlice = []byte{termSeparator}
)

type chunkedContentCoder struct {
final []byte
chunkSize uint64
currChunk uint64
chunkLens []uint64

compressed []byte // temp buf for snappy compression

w io.Writer
bytesWritten uint64 // atomic access to this variable
progressiveWrite bool

chunkMeta []MetaData
chunkMetaBuf bytes.Buffer
chunkBuf bytes.Buffer

chunkMeta []MetaData

compressed []byte // temp buf for snappy compression

// atomic access to this variable
bytesWritten uint64
}

// MetaData represents the data information inside a
Expand All @@ -64,7 +63,8 @@ type MetaData struct {
// newChunkedContentCoder returns a new chunk content coder which
// packs data into chunks based on the provided chunkSize
func newChunkedContentCoder(chunkSize uint64, maxDocNum uint64,
w io.Writer, progressiveWrite bool) *chunkedContentCoder {
w io.Writer, progressiveWrite bool,
) *chunkedContentCoder {
total := maxDocNum/chunkSize + 1
rv := &chunkedContentCoder{
chunkSize: chunkSize,
Expand Down Expand Up @@ -191,7 +191,6 @@ func (c *chunkedContentCoder) Add(docNum uint64, vals []byte) error {
//
// | ..... data ..... | chunk offsets (varints)
// | position of chunk offsets (uint64) | number of offsets (uint64) |
//
func (c *chunkedContentCoder) Write() (int, error) {
var tw int

Expand Down

0 comments on commit e34dce0

Please sign in to comment.