why mdbx_env_set_option MDBX_opt_sync_period may cause Assertion failed: (data->iov_len == 0 && olddata.iov_len == 0), function mdbx_cursor_put, file mdbx, line 18172.

[gcxfd]

I'm write a wrap for libmdbx in rust , I found when use MDBX_opt_sync_period 65536, open env with MDBX_SAFE_NOSYNC

then when I set a MDBX_DUPSORT value with the same value the second time

some times (not every time) may cause

Assertion failed: (data->iov_len == 0 && olddata.iov_len == 0), function mdbx_cursor_put, file mdbx, line 18172.

I view the c code , found error cause from there (and the real error is olddata.iov_len!=0)

      if (likely(rc == MDBX_SUCCESS)) {
        if (exact) {
          if (unlikely(flags & MDBX_NOOVERWRITE)) {
            mdbx_debug("duplicate key [%s]", DKEY_DEBUG(key));
            *data = olddata;
            return MDBX_KEYEXIST;
          }
          if (unlikely(mc->mc_flags & C_SUB)) {
            /* nested subtree of DUPSORT-database with the same key,
             * nothing to update */
            mdbx_assert(env, data->iov_len == 0 && olddata.iov_len == 0);
            return MDBX_SUCCESS;

but when I comment mdbx_env_set_option MDBX_opt_sync_period , still use MDBX_SAFE_NOSYNC, the error never happened

code https://github.com/rmw-lib/mdbx/tree/5a7ee9598f6b99342eefa9eae17df886c7f56238
screenshot

[erthink]

I assume that the reason is that you are using a transaction or a cursor from different threads.
And using MDBX_opt_sync_period simply reveals such error, since it changes the latency of some transactions commitment.

[erthink]

I had spent some time for digging your issue.
During this I couldn't reproduce your problem, but the one oddity was noticed:

[ 31855 child_1.1  erro ] mdbx: fatal failure: mdbx_sync_locked, 10771
[ 31855 child_1.1  fail ] mdbx_sync_locked: assert: memcmp(&head->mm_dbs, &pending->mm_dbs, sizeof(head->mm_dbs)) == 0
FAILED

So I continued to dig further, but so far I have not been able to reproduce the error locally or find a potential cause.
For now I suspect that the noticed failure on CI is related to an old version of GCC 7.x or Ubuntu 18.04, but I will continue digging when got a time.

However, along the way I rewrote/simplified one of the functions (the new code is now available in the devel branch).
It will be nice if you try this version.

If the problems persist, then for further analysis I need a stack trace of assertion failure from build with the MDBX_DEVEL=2 option (you may just hack it here).

[gcxfd]

@erthink

I run a docker to show this error , you can ssh login see the error

ssh root@39.103.201.100 -p 2222

password is 4ga3agYdpP

when ssh login , run tmux att can see error

the c file is ~/mdbx-ffi/libmdbx/mdbx.c

cd /root/mdbx-issue-248 
cargo run --example main

can build and run demo

[gcxfd]

by the way , I found another error

main: mdbx:17316: mdbx_cursor_set: Assertion `!"Invalid key-size"' failed.

can reshow in the above ssh machine

in dir
/root/mdbx-invalid-key-size

cargo run --example main

see src/range.rs line 54

when run
self.cursor.get(key, &mut self.val, op)
with op = MDBX_SET_LOWERBOUND and key = [1, 0, 0, 0]

happend

main: mdbx:17316: mdbx_cursor_set: Assertion `!"Invalid key-size"' failed.

this database flag is MDBX_DUPFIXED|MDBX_DUPSORT|MDBX_INTEGERDUP|MDBX_INTEGERKEY

edit ~/mdbx-ffi/libmdbx/mdbx.c can debug

[erthink]

I have reproduced both cases:

1. The Assertion failed: (data->iov_len == 0 && olddata.iov_len == 0), function mdbx_cursor_put, file mdbx, line 18172. is an extra assertion which should removed or refined, i.e. the bug is the assertion itself but nothing else.
   The unusual thing is that the problem is reproduced only on a debug build and when trying to re-insert a complete duplicate of a key-value (when the given value is already exists for a given key).
   Actually your test case has a set of oddities and so reproduce the problem on the third run from the clean state (after removal the DB).
   I will fix this issue soon, but you should pay attention to your binding code: for now it don't properly close DB and don't pass flags as specified in the example source code.

2. The mdbx:17316: mdbx_cursor_set: Assertion !"Invalid key-size" failed is assertion which catch wrong API usage, i.e. helps to debug your code.
   Using MDBX_SET_LOWERBOUND for a data set with duplicates assumes that a valid value is passed in the data argument, which will be used to position the cursor among the values (i.e. duplicates) if the requested key exists.
   But in your example the data={nullptr, 0} is used with MDBX_DUPSORT dataset.
   I will think about how to make the libmdbx behavior less unexpected and more convenient.

Thank for reporting.

[erthink]

@gcxfd, please check and close issue on succes.

[gcxfd]

thanks

I add the flag for env by proc_macro
https://github.com/rmw-lib/mdbx/blob/master/mdbx-proc/src/lib.rs

I closeed the database by

impl Drop for Env {
  fn drop(self: &mut Env) {
    ok_or_log!(mdbx_env_close_ex(self.0, false));
  }
}

https://github.com/rmw-lib/mdbx/blob/0c7f3ed8298ba242506baa1cfb4d0cbfd0c14499/src/env.rs#L182

I will write document for those when finish coding