UnQLite Users Forum

Infinite loop in lhPageFreeSpace. (Thread Safety ?)

append delete snoits

Got an issue with the library while using it.

I'm using it in multi-threaded mode with 12 threads that do fetch and store. One of them do the begin and commit.

At some point the database seems to be corrupted.

I've noticed that passing a negative value length (-1) will corrupt the database, I tough at that point that the value had the same behavior than the key but it's not the vase (search for the first null value when a negative value is provided).

My question is, is there any other value that should not be feed, like zero that could corrupt the database ?

I'm on Windows and the DB is not that big, few megabytes. I'm not relying yet on Jx9 scripting API but I was thinking of using it later since the stored data type is Json values.

The mutex seems to work properly. Since when I'm in the infinite loop, all my other thread that poke the db are stuck on the internal mutex.

So I'm stuck in that loop

for(;;){
/* Offset of the next free block */
SyBigEndianUnpack16(zRaw,&iNext);
zRaw += 2;
/* Available space on this block */
SyBigEndianUnpack16(zRaw,&iAmount);
nFree += iAmount;
if( iNext < 1 ){
/* No more free blocks */
break;
}
/* Point to the next free block*/
zRaw = &pPage->pRaw->zData[iNext];
if( zRaw >= zEnd ){
/* Corrupt page */
return UNQLITE_CORRUPT;
}
}

I'm in release mode so with that in mind
iAmount = 26
nFree = 42518
iNext = 12
zRaw[0] = 0
zRaw[1] = 12 // Make me think that iNext value is valid
zRaw[2] = 0
zRaw[3] = 26 // Make me think that iAmount value is valid
zRaw == &pPage->pRaw->zData[iNext] // Meaning it point on itself

Thanks in advance for your support.

:: @snoits added on 07 Dec ’15 · 18:22

By negative value (-1). I mean of lenght -1 as for the key lenght. -1 begin a special value that let the API look for the data len by itself.

:: @snoits added on 07 Dec ’15 · 19:28

Another detail to be considered. I was not doing the begin at first. Only dealing with the fetch, store and commit command at first. I'm now adding the begin in it.

I also noticed that the database stored was corrupted. Once I delete it, for the last few test I've done, I'm not able to reproduce the problem yet.

Reply RSS

Replies

append delete #1. chm

Hey,

Passing -1 is intended only for null terminated key strings '\0' and not the data since data can be take any form i.e. Binary, Text, etc. so avoid -1 and try to reproduce this scenario using a fresh new DB.

Also, make sure that UnQLite is compiled with threading support (UNQLITE_ENABLE_THREADS directive) and the run-time is set to multi-threading via unqlite_lib_config(UNQLITE_LIB_CONFIG_THREAD_LEVEL_MULTI); before doing any serious work (First call to UnQLite).

http://unqlite.org/c_api_const.html#compile_time
http://unqlite.org/c_api/unqlite_lib.html

append delete #2. h

Yea

append delete #3. snoits

Hi unqlite team,

I finally had to fix the check done for corrupted pages.

From
zRaw >= zEnd

To
zRaw >= zEnd || nFree > pPage->pHash->iPageSize

I had a case, where internal linked list of free block was circular. And so the first check was not enough.

So there's a bug somewhere that create that circular linked list. If at some point you figure out what it's I will gladly take the fix and remove that check.

append delete #4. devel

Reply

(Leave this as-is, it’s a trap!)

There is no need to “register”, just enter the same name + password of your choice every time.

Pro tip: Use markup to add links, quotes and more.

Your friendly neighbourhood moderators: chm_at_symisc, devel_at_symisc