Reduce Heap Allocations for CRUD operations

[tomijaga]

Problem

The heap allocations are about 26x (160MB for insertions over 6.9MB) more in ZenDB than in the MemoryB+Tree.
This limits the amount of documents we can process in memory. We need to look through the ZenDB codebase to find areas that use a lot of heap memory and try to optimize them.

Memory B+Tree Compared to the Heap B+Tree Performance

Instructions

	insert()	get()	replace()	entries()	remove()
B+Tree	180_319_890	94_999_826	101_546_816	3_558_054	203_223_917
Memory B+Tree	291_304_055	238_893_750	508_478_474	33_196_692	479_669_938

Garbage Collection

	insert()	get()	replace()	entries()	remove()
B+Tree	330.27 KiB	172.64 KiB	638.32 KiB	16.49 KiB	172.65 KiB
Memory B+Tree	6.88 MiB	3.38 MiB	6.83 MiB	1.08 MiB	8.91 MiB

ZenDB Performance

Instructions

	#stableMemory no index (no sorting)	#stableMemory no index (sorted by ts)	#stableMemory 7 single field indexes (sorted by ts)	#stableMemory 6 fully covered indexes (sorted by ts)
insert with no index	2_256_263_411	2_256_267_654	2_256_174_111	2_256_183_011
create and populate indexes	2_202	5_228	15_208_900_242	17_895_052_363

Garbage Collection

	#stableMemory no index (no sorting)	#stableMemory no index (sorted by ts)	#stableMemory 7 single field indexes (sorted by ts)	#stableMemory 6 fully covered indexes (sorted by ts)
insert with no index	160.56 MiB	160.56 MiB	160.56 MiB	160.56 MiB
create and populate indexes	16.36 KiB	16.36 KiB	1.12 GiB	1.4 GiB

[tomijaga]

Related - NatLabs/serde#40