HeliDB 1.1 test report

2009.07.26
Karl Gustafsson

Unit tests
  64 bit Linux
  32 bit Linux
  64 bit Solaris 10
  Mac OS X
  64 bit Windows Vista
  32 bit Windows Vista

Performance tests
  Compare graphs

Unit tests

Unit tests were run as described in the Programmer's guide.

64 bit Linux

OS:: Xubuntu 9.04 64 bit, Linux 2.6.28-11-generic SMP
File system:: tmpfs
Hardware:: Intel Core 2 Quad Q9550 @ 2.83GHz
8192 Mb RAM
Western Digital Raptor (3.5", 10,000 RPM, SATA)

JVM	Results
Sun JDK 1.5.0_19 (64 bit)	OK, 11/11
Sun JDK 1.6.0_14 (64 bit)	OK, 12/12
IBM J9 JVM build 2.3 1.5.0 SR9-SSU (64 bit)	OK, 11/11
IBM J9 JVM build 2.4 1.6.0 SR5 (64 bit)	OK, 12/12
OpenJDK 6b14-1.4.1-0ubuntu10 (64 bit)	OK, 12/12

32 bit Linux

OS:: Xubuntu 9.04 32bit, Linux 2.6.28-13-generic
File system:: tmpfs
Hardware:: Intel Pentium M 2.0 GHz stepping 08
2048 Mb RAM
Hitachi HTE726060M9AT00 (2.5", 7,200 RPM, ATA)

JVM	Results
Sun JDK 1.5.0_19	OK, 11/11
Sun JDK 1.6.0_14	OK, 12/12
IBM J9 JVM build 2.3 1.5.0 SR9-SSU	OK, 11/11
IBM J9 JVM build 2.4 1.6.0 SR5	OK, 12/12
OpenJDK 6b14-1.4.1-0ubuntu10	OK, 12/12

64 bit Solaris 10

OS:: Solaris 10 5/09
File system:: UFS
Hardware:: Intel Xeon E5420 @ 2.5GHz
8192 Mb RAM
RAID 10 array

JVM	Results
Sun JDK 1.5.0_19 (64 bit)	OK, 11/11
Sun JDK 1.6.0_14 (64 bit)	OK, 12/12

Mac OS X

OS:: Mac OS X 10.5.7
File system:: HFS
Hardware:: Intel Core 2 Duo @ 2GHz
2048 Mb RAM

JVM	Results
Apple JDK 1.5.0_19_b02-304 (64 bit)	OK, 11/11
Apple JDK 1.6.0_13_b03-211 (64 bit)	OK, 12/12

64 bit Windows Vista

OS:: Windows Vista Business SP2 64bit
File system:: NTFS
Hardware:: Intel Core 2 Duo T7500 @ 2.2GHz
4096 Mb RAM
Seagate ST9160823AS (2.5", 7,200 RPM, SATA)

JVM	Results
Sun JDK 1.5.0_19 (64 bit)	OK, 11/11
Sun JDK 1.6.0_14 (64 bit)	OK, 12/12

32 bit Windows Vista

OS:: Windows Vista Enterprise SP2
File system:: NTFS
Hardware:: Intel Core2 Duo U7600 @ 1.2Ghz
2048 Mb RAM
Toshiba MK1011GAH (1.8", 4,200 RPM, ATA)

JVM	Results
Sun JDK 1.5.0_19	OK, 11/11
Sun JDK 1.6.0_14	OK, 12/12

Performance tests

Performance tests were run as described in the Programmer's guide. The tests suite contains the following tests, run for n databases in each suite:

Insert x records in an empty database using Database.insert. Commit.
Find x records from the database using Database.get.
Iterate over x keys using Database.keyIterator.
Iterate over x records using Database.iterator.
Insert y additional records using Database.insert. Commit.
Update the y additional records using Database.update. Commit.
Delete the y additional records using Database.delete. Commit.
Update x records using Database.update. Commit.
Delete x/3 records (every 3rd record) using Database.delete. Commit.
Compact the database using Database.compact. Commit.

The tests were run for the following configurations:

#	Database	DatabaseBackend	Additional settings
1	SimpleDatabase	ConstantRecordSizeHeapBackend
2	SimpleDatabase	BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend	B+ Tree node size: 4096 bytes
3	SimpleDatabase	BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
4	SimpleDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes
5	SimpleDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes, node cache: 10 nodes
6	SimpleDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
7	SimpleDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes, node cache: 50 nodes
8	SimpleDatabase	HeapBackend
9	SimpleDatabase	BPlusTreeIndexBackend and HeapBackend	B+ Tree node size: 4096 bytes
10	SimpleDatabase	BPlusTreeIndexBackend and HeapBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
11	LoggingTransactionalDatabase	ConstantRecordSizeHeapBackend
12	LoggingTransactionalDatabase	BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend	B+ Tree node size: 4096 bytes
13	LoggingTransactionalDatabase	BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
14	LoggingTransactionalDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes
15	LoggingTransactionalDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
16	LoggingTransactionalDatabase	HeapBackend
17	LoggingTransactionalDatabase	BPlusTreeIndexBackend and HeapBackend	B+ Tree node size: 4096 bytes
18	LoggingTransactionalDatabase	BPlusTreeIndexBackend and HeapBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
19	ShadowCopyTransactionalDatabase	ConstantRecordSizeHeapBackend
20	ShadowCopyTransactionalDatabase	BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend	B+ Tree node size: 4096 bytes
21	ShadowCopyTransactionalDatabase	BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
22	ShadowCopyTransactionalDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes
23	ShadowCopyTransactionalDatabase	ConstantRecordSizeBPlusTreeBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes
24	ShadowCopyTransactionalDatabase	HeapBackend
25	ShadowCopyTransactionalDatabase	BPlusTreeIndexBackend and HeapBackend	B+ Tree node size: 4096 bytes
26	ShadowCopyTransactionalDatabase	BPlusTreeIndexBackend and HeapBackend	B+ Tree node size: 4096 bytes, node cache: 30 nodes

The tests for each database configuration were run with the following values for n, x and y, with different maximum y values for different test suites.

n	x	y
500	0	5
500	10	5
500	50	5
200	100	5
100	500	10
100	1,000	10
20	5,000	50
10	10,000	50
1	50,000	100
1	100,000	100
1	500,000	100
1	1,000,000	100
1	5,000,000	100

The tests with constant record size backends were run with Integer keys and Long values. The tests with variable record size backends were run with String keys and values varying randomly between 4 and 20 characters in size. (All test data is generated before the tests are run.)

The test were run on JDK 1.6.0_14 on a 64 bit Linux.

SimpleDatabase, ConstantRecordSizeHeapBackend

SimpleDatabase, BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend

B+ Tree node size = 4096 bytes, no LRU node cache

SimpleDatabase, BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

SimpleDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, no LRU node cache

SimpleDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, LRU node cache = 10 nodes

SimpleDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

SimpleDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, LRU node cache = 50 nodes

SimpleDatabase, HeapBackend

SimpleDatabase, BPlusTreeIndexBackend and HeapBackend

B+ Tree node size = 4096 bytes, no LRU node cache

SimpleDatabase, BPlusTreeIndexBackend and HeapBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

LoggingTransactionalDatabase, ConstantRecordSizeHeapBackend

LoggingTransactionalDatabase, BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend

B+ Tree node size = 4096 bytes, no LRU node cache

LoggingTransactionalDatabase, BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

LoggingTransactionalDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, no LRU node cache

LoggingTransactionalDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

LoggingTransactionalDatabase, HeapBackend

LoggingTransactionalDatabase, BPlusTreeIndexBackend and HeapBackend

B+ Tree node size = 4096 bytes, no LRU node cache

LoggingTransactionalDatabase, BPlusTreeIndexBackend and HeapBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

ShadowCopyTransactionalDatabase, ConstantRecordSizeHeapBackend

ShadowCopyTransactionalDatabase, BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend

B+ Tree node size = 4096 bytes, no LRU node cache

ShadowCopyTransactionalDatabase, BPlusTreeIndexBackend and ConstantRecordSizeHeapBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

ShadowCopyTransactionalDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, no LRU node cache

ShadowCopyTransactionalDatabase, ConstantRecordSizeBPlusTreeBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

ShadowCopyTransactionalDatabase, HeapBackend

ShadowCopyTransactionalDatabase, BPlusTreeIndexBackend and HeapBackend

B+ Tree node size = 4096 bytes, no LRU node cache

ShadowCopyTransactionalDatabase, BPlusTreeIndexBackend and HeapBackend

B+ Tree node size = 4096 bytes, LRU node cache = 30 nodes

Compare graphs

[Linear axes]
[Logarithmic x-axis]
[Logarithmic y-axis]
[Logarithmic x- and y-axes]