Multiple column index vs multiple indexes

After my previous post there were questions raised about Index Merge on Multiple Indexes vs Two Column Index efficiency. I mentioned in most cases when query can use both of the ways using multiple column index would be faster but I also went ahead to do some benchmarks today.

I’m using couple of simple tables:

CREATE TABLE `t1000idxmerge` (
  `i` int(11) NOT NULL,
  `j` int(11) NOT NULL,
  `val` char(10) NOT NULL,
  KEY `i` (`i`),
  KEY `j` (`j`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
CREATE TABLE `t1000idx2` (
  `i` int(11) NOT NULL,
  `j` int(11) NOT NULL,
  `val` char(10) NOT NULL,
  KEY `i` (`i`,`j`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

I have populated this table with random data for i and j having both of them having 1000 of distinct values, independent on each other. I also created couple of other tables with same data just with really low cardinality with i and j having just 3 values each. The table contained about 18M rows though was small enough to fit in the systems memory.

I’ve benchmarked simple queries using where clause which covers multiple columns:

Q1 SELECT sum(length(val)) FROM  T  WHERE i=2 AND j=1
Q2 SELECT sum(length(val)) FROM  T  WHERE i=2 AND j BETWEEN 1 and 2
Q3 SELECT sum(length(val)) FROM  T  WHERE j=2 AND i BETWEEN 1 and 2
Q4 SELECT sum(length(val)) FROM  T  WHERE i=2 OR j=1
Q5 SELECT sum(length(val)) FROM  T  WHERE j=2 AND i BETWEEN 100 and 200

As some of them there way too fast if run once I ran them multiple times and measured time appropriately. To remove the overhead of starting MySQL etc from equation I also measured execution of “SELECT 1″ query using same script and subtracted this time from result in the table.

time for ((i=0;i<100;i+=1)); do mysql test -e "SELECT sum(length(val)) FROM t1000idxmerge WHERE i=2 AND j=1"; done > /dev/null

In the result table I compute per query results and present results in milliseconds.

Note1: Q1 will not use Index Merge technique for low cardinality table but instead pick to do single index scan. I’m not aware of the optimizer hint which would allow to force index merge as you can do with index accesses in general.

Note2 Q2/Q3 can’t use Index Merge however as it is currently implemented so they would use single index range scan. The 2 indexes however benefits to Q3 because it can only use first keypart of index (j,i) to resolve BETWEEN part of the clause which may not be very selective.

Note3 You may be surprised why 2 column index is faster for Q3 in case of low cardinality even though MySQL can’t use index well. You’re right MySQL can’t and MySQL does not – Full table scan is performed and in this case turns to be faster than scanning 1/5th of the table using index. Also Full Table Scan is preferred for Q4 in all cases but in case of high cardinality multiple index configuration.

Note4 Q5 was just run on high cardinality tables to show what difference large BETWEEN can make.

Conclusion: For benchmarked queries we can see Multiple Column index beats Index Merge in all cases when such index can be used. It is also worth to watchout a MySQL may decide not to do Index merge (either intersection or union) but instead do full table scan or access table picking only one index on the pair.

The post Multiple column index vs multiple indexes appeared first on MySQL Performance Blog.