Query using LIKE '% term %' is very slow

Question

I have a InnoDB table, about 1,000,000 rows:

Data            164.7   MiB
Index           250.1   MiB
Overhead        168.0   MiB
Effective       246.8   MiB
Total           414.8   MiB

Table structure:

CREATE TABLE IF NOT EXISTS `gift` (
  `tm` varchar(15) NOT NULL,
  `col` smallint(2) NOT NULL,
  `myindex` varchar(255) NOT NULL,
  `date` int(10) NOT NULL,
  KEY `tm` (`tm`),
  KEY `date` (`date`),
  KEY `myindex` (`myindex`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

tm is a un-duplicate string generated by PHP (products id),

date store use strtotime('now'),

myindex store search keywords

because some products have long descriptions(more than 255 characters), so col is separate long descriptions with many parts (the same products, keywords part 1, keywords part 2...).

A query like this will make (10 total, Query took 51.0829 sec)

SELECT * 
FROM gift
WHERE myindex LIKE  '% keywords1 %'
AND myindex LIKE  '% keywords2 %'
GROUP BY tm
ORDER BY DATE DESC 
LIMIT 10

make an explain, that get:

id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   SIMPLE      gift    index   NULL            date    4   NULL 10 Using where; Using temporary

possible_keys is null, Have I made a bad index? How to make it fastest?

here is my.cnf, my server has 4GB ram, mysql also support fulltext search.

And I tried set key_buffer_size and innodb_buffer_pool_size to 512MB, but the server would easly crash down with warning apache child process still did not exit sending a sigterm. Must I left at least 1GB ram for system and apache? (Only about 2.5-3GB for my.cnf?)

[mysqld]
character_set_server=utf8
port        = 3306
socket      = /var/lib/mysql/mysql.sock
skip-external-locking
skip-networking
key_buffer = 256M
tmp_table_size = 64M
max_connections = 300
wait_timeout=15
back_log = 2048
key_buffer_size = 384M
max_allowed_packet = 2M
table_cache = 2048
table_open_cache = 2048
sort_buffer_size = 6M
read_buffer_size = 4M
net_buffer_length = 92K
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 256M
thread_cache = 384
query_cache_size= 256M
bulk_insert_buffer_size = 192M
ft_min_word_len=2
skip-networking

binlog_format=mixed

innodb_buffer_pool_size = 384M
innodb_buffer_pool_instances=4

innodb_use_sys_malloc = 0

[mysqldump]
quick
max_allowed_packet = 8M

[mysql]
no-auto-rehash

[myisamchk]
key_buffer_size = 96M
sort_buffer_size = 96M
read_buffer = 3M
write_buffer = 3M

Answer 1

First of all, you are using an illegal grouping. check first paragraph of http://dev.mysql.com/doc/refman/5.0/en/group-by-extensions.html

Second, seems like the group by is not even necessary, and is causing your 'temporary' to show in explain

Third,(don't quote me o nthis, not 100% sure) your temorary tmp_table_size = 64M may be too small for this query alone, as it stores the filtered results in memory before grouping

4th, if you do need to group, assign an integer to that string, and group on integer.

5th, if you have keywords, store them in an item-keyword table, and use them with a join. Will give you some performance hit due to the join, but it will allow you to use an index. Indexes only come into play when you are comparing strings from the beginning (hence using a length 6-10 index on varchar might be just as effective as using a 100 length index)

happy optimising :)

Answer 2

Your actual problem is that it is not possible for MySQL to use an index for a condition such as myindex LIKE '% keywords1 %'. This topic is covered far and wide on the web, so I'd suggest you look for full-text indexing solution. You can use either one internal (indexes in MySQL) or external (Sphinx, Lucene, etc.).