perl - グループ化よりも高速に個別を選択

Question

次の検索を実行したい:

schema->resultset('Entity')->search({ 
        -or => { "me.user_id" => $user_id, 'set_to_user.user_id' => $user_id } 
    }, {
        'distinct' => 1,
        'join' => {'entity_to_set' => {'entity_set' => 'set_to_user'}},
        'order_by' => {'-desc' => 'modified'},
        'page' => 1,'rows' => 100
    });

以下に示すようなテーブルを含むデータベース。

CREATE TABLE entity (
  id varchar(500) NOT NULL,
  user_id varchar(100) NOT NULL,
  modified timestamp NOT NULL,
  PRIMARY KEY (id, user_id),
  FOREIGN KEY (user_id) REFERENCES user(id) ON DELETE CASCADE ON UPDATE CASCADE
);

CREATE TABLE entity_to_set (
  set_id varchar(100) NOT NULL,
  user_id varchar(100) NOT NULL,
  entity_id varchar(500) NOT NULL,
  PRIMARY KEY (set_id, user_id, entity_id),
  FOREIGN KEY (entity_id, user_id) REFERENCES entity(id, user_id) ON DELETE CASCADE ON UPDATE CASCADE,
  FOREIGN KEY (set_id) REFERENCES entity_set(id) ON DELETE CASCADE ON UPDATE CASCADE
);

CREATE TABLE entity_set (
  id varchar(100) NOT NULL,
  PRIMARY KEY (id)
);

CREATE TABLE set_to_user (
  set_id varchar(100) NOT NULL,
  user_id varchar(100) NOT NULL,
  PRIMARY KEY (set_id, user_id),
  FOREIGN KEY (user_id) REFERENCES user(id) ON DELETE CASCADE ON UPDATE CASCADE,
  FOREIGN KEY (set_id) REFERENCES entity_set(id) ON DELETE CASCADE ON UPDATE CASCADE
);

CREATE TABLE user (
  id varchar(100) NOT NULL,
  PRIMARY KEY (id)
);

私は約 6000 entity、 6000 entity_to_set、 10 entity_set、および 50を持っていset_to_userます。

さて、残念なことに、このクエリには時間がかかります (1 ～ 2 秒)。を含むエンティティテーブルのみに対してクエリを実行するORDER BYと、結果はほとんど瞬時に得られます。これをデバッグするための最初のステップとして、DBIC コードが次のようになる実際の SQL クエリを見つけました。

SELECT me.id, me.user_id, me.modified FROM entity me
LEFT JOIN entity_to_set entity_to_set ON ( entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id ) 
LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id 
LEFT JOIN set_to_user set_to_user ON set_to_user.set_id = entity_set.id 
WHERE ( ( set_to_user.user_id = 'Craigy' OR me.user_id = 'Craigy' ) ) 
GROUP BY me.id, me.user_id, me.modified ORDER BY modified DESC LIMIT 100;

そしてここにの結果がありますEXPLAIN QUERY PLAN

0|0|0|SCAN TABLE entity AS me USING INDEX sqlite_autoindex_entity_1 (~1000000 rows)
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING COVERING INDEX entity_to_set_idx_cover (entity_id=? AND user_id=?) (~9 rows)
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows)
0|3|3|SEARCH TABLE set_to_user AS set_to_user USING COVERING INDEX sqlite_autoindex_set_to_user_1 (set_id=?) (~5 rows)
0|0|0|USE TEMP B-TREE FOR ORDER BY

どこentity_to_set_idx_coverですか

CREATE INDEX entity_to_set_idx_cover ON entity_to_set (entity_id, user_id, set_id);

ここでの問題は、結合を行っていないときに使用されるインデックスではなく、ソートに使用される b ツリーです。

DBIx::Classがステートメントに変換されていることに気付きました ('distinct' => 1ドキュメンテーションには、ここで同等であると書かれていると思います)。ステートメントを削除し、代わりに次のクエリを使用しましたGROUP BYGROUP BYSELECT DISTINCT

SELECT DISTINCT me.id, me.user_id, me.modified FROM entity me
LEFT JOIN entity_to_set entity_to_set ON ( entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id ) 
LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id 
LEFT JOIN set_to_user set_to_user ON set_to_user.set_id = entity_set.id 
WHERE ( ( set_to_user.user_id = 'Craigy' OR me.user_id = 'Craigy' ) ) 
ORDER BY modified DESC LIMIT 100;

同じ結果が得られると私は信じています。このEXPLAIN QUERY PLANクエリの

0|0|0|SCAN TABLE entity AS me USING COVERING INDEX entity_sort_modified_user_id (~1000000 rows)
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING COVERING INDEX entity_to_set_idx_cover (entity_id=? AND user_id=?) (~9 rows)
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows)
0|3|3|SEARCH TABLE set_to_user AS set_to_user USING COVERING INDEX sqlite_autoindex_set_to_user_1 (set_id=?) (~5 rows)

entity_sort_modified_user_idを使用して作成されたインデックスはどこですか

CREATE INDEX entity_sort_modified_user_id ON entity (modified, user_id, id);

これはほぼ瞬時に実行されます (b ツリーなし)。

編集:ORDER BYが昇順の場合でも問題が発生すること、およびインデックスがこれらのクエリに与える影響を示すために、同じテーブルに対する同様のクエリを次に示します。最初の 2 つのクエリはそれぞれとを使用SELECT DISTINCTしたインデックスなしGROUP BYで、次の 2 つは同じクエリとインデックスを使用したものです。

sqlite> EXPLAIN QUERY PLAN SELECT DISTINCT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON ( entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id ) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE ( me.user_id = 'Craigy' AND entity_set.id = 'SetID' ) ORDER BY modified LIMIT 100;
0|0|0|SCAN TABLE entity AS me (~100000 rows)
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows)
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows)
0|0|0|USE TEMP B-TREE FOR DISTINCT
0|0|0|USE TEMP B-TREE FOR ORDER BY
sqlite> EXPLAIN QUERY PLAN SELECT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON ( entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id ) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE ( me.user_id = 'Craigy' AND entity_set.id = 'SetID' ) GROUP BY me.id, me.user_id, me.modified ORDER BY modified LIMIT 100;
0|0|0|SCAN TABLE entity AS me USING INDEX sqlite_autoindex_entity_1 (~100000 rows)
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows)
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows)
0|0|0|USE TEMP B-TREE FOR ORDER BY
sqlite> CREATE INDEX entity_idx_user_id_modified_id ON entity (user_id, modified, id);
sqlite> EXPLAIN QUERY PLAN SELECT DISTINCT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON ( entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id ) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE ( me.user_id = 'Craigy' AND entity_set.id = 'SetID' ) ORDER BY modified LIMIT 100;
0|0|0|SEARCH TABLE entity AS me USING COVERING INDEX entity_idx_user_id_modified_id (user_id=?) (~10 rows)
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows)
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows)
sqlite> EXPLAIN QUERY PLAN SELECT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON ( entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id ) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE ( me.user_id = 'Craigy' AND entity_set.id = 'SetID' ) GROUP BY me.id, me.user_id, me.modified ORDER BY modified LIMIT 100;
0|0|0|SEARCH TABLE entity AS me USING COVERING INDEX entity_idx_user_id_modified_id (user_id=?) (~10 rows)
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows)
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows)
0|0|0|USE TEMP B-TREE FOR GROUP BY
0|0|0|USE TEMP B-TREE FOR ORDER BY

私の質問は、DBIx::Class コードを修正して、SELECT DISTINCTクエリと同じように実行するにはどうすればよいかということです。または、そのままの状態で機能するようにインデックスを追加するにはどうすればよいですか? それとも、他の種類の修正が必要ですか?

perl - グループ化よりも高速に個別を選択

1 に答える 1

Related

Reference