Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Give mv_max_matches (aka mm) some real teeth
* Make mv_max_matches stop all further searching once the limit is hit,
rather than loading the entire result set into memory and then truncating
it.

In my tests with a 700,000+ row products table, this is dramatically
faster when limiting results to 1000, but surprisingly it adds little
overhead even when still fetching the entire table.

* Add pragma max_matches, which takes precedence over user-supplied
mv_max_matches unless the user-supplied argument is more restrictive.

Please note that e.g. [pragma max_matches 1000] on a search landing page
will be processed too late to affect the search, because searches are
done in an ActionMap that runs before the page is seen.

You can use a catalog Pragma directive like this:

Pragma max_matches=1000

Which will affect everything in the catalog, including the admin.

If you want to make exceptions to the mv_max_matches limit based on URL,
you can instead use an Autoload that calls a GlobalSub, like this:

interchange.cfg:

GlobalSub <<EOR
sub set_pragma_max_matches {
    $::Pragma->{max_matches} = 1000
        unless $Vend::FinalPath =~ m{^/admin/};
    return;
}
EOR

catalog.cfg:

Autoload set_pragma_max_matches
  • Loading branch information
jonjensen committed Nov 17, 2009
1 parent dee1f77 commit 72809c4
Showing 1 changed file with 18 additions and 4 deletions.
22 changes: 18 additions & 4 deletions lib/Vend/DbSearch.pm
Expand Up @@ -232,7 +232,20 @@ sub search {
}

$s->save_specs();
foreach $searchfile (@searchfiles) {

# set max_matches based on the lower of the pragma & the search parameter
# (so end-users can further restrict the size of the result set, but not increase it)
my $max_matches;
{
no warnings 'uninitialized';
$max_matches = $::Pragma->{max_matches};
undef $max_matches if $max_matches < 1;
my $search_mm = $s->{mv_max_matches};
$max_matches = $search_mm
if $search_mm > 0 and (!$max_matches or $search_mm < $max_matches);
}

SEARCHFILE: for $searchfile (@searchfiles) {
my $lqual = $qual || '';
$searchfile =~ s/\..*//;
my $db;
Expand Down Expand Up @@ -276,6 +289,7 @@ sub search {
while($ref = $dbref->each_nokey($lqual) ) {
next unless $limit_sub->($ref);
push @out, $return_sub->($ref);
last SEARCHFILE if $max_matches and @out >= $max_matches;
}
}
elsif(defined $limit_sub) {
Expand All @@ -287,6 +301,7 @@ sub search {
next unless &$f();
next unless $limit_sub->($ref);
push @out, $return_sub->($ref);
last SEARCHFILE if $max_matches and @out >= $max_matches;
}
}
elsif (!defined $f) {
Expand All @@ -301,6 +316,7 @@ sub search {
$_ = join "\t", @$ref;
next unless &$f();
push @out, $return_sub->($ref);
last SEARCHFILE if $max_matches and @out >= $max_matches;
}
}
$s->restore_specs();
Expand Down Expand Up @@ -329,9 +345,7 @@ sub search {
@out = grep ! $seen{$_->[0]}++, @out;
}

if($s->{mv_max_matches} and $s->{mv_max_matches} > 0) {
splice @out, $s->{mv_max_matches};
}
splice @out, $max_matches if $max_matches;

$s->{matches} = scalar(@out);

Expand Down

0 comments on commit 72809c4

Please sign in to comment.