There isn't really a good way to do this. mailer.py is designed so that any commit that isn't matched to another group goes to the defaults group.
The documentation in mailer.conf.example hints at this but doesn't really explain it very well:
The options specified in the [defaults] section are always selected. The
presence of a non-matching for_repos has no relevance. Note that you may
still use a for_repos value to extract useful information (more on this
later). Any user-defined groups without a for_repos, or which contains
a matching for_repos, will be selected for potential use.
The subset of user-defined groups identified by the repository are further
refined based on the for_paths option. A group is selected if at least
one path(*) in the commit matches the for_paths regular expression. Note
that the paths are relative to the root of the repository and do not
have a leading slash.
What is says for for_repos also applies to for_paths with respect to the defaults group. I.E. that that for_paths is only useful for variable extraction.
One option without making any code changes would be to set your to_addr in your [defaults] to an address like devnull@example.com which you just throw away. Then set a different group up with a different to_addr that will actually be delivered someplace.
If you're willing to modify your mailer.py a tad you can avoid this by commenting out the following two lines in the which_groups function of the Config class:
if not groups:
groups.append((None, self._default_params))
As a Subversion developer long term I think we should add an option to mailer.py to request that no mail be generated by the defaults section. Additionally, we should fix the documentation to be clearer about this behavior.