memcached is required for mtpolicyd. A package of memcached should come with your os distribution. On Debian based distributions it can be installed by:
apt-get install memcached
Check /etc/default/memcached if the service is enabled:
ENABLE_MEMCACHED=yes
Start the memcached service
/etc/init.d/memcached start
Packages for mtpolicyd are included in the Debian package repositories.
Since mtpolicyd source is shipped as a perl/CPAN package it could be installed from CPAN. To install the Mail::Mtpolicyd package with all dependencies required make sure you have installed cpanminus:
apt-get install cpanminus
Then install the Mail::Mtpolicyd distribution with:
cpanm Mail::MtPolicyd
It is recommended to create an system user and group for the daemon.
You can get a default configuration file etc/mtpolicyd.conf from the tarball.
The init scripts for the debian packages are located at debian/mtpolicyd.init and for redhat systems at rpm/mtpolicyd.init within the tarball.
Now the daemon should be up:
$ ps -u mtpolicyd f
PID TTY STAT TIME COMMAND
2566 ? Ss 0:12 /usr/bin/mtpolicyd (master)
2731 ? S 0:28 \_ /usr/bin/mtpolicyd (idle)
19464 ? S 0:26 \_ /usr/bin/mtpolicyd (idle)
28858 ? S 0:26 \_ /usr/bin/mtpolicyd (idle)
32372 ? S 0:24 \_ /usr/bin/mtpolicyd (idle)
And it should be listening on localhost:12345:
$ netstat -aenpt | grep :12345 tcp 0 0 127.0.0.1:12345 0.0.0.0:* LISTEN 0 17333578 -
Now test it with a simple query:
$ policyd-client -h localhost:12345
Paste the following request to the command:
reverse_client_name=smtp.google.com sender=bob@gmail.com client_address=192.168.1.1 recipient=ich@markusbenning.de helo_name=smtp.google.com
Terminate the request by a blank line. Just press enter.
The mtpolicyd should respond with a action like:
PREPEND X-MtScore: NO score
Open you postfix main.cf configuration file in a text editor. It should be located below /etc/postfix.
Add a 'check_policyd_service inet:127.0.0.1:12345' check to your smtpd_recipient_restrictions. It should look like this one:
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service inet:127.0.0.1:12345
Now restart postfix. Now follow your maillog as new mails arrive. There should be a mtpolicyd line for every query.
Your mtpolicyd is now configured and running with the default configuration.
You may now want to continue with reading Mail::MtPolicyd::Cookbook::DefaultConfig which explains what the default configuration does.
mtpolicyd makes use of Moose. If you're not yet familiar with Moose you should start reading the Moose::Intro first.
A plugin in mtpolicyd is basically a class which inherits from Mail::MtPolicyd::Plugin and is located below the Mail::MtPolicyd::Plugin:: namespace:
package Mail::MtPolicyd::Plugin::HelloWorld;
use Moose;
use namespace::autoclean;
# VERSION
# ABSTRACT: a mtpolicyd plugin which just returns a hello world reject
extends 'Mail::MtPolicyd::Plugin';
use Mail::MtPolicyd::Plugin::Result;
sub run {
my ( $self, $r ) = @_;
return Mail::MtPolicyd::Plugin::Result->new(
action => 'reject Hello World!',
abort => 1,
);
}
__PACKAGE__->meta->make_immutable;
1;
Every plugin must implement a run() method. mtpolicyd will call run() every time your module is called from the configuration to process a request. A Mail::MtPolicyd::Request object containing the current request is passed to the method. The run() method must return undef or a <Mail::MtPolicyd::Plugin::Result> object. If undef is return mtpolicyd will continue with the next plugin. If a result is returned mtpolicyd will push the result to the list of results and abort processing the request if abort is set.
After you placed the module with your lib search path you should be able to use the plugin within mtpolicyd.conf:
<Plugin hello-world>
module = "HelloWorld"
</Plugin>
For now our plugin will just return an "reject Hello World!" action to the MTA.
All options defined in the configuration file will be passed to the object constructor new() when creating an object of your plugin class.
The parameter "module" is not passed to the object constructor because it contains the name of your class.
You can defined configuration parameters by adding attributes to your class.
You're class already inherits 3 attributes from the Plugin base class:
Which contains the name of your <Plugin> section.
Which contains the level used when your plugin calls $self->log( $r, '...');.
Tells mtpolicyd what to do when the plugin dies.
If set to "continue" mtpolicyd will continue processing and just leaves a line in the log.
Add a new attribute to your plugin class:
has 'text' => ( is => 'rw', isa => 'Str', default => 'Hello World!');
Return this string instead of the hard coded string:
action => 'reject '.$self->text,
The string is now configurable from the configuration:
<Plugin hello-world>
module = "HelloWorld"
text = "Hello Universe!"
</Plugin>
How to achieve common task within your plugin.
You can output log messages to the mail.log from within your plugin by calling:
$self->log( $r, '<log message>' );
The log() method is inherited from the Plugin base class.
The default log_level used is 4.
To debug your plugin you can overwrite the log_level in your plugins configuration:
<Plugin hello-world>
module = "HelloWorld"
log_level = 2
</Plugin>
This will cause your plugin to log with an higher log_level of 2.
If your plugin is called from the smtpd_recipient_restrictions if will be called once for every recipient. If your plugin does an lookup (dns, database, ...) should cache the result.
The Mail::MtPolicyd::Request object implements the method do_cached() to achieve this:
my ( $ip_result, $info ) = $r->do_cached('rbl-'.$self->name.'-result',
sub { $self->_rbl->check( $ip ) } );
The first parameter is the key in the session to store the cached result. The second parameter is a function reference.
It will check if there's already an result stored in the given key within the session. In this case it will return the cached result as an array. If there is no result it will execute the code reference, store the result within the session and will also return an array containing the return values of the result.
If your plugin is called from the smtpd_recipient_restrictions if will be called once for every recipient but some tasks should only be performed once per mail.
The Mail::MtPolicyd::Request object implements the method is_already_done() to achieve this:
if( defined $self->score && ! $r->is_already_done( $self->name.'-score' ) ) {
$self->add_score($r, $self->name => $self->score);
}
The method takes the key in the session in which the flag is stored.
The example above will add a new score to the scoring, but only once per mail since the session is persisted across different checks.
To add scoring to your plugin your plugin needs to consume the role Mail::MtPolicyd::Plugin::Role::Scoring.
This will add the method add_score( $r, $key, $value ) to your plugin class.
The $key is a name for the score you'll see when you display the detailed scores. eg. with the AddScoreHeader or ScoreAction plugin.
The $value is positive or negative number. In most cases you want to make this value configurable.
It is also recommended that you check that you add an score only once. See is_already_done() method above.
Here is an example:
with 'Mail::MtPolicyd::Plugin::Role::Scoring'; has 'score' => ( is => 'rw', isa => 'Maybe[Num]' );
And somewhere in your run() method:
if( defined $self->score && ! $r->is_already_done( $self->name.'-score' ) ) {
$self->add_score( $r, $self->name => $self->score );
}
To add user configurable parameters to your plugin the class must consume the Mail::MtPolicyd::Plugin::Role::UserConfig role.
with 'Mail::MtPolicyd::Plugin::Role::UserConfig' => {
'uc_attributes' => [ 'enabled', 'mode' ],
};
The regular attributes:
has 'enabled' => ( is => 'rw', isa => 'Str', default => 'on' ); has 'mode' => ( is => 'rw', isa => 'Str', default => 'reject' );
The UserConfig role adds the get_uc( $session, $param ) method to your class. To retrieve the user-configurable values for this attributes use:
my $session = $r->session; my $mode = $self->get_uc( $session, 'mode' ); my $enabled = $self->get_uc( $session, 'enabled' );
Per user configuration in mtpolicyd works like this:
A plugin like SqlUserConfig retrieves configuration values and stores them in the current session.
For example it may set the following key value:
hello_world_enabled = off
Our HelloWorld plugin may be configured like this:
<Plugin hello-world> module = "HelloWorld" enabled = on uc_enabled = "hello_world_enabled" </Plugin>
If the key "hello_world_enabled" is defined in the session it will use its value for $mode. If it is not defined it will fall back to value of the "enabled" attribute.
The Mail::MtPolicyd::Plugin::Result object has an extra constructor for returning a PREPEND action for setting a header:
Mail::MtPolicyd::Plugin::Result->new_header_once( $is_already_done, $header_name, $value );
It could be used like this:
return Mail::MtPolicyd::Plugin::Result->new_header_once(
$r->is_already_done( $self->name.'-tag' ),
$header_name, $value );
When mtpolicyd is called with the option --cron <task1,task2,...> it will execute all plugins that implement a cron() function.
The function is expected to take the following parameters:
$plugin->cron( $server, @tasks );
By default mtpolicyd ships with a crontab that will execute the tasks hourly,daily,weekly and monthly.
A plugin that implements a weekly task may look like this:
sub cron {
my $self = shift;
my $server = shift;
if( grep { $_ eq 'weekly' } @_ ) {
# do some weekly tasks
$server->log(3, 'i am a weekly task');
}
}
The $server object could be used for logging.
To see the output on the command line you may call mtpolicyd like this:
mtpolicyd -f -l 4 --cron=weekly
The mtpolicyd could be used to implement a smtp level accounting and quota system.
This guide explains how to setup accounting and quotas based on the sender ip on a monthly base and configurable quota limits.
The how to expects that mtpolicyd is already installed, working and assumes a MySQL database is used to hold accounting data and quota configuration.
The accounting and quota checks should be implemented in postfix smtpd_end_of_data_restrictions. If you're already using mtpolicyd for other check it may be necessary to setup a second virtual host for the accounting/quota configuration. Otherwise you can use the default port 12345 virual host.
First tell mtpolicyd to also listen on an addition port. In the global configuration add the new port to the port option:
port="127.0.0.1:12345,127.0.0.1:12346"
Then add a new virtual host at the end of the configuration file:
<VirtualHost 12346>
name="accounting"
# TODO: add plugins...
</VirtualHost>
Now add the Accounting plugin to your virtual host:
<Plugin AcctIP>
module = "Accounting"
fields = "client_address"
# time_pattern = "%Y-%m"
# table_prefix = "acct_"
</Plugin>
And the restart mtpolicyd to reload the configuration.
The plugin will create a table for every field listed in "fields". By default the table prefix is acct_ so the table name will be acct_client_address in our example. The plugin will create a row within this table for every client_address and expanded time_pattern:
mysql> select * from acct_client_address; +----+-------------------+---------+-------+------------+---------+-----------+ | id | key | time | count | count_rcpt | size | size_rcpt | +----+-------------------+---------+-------+------------+---------+-----------+ | 1 | 2604:8d00:0:1::3 | 2015-01 | 18 | 18 | 95559 | 95559 | | 2 | 2604:8d00:0:1::4 | 2015-01 | 21 | 21 | 99818 | 99818 | ... +----+-------------------+---------+-------+------------+---------+-----------+
To active the check add the policyd to your smtpd_end_of_data_restrictions in main.cf:
smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:12346
If you have multiple smtpd process configured in a smtp-filter setup make sure only one smtpd is doing accounting/quota checks. Deactivate the restrictions by adding the following option the the re-inject smtpd processes in master.cf:
-o smtpd_end_of_data_restrictions=
To limit the number of messages a client_address is allowed to send add the following Quota plugin to your virtual host configuration before the Accounting plugin:
<Plugin QuotaIP>
module = "Quota"
field = "client_address"
metric = "count"
threshold = 1000
action = "defer you exceeded your monthly limit, please insert coin"
# time_pattern = "%Y-%m"
# table_prefix = "acct_"
</Plugin>
Create the following table structure in your MySQL database:
CREATE TABLE `relay_policies` (
`id` int(11) NOT NULL auto_increment,
`desc` VARCHAR(64) NOT NULL,
`config` TEXT NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
INSERT INTO relay_policies VALUES(1, 'standard relay host', '{"quota_count":"10000"}');
INSERT INTO relay_policies VALUES(2, 'premium relay host', '{"quota_count":"100000"}');
CREATE TABLE `relay_hosts` (
`id` int(11) NOT NULL auto_increment,
`client_address` VARCHAR(64) NOT NULL,
`relay_policy` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `relay_policy` (`relay_policy`),
CONSTRAINT `relay_hosts_ibfk_1` FOREIGN KEY (`relay_policy`) REFERENCES `relay_policies` (`id`)
) ENGINE=InnoDB;
INSERT INTO relay_hosts VALUES(NULL, '2604:8d00:0:1::3', 1);
INSERT INTO relay_hosts VALUES(NULL, '2604:8d00:0:1::4', 2);
You can use the following SELECT statement to retrieve the configuration for a relay_host:
mysql> SELECT p.config FROM relay_policies p JOIN relay_hosts h ON (h.relay_policy = p.id) WHERE h.client_address = '2604:8d00:0:1::4';
+--------------------------+
| config |
+--------------------------+
| {"quota_count":"100000"} |
+--------------------------+
1 row in set (0.00 sec)
To load the (JSON) configuration into the mtpolicyd session variables use the SqlUserConfig plugin and this SQL statement:
<Plugin QuotaPolicyConfig>
module = "SqlUserConfig"
sql_query = "SELECT p.config FROM relay_policies p JOIN relay_hosts h ON (h.relay_policy = p.id) WHERE h.client_address=?"
field = "client_address"
</Plugin>
This plugin must be added before your Accounting and Quota plugins.
To use the quota_count value instead of the default threshold adjust your Quota plugin configuration:
<Plugin QuotaIP>
module = "Quota"
field = "client_address"
metric = "count"
threshold = 1000
uc_threshold = "quota_count"
action = "defer you exceeded your monthly limit, please insert coin"
# time_pattern = "%Y-%m"
# table_prefix = "acct_"
</Plugin>
If the session variable quota_count is defined it will be used as threshold instead of the value configured in mtpolicyd.conf.