newspaint

Documenting Problems That Were Difficult To Find The Answer To

Dovecot v2.3.2.1 and Solr v7.4.0

I found the instructions for getting Solr full-text searching (FTS) working with Dovecot rather difficult to follow.

I started off by downloading the latest build of Dovecot (v2.3.2.1 as of this article) because the Ubuntu build of dovecot (v2.2.2) does not have the solr plugin compiled,

After extracting the dovecot sources I ran the following commands to create a build:

$ sudo apt-get install clang-6.0 libmysqlclient-dev libexpat1-dev libssl-dev libsqlite3-dev

$ ./configure --prefix=/opt/dovecot-2.3.2.1 -with-solr --with-mysql --with-ssl=openssl --with-sqlite
$ nice make
$ sudo make install

For Solr I had to install a Java runtime:

$ sudo apt-get install openjdk-9-jre

After installing Solr and getting it running I ran the following command to create a core specifically for dovecot:

$ sudo -u solr -- bin/solr create_core -c dovecot

I had installed Solr in /opt/solr so the next thing I did was delete /opt/solr/server/solr/dovecot/conf/managed-schema and copied the dovecot-2.3.2.1/doc/solr-schema.xml file from the dovecot source to /opt/solr/server/solr/dovecot/conf/schema.xml and changing owner to solr:solr.

Then I had to force a reload of the Solr core, but there were problems.

The following substitutions were necessary in managed-schema (which is what the schema.xml file gets converted to on a reload of the core in Solr):

  • s/”text”/”text_general”/
  • s/”boolean”/”booleans”/
  • s/”plong”/”plongs”/

Then I had to comment out the following blocks in solrconfig.xml:

<!--
    <lst name="typeMapping">
      <str name="valueClass">java.util.Date</str>
      <str name="fieldType">pdates</str>
    </lst>
-->

<!--
    <lst name="typeMapping">
      <str name="valueClass">java.lang.Number</str>
      <str name="fieldType">pdoubles</str>
    </lst>
-->

I updated my /etc/dovecot/dovecot.conf:

mail_plugins = $mail_plugins fts fts_solr zlib

protocol imap {
  mail_plugins = $mail_plugins imap_zlib
}

plugin {
  fts = solr
  fts_solr = url=http://127.0.0.1:8983/solr/dovecot/ break-imap-search
  fts_autoindex=yes
  fts_autoindex_max_recent_msgs=5000
  fts_index_timeout=120
}

Finally I set up some cron jobs to ensure that Solr commits were conducted on a regular basis, an “optimize” was run every so often, and a re-index was done every so often:

*/5 * * * * ubuntu wget -O - 'http://127.0.0.1:8983/solr/dovecot/update?commit=true'
43 1,13 * * * ubuntu wget -O - 'http://127.0.0.1:8983/solr/dovecot/update?optimize=true'

37 * * * * root /opt/dovecot-2.3.2.1/bin/doveadm -c /etc/dovecot/dovecot.conf -v index -u "*" -n 20000 "*"

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: