There really isn't much to tune in MongoDB as a sysadmin. In fact, there are only a handful of parameters you can change on the server. The real tuning happens in the queries and indexes that need to be present. And for that purpose, the 'dex' tool is super valuable.

Dex is a command line tool that can parse the MongoDB logs, compare the queries that are logged to the running MongoDB instance and tell you which indexes should be added to help your performance.

Installing Dex for MongoDB

The dex tool is installed via pip, the Python package manager. For RHEL/CentOS, proceed as follows:

$ yum install python-pip
$ pip install dex

Once completed, the dex command line tool will be available to you.

Run dex to tune your MongoDB

Next, run the dex tool against your MongoDB logs and your running MongoDB instance.

$ cd /var/log/mongodb/
$ dex -f mongod.log mongodb://localhost

This will read all the logs in the ‘mongod.log’ file and check, for each query, if the necessary indexes exist. If they don’t, you can find output like this:

$ dex -f mongod.log mongodb://localhost
{
    'runStats': {
        'linesRecommended': 66,
        'linesProcessed': 190,
        'linesPassed': 522
    },
    'results': [
        {
            'queryMask': '{"$query":{"columnId":"<val>"}}',
            'namespace': 'collection.table',
            'recommendation': {
                'index': '{"columnId": 1}',
                'namespace': 'collection.table',
                'shellCommand': 'db["collection"].ensureIndex({"columnId": 1}, {"background": true})'
            },
            'details': {
                'count': 55,
                'totalTimeMillis': 213856,
                'avgTimeMillis': 3888
            }
        }
    ]
}

If there’s nothing to report, the output looks like this.

$ dex -f mongod.log mongodb://localhost
{
    'runStats': {
        'linesRecommended': 0,
        'linesProcessed': 10934,
        'linesPassed': 41372
    },
    'results': []
}

It checked 10.934 lines in the mongod.log file and could not report any improvements. Good news: your MongoDB seems to be running just fine!

Additional dex parameters: slow log queries, watch mode

Maybe you don’t want to check every query in the log. You can add the --slowms parameter to only check queries that exceed that slow log timeout.

$ dex -f mongod.log mongodb://localhost --slowms 200

The above will only check the queries that ran for more than 200ms.

Alternatively, you can use the --watch parameter to not parse the entire logfile on startup, but only process new entries. This can be useful if your logs already exceed a couple of GB’s in size and you don’t want to bother parsing those old values.

$ dex -f mongod.log mongodb://localhost --watch

This significantly speeds up the startup of dex.

Applying MongoDB performance recommendations

The dex tool offers you a really simple way to apply the indexes it recommends: the shell command is printed in the output! Here’s an example:

...
  'recommendation': {
    'index': '{"columnId": 1}',
    'namespace': 'collection.table',
    'shellCommand': 'db["collection"].ensureIndex({"columnId": 1}, {"background": true})'
  },
...

The shellCommand in the output is the exact command to create the index.

In order to apply it, connect to the running MongoDB instance using the mongo client-tool, the second argument is the collection you want to open.

$ mongo collection
> db["collection"].ensureIndex({"columnId": 1}, {"background": true})

The result is the index being added in a background fashion: this does not block the rows and the application can continue to operate. There are a couple of gotcha’s to background indexes in MongoDB, but to me they outweigh the downside of having both a read+write lock on the data.

If you’re working with MongoDB, make sure to use the dex tool to help troubleshoot performance issues.