Pantheon Community

Logging alternatives to dblog

We’re interested in switching from dblog to a different logging solution for performance reasons. It seems a common solution is to switch to using syslog, but according to https://pantheon.io/docs/logs/#can-i-log-to-the-system-logger-and-access-syslog we don’t have access to that on Pantheon.

So perhaps we could log to a file that we can download and import into a separate log analysis tool or send the log information directly to a separate log analysis tool somehow.

Anyway, we’d be interested in knowing how other Pantheon users have tackled this problem. What recommendations do you have? What approach did you use? What are the pros and cons of your solution?

1 Like

This is one of those “holy grail” things that I’ve been after for years. The solution that I think makes the most sense is to use a solution that pushes Watchdog messages out over http to an endpoint for an external log aggregator like Loggly, Splunk, or maybe your own locally-hosted ELK stack.

Amitai Burstein from Gizra wrote about this years ago, and Giza actually published a D7 contrib module (later ported to D8) to support this very use case:

https://www.gizra.com/content/logs-easy-way/

https://www.drupal.org/project/logs_http

The one catch for our team to use this is that our on-prem Splunk instance only accepts HTTP endpoint connections from whitelisted IP addresses, and that doesn’t work with Pantheon because of https://pantheon.io/docs/outgoing-ips/.

We’ve investigated this with Pantheon support and our own logging team multiple times over the past few years, with no easy solutions in sight. So my advice to you is to try to use a log aggregator that doesn’t require IP whitelisting. :slight_smile:

As a final note – you could probably easily modify the code from the logs_http module to do what you suggested and write the Watchdog logs to a file. But the catch there is you would be writing them to the files directory for your Drupal site (since that’s the only part of the container filesystem that is writable by your Drupal site), which doesn’t seem like a great idea.

One other possible solution that was suggested was to use the drush watchdog:show command to pull the logs on an external host and write them to your log aggregator from there. That might work okay for a couple of sites, but probably wouldn’t scale very well.

1 Like

Thank you for the helpful advice. If anyone is interested, I’ve been experimenting with several options and have found several potential solutions using the https://www.drupal.org/project/monolog or https://www.drupal.org/project/logs_http:

  1. The Monolog LogglyHandler can send watchdog logs to Loggly
  2. The Monolog SyslogUdpHandler can send watchdog logs to Papertrail
  3. The Monolog SyslogUdpHandler can send watchdog logs to Logit . io
  4. logs_http can send watchdog logs to Logz . io

For example, here is the Monolog configuration settings for your services.yml to redirect watchdog logs to Loggly, Papertrail, and Logit . io:

parameters:
  monolog.channel_handlers:
    # Log to the syslog by default.
    default:
      handlers: ['loggly', 'papertrail', 'logitio']
      formatter: 'loggly'

services:
  monolog.handler.loggly:
    class: Monolog\Handler\LogglyHandler
    arguments: ['{{your-unique-URL-token}}/tag/monolog']
  monolog.handler.papertrail:
    class: Monolog\Handler\SyslogUdpHandler
    arguments: ['logs4.papertrailapp.com', {{your-unique-port-number}}]
  monolog.handler.logitio:
    class: Monolog\Handler\SyslogUdpHandler
    arguments: ['{{your-unique-subdomain}}.logit.io', 21869]