It would be great if Pantheon offered a feature to pipe access, db, and error logs to log aggregation systems like Logstash, Graylog, Sumo Logic, etc. Bonus points if we can also pipe Drupal watchdog messages via this process as well.
We aggregate our log files on an enterprise level. Knowing what is happening or has happened in our applications gives us a more holistic view of any events that happen across our organization.
Currently we are piping the logs to our system using a periodic CircleCI job to grab the logs with the suggested script. However, this method is cumbersome and error prone:
- No guarantee we get all the logs since the containers can come and go on a whim. It’s possible this process will miss some log entries
- It can be difficult to not duplicate log entries on subsequent log downloads
- It requires us to maintain a separate CircleCI job
- A pull method that happens periodically can cause a delay in getting the data
- It’s probably not the greatest for Pantheon’s resources to expect clients to rsync/ssh potentially large files constantly.