Open edX: Monitoring
25 Sep 2014

Setting up Sentry crash reporting

If you want to provide a reliable service, it’s extremely important to be aware of when things are going wrong on the website.

Sentry is a wonderful free system to catch Python exceptions.

For Django, we use raven to catch and report the errors back to Sentry.

  1. Set up a Sentry server

    This is left as an exercise to the reader. Commercial hosting is available if you don’t want to administer another service (or email me and I’ll do it for a fixed fee).

  2. Set up two new services (LMS and Studio) in Sentry and get their DSN strings

    The DSN strings are inside your Sentry instance under the Settings->Python->Django page.

  3. Modify /edx/app/edxapp/edx-platform/lms/envs/common.py

    Add to the end of the file:

    # Sentry integration
    INSTALLED_APPS += ('raven.contrib.django.raven_compat',)
    
    RAVEN_CONFIG = {
        'dsn': '<your-DSN-string>',
    }
    
  4. Install the raven module

    Ensure that you’re using the edxapp virtualenv. One easy way to do this is by typing which python. If you get back /usr/bin/python, you’re NOT in the virtualenv. If you get back /edx/app/edxapp/venvs/edxapp/bin/python, you are.

    Then,

    pip install raven
    

    If you’re using an edx-platform fork, you might want to add raven to edx-platform/requirements/edx/base.txt so it gets installed automatically (e.g. when you bring up Devstack).

  5. Repeat for Studio

    This time, modify /edx/app/edxapp/edx-platform/cms/envs/common.py and use the DSN string for Studio.

  6. Restart everything

    /edx/bin/supervisorctl restart edxapp:
    
  7. Test

    Hopefully, your Open edX instance doesn’t regularly give 500 errors. If you want to verify that things are working, we need to induce some.

    TODO: describe how to modify some edx-platform code to break.

Server/uptime monitoring

I strongly recommend setting up something for server monitoring; it will alert you when the server goes down, and it’ll warn you if you’re running out of memory.

I’m using Observium, primarily because it has a TurnKey Linux image and modern web interface.

Go to RamNode, set up their cheapest VM (right now, $5/quarter) and load the Observium image.

Cheap, reliable and will help you sleep at night.


comments powered by Disqus