newrelic - How to diagnose intermittent uwsgi errors? -


first let me briefly describe our set before ask question proper:

we have web application server (virtual machine) running django application. nginx @ front, uwsgi running under that, newrelic application wrapper followed django et al., database separate postgresql server located via smartstack (synapse/nerve)

the issue face (happened once 2 weeks ago, , twice in last 2 days), 1 or 2 of uwsgi worker processes trip , start producing "django.db.utils.interfaceerror: connection closed" on of requests.

slightly redacted stack trace (user , application_name):

traceback (most recent call last):   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/newrelic-2.8.0.7/newrelic/api/web_transaction.py", line 863, in __call__   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/newrelic-2.8.0.7/newrelic/api/function_trace.py", line 90, in literal_wrapper   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/newrelic-2.8.0.7/newrelic/api/web_transaction.py", line 752, in __call__   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 194, in __call__     signals.request_started.send(sender=self.__class__)   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 185, in send     response = receiver(signal=self, sender=sender, **named)   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/db/__init__.py", line 91, in close_old_connections     conn.abort()   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 374, in abort     self.rollback()   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 177, in rollback     self._rollback()   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 141, in _rollback     return self.connection.rollback()   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__     six.reraise(dj_exc_type, dj_exc_value, traceback)   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 141, in _rollback     return self.connection.rollback()   file "/home/user/webapps/application_name/local/lib/python2.7/site-packages/newrelic-2.8.0.7/newrelic/hooks/database_dbapi2.py", line 82, in rollback django.db.utils.interfaceerror: connection closed 

the stack trace never gets in our application, touches new relic , django. once worker trips, doesn't recover , further requests result in 500's in uwsgi logs , 502's on front side. assume database connectivity fine because sibling workers continue function normally, , restarting uwsgi instantly fixes problem.

my question how 1 go diagnosing issue pinpoint root cause, have checked know how check (memory, cpu, logs, database connectivity) , things don't understand trying read on (file descriptors mainly).

for updated new relic (stack trace older version) it's thing felt do.

i appreciate feedback, many google searches have proved fruitless.

replies may delayed, timezone says it's time sleep. also, apologies if should on serverfault or something, figured it's closer application debug issue server config issue.


Comments

Popular posts from this blog

java - Intellij Synchronizing output directories .. -

git - Initial Commit: "fatal: could not create leading directories of ..." -