Ticket #12 (closed defect: invalid)

Opened 5 years ago

Last modified 5 years ago

Endless loop when using threaded fcgi and an exception is thrown under certain circumstances

Reported by: trac Assigned to: asaddi
Priority: major Milestone: flup.server-1.0
Component: flup.server Version: 0.5
Keywords: Cc:

Description

I'm using flup.server.fcgi. When there is an uncatched exceptions (so that flup would usually translate it into nice html) there is an infinite block if the file containing the WSGIServer().run() call is not the main file (e.g. it is imported by another file). I'm pretty sure that's a threading related problem. The problem does not exist when using flup.server.fcgi_fork.

Test case: 

  #!/usr/bin/python
  # app.py
  def Application(environ, start_response):
        start_response('200 OK', [('Content-Type', 'text/plain')])
        raise RuntimeError('Expected exception')
        return ['Hello World!\n']
  
  from flup.server.fcgi_fork import WSGIServer
  WSGIServer(Application, bindAddress='/tmp/app.fcgi.sock').run()

#!/usr/bin/python
# breaks.py
import app

Starting the server using app.py (./app.py) everything works as expected. Starting it through ./breaks.py I never get an answer to the request (no browser bug, same behaviour with curl). I'm using lighttpd for testing, but I'm pretty sure it's irrelevant. Anyway, here is my config file: 

server.modules = (
        "mod_access",
        "mod_fastcgi"
)
server.document-root = "/tmp/webinterface/public"
server.bind = "127.0.0.1"
server.port = 8000

fastcgi.server = ( "/sceradon" => ((
        "socket" => "/tmp/app.fcgi.sock",
        "check-local" => "disable"
)))

Change History

02/20/07 14:50:34 changed by asaddi

  • status changed from new to assigned.

Initial observations:

  • Reproducible with ajp (ajp_fork does not exhibit problem)
  • Problem does not occur when debug=False. => Most likely cgitb issue.

02/20/07 15:16:12 changed by trac

It's also reproducable with scgi (and not with scgi_fork). I also discovered that debug=False does not cause the problem. The last python call flup does is really import cgitb, so the problem does not lay within flup itself. Maybe cgitb is not thread-safe (just an assumption, I don't know anything about that).

(follow-up: ↓ 4 ) 02/20/07 15:53:23 changed by asaddi

At this point, I think it's some sort of strange Python import/thread issue. The following code reproduces the problem:

foo.py 

import sys
import thread
import time

def foo():
    try:
        raise RuntimeError('Expected exception')
    except:
        import cgitb
        print cgitb.html(sys.exc_info()),

thread.start_new_thread(foo, ())

# Don't let main thread exit yet
time.sleep(5)

bar.py 

import foo

Executing foo.py works. Executing bar.py leads to an infinite loop/recursion. On my machine, Python eventually core dumps.

I don't know enough about the internals of Python (or the import process), but it probably has something to do with the fact that the import foo is still in progress when all the thread/exception/cgitb stuff happens.

As far as flup is concerned, perhaps it should be enough to document that calling WSGIServer.run from an imported module is a Bad Thing and not supported. ;)

(in reply to: ↑ 3 ) 02/21/07 02:59:33 changed by trac

At this point, I think it's some sort of strange Python import/thread issue. The following code reproduces the problem:

It's reproducable with that code here too.

Executing foo.py works. Executing bar.py leads to an infinite loop/recursion. On my machine, Python eventually core dumps.

I don't see a loop here (this should cause high cpu usage, shouldn't it?). Seems to be really some kind of blocking.

I don't know enough about the internals of Python (or the import process), but it probably has something to do with the fact that the import foo is still in progress when all the thread/exception/cgitb stuff happens. As far as flup is concerned, perhaps it should be enough to document that calling WSGIServer.run from an imported module is a Bad Thing and not supported. ;)

Ok, thanks for tracking this down! I'll work around it by moving the WSGIServer()... code into a function which I'll then call from outside. I'm also going to report that as a python bug.

02/21/07 04:16:11 changed by trac

I just discovered it's not even related to exceptions or the import cgitb inside foo(). This also triggers the problem: 

import sys
import thread
import time
import cgitb

def foo():
    print cgitb.html(sys.exc_info()),

thread.start_new_thread(foo, ())

# Don't let main thread exit yet
time.sleep(5)

02/21/07 09:59:34 changed by asaddi

  • status changed from assigned to closed.
  • resolution set to invalid.

I see you've opened a bug for Python about this. I'm going to close this ticket and make a note of the issue somewhere (probably the FAQ).

Thanks!