This tracker is in read-only model To report bugs, please use the appropriate Github repository.

 

Issue169

Title repoze.profile: useless profile recorded when application is a generator
Priority bug Status resolved
Superseder Nosy List jkarp, tseaver
Assigned To tseaver Topics

Created on 2010-11-24.15:38:37 by jkarp, last changed 2010-11-29.13:18:58 by tseaver.

Messages
msg462 (view) Author: tseaver Date: 2010-11-29.13:18:58
Fix released to PyPI: http://pypi.python.org/pypi/repoze.profile/1.2
msg461 (view) Author: jkarp Date: 2010-11-29.12:49:26
Thanks! That works for me. I appreciate your work, repoze.profile was just what
I was looking for.
msg460 (view) Author: tseaver Date: 2010-11-24.21:56:02
I just checked in a change which moves the conditional list inside the
profiled code.  Please give it a try.

I made creation of the list condition in order to avoid injecting any
unnecessary overhead into the profiled code.  I'm probably being too fussy,
but there you go.
msg459 (view) Author: jkarp Date: 2010-11-24.18:02:47
I tried the changes you made; I don't they fix it. You made it iterate inside
__call__, but that is not enough, iteration needs to happen inside
self.profiler.runctx() or the profiler won't capture the execution.

The change I originally suggested seems to work. And it shouldn't break
anything, because PEP 333 guarantees a WSGI application will produce an
iterable, which list() will handle correctly, regardless of whether its a
generator under the covers or not.

Anyhow, paste.debug.profile takes a similar approach to what I'm suggesting. It
creates a nested function that does [].extend(app_iter), and profiles that function.
msg458 (view) Author: tseaver Date: 2010-11-24.16:36:42
Fix checked in on the trunk for the next release.
msg457 (view) Author: tseaver Date: 2010-11-24.16:14:26
Thanks for the report!  I agree that your suggested solution is probably
the correct one, with the exception that I would have it apply 'list()'
only for results which were generators:  hmm, how do I detect that?
msg456 (view) Author: jkarp Date: 2010-11-24.15:38:37
When repoze.profile is given an application to wrap that takes the form of a
generator, you end up with a useless profile that has only the entries
"<string>:1(<module>)" and "{method 'disable' of '_lsprof.Profiler' objects}".

This is because the way the profiling is being invoked, it only includes the
assignment of a generator object to app_iter. For the application code to be
executed, the generator must be iterated over, and that isn't happening until
after profiling has stopped.

One band-aid solution might be to change repoze.profile to profile
'list(self.app(environ, start_response))' instead of 'self.app(environ,
start_response)'.
History
Date User Action Args
2010-11-29 13:18:58 tseaver set status: chatting -> resolved
messages: + msg462
2010-11-29 12:49:27 jkarp set status: testing -> chatting
messages: + msg461
2010-11-24 21:56:03 tseaver set status: chatting -> testing
messages: + msg460
2010-11-24 18:02:47 jkarp set status: resolved -> chatting
messages: + msg459
2010-11-24 16:36:42 tseaver set status: chatting -> resolved
messages: + msg458
2010-11-24 16:14:27 tseaver set status: unread -> chatting
assignedto: tseaver
messages: + msg457
nosy: + tseaver
2010-11-24 15:38:37 jkarp create