I wouldn't expect it ever. Rapid development not rapid execution. However, I didn't expect the difference in speed to be so large, considering it is calling into the Java lucene.

This might be overkill, but would it be possible to setup a mini-Java Lucene process to handle the indexing and queries, and talk to your Python process over a socket?

Like I said, overkill, but it could be worth it.

I say this being in the same boat: coming from the JVM to Python and looking back at some of the things I'm missing.