Problem: do threads always run for at least TIME_QUANTUM_USEC before termination, on Linux?

[Sav Erio]

I have a GUI project (Tk), which makes use of threads for performing
background filesystem searches based on a filename pattern.

My system is a 4 cores, on recent Linux; the interpreter is MRI 2.3.

At any given time, there should be at most one search background
thread, so the requests for new searches cause termination of the one
currently executing.

The search code is properly structured: it's a simple loop through
directories, that exits either when the directories are exhausted, or
when a termination flag is passed (by the project scheduler) to the
thread.

The problem I've observed is that search threads take at least 100ms
before terminating, so that when searches come in in quick succession,
they start to pile up.

I've had a look at the source code, and I have the suspicion that this
is related to `TIME_QUANTUM_USEC`. In the 2.1 changelog, `When running
on uniprocessor systems, every th.kill needs TIME_QUANTUM_USEC time`
is mentioned (I'm not sure if "processor" refers to CPU or core).

I've tried a very simplified version of the project, and on
JRuby/Rubinius the time slice is significantly smaller (at least one
order of magnitude smaller).

Is this behavior expected, or I'm unintentionally causing it? In the
latter case, which conditions could potentially cause it?

Thanks,
Saverio

Unsubscribe: <mailto:ruby-talk-***@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

[Eric Wong]

Post by Sav Erio
I have a GUI project (Tk), which makes use of threads for performing
background filesystem searches based on a filename pattern.
My system is a 4 cores, on recent Linux; the interpreter is MRI 2.3.
At any given time, there should be at most one search background
thread, so the requests for new searches cause termination of the one
currently executing.
The search code is properly structured: it's a simple loop through
directories, that exits either when the directories are exhausted, or
when a termination flag is passed (by the project scheduler) to the
thread.
The problem I've observed is that search threads take at least 100ms
before terminating, so that when searches come in in quick succession,
they start to pile up.

Is it 100% necessary to join the threads you're killing? And do
you need to kill threads, instead of reusing them with Queue or
similar?

Post by Sav Erio
I've had a look at the source code, and I have the suspicion that this
is related to `TIME_QUANTUM_USEC`. In the 2.1 changelog, `When running
on uniprocessor systems, every th.kill needs TIME_QUANTUM_USEC time`
is mentioned (I'm not sure if "processor" refers to CPU or core).

Right, this seems to be a problem if you have 3 or more threads
running. If you have only 2 threads (one doing work, the other
doing th.kill+th.join), it's OK.

In other words, I think your other threads are hogging up time
from the background thread you want to kill, as well as the
thread joining the to-be-killed background thread.

You can try lowering thread priorities to shorten timeslices,
but I'm not sure how effective it'd be...

Post by Sav Erio
I've tried a very simplified version of the project, and on
JRuby/Rubinius the time slice is significantly smaller (at least one
order of magnitude smaller).

They don't have GVL; totally different threading implementations.

Post by Sav Erio
Is this behavior expected, or I'm unintentionally causing it? In the
latter case, which conditions could potentially cause it?

Expected, I think so :< Ideal? No...

GVL is round-robin, and we cannot control runqueue order easily
because it's in the kernel. Killed threads should probably jump
to the head of the queue...

Unsubscribe: <mailto:ruby-talk-***@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

[Sarkar Chinmoy]

Please DO NOT send any more e-mails. I want to UNSUBSCRIBE

Post by Sav Erio
I have a GUI project (Tk), which makes use of threads for performing
background filesystem searches based on a filename pattern.
My system is a 4 cores, on recent Linux; the interpreter is MRI 2.3.
At any given time, there should be at most one search background
thread, so the requests for new searches cause termination of the one
currently executing.
The search code is properly structured: it's a simple loop through
directories, that exits either when the directories are exhausted, or
when a termination flag is passed (by the project scheduler) to the
thread.
The problem I've observed is that search threads take at least 100ms
before terminating, so that when searches come in in quick succession,
they start to pile up.

Is it 100% necessary to join the threads you're killing?  And do
you need to kill threads, instead of reusing them with Queue or
similar?

Post by Sav Erio
I've had a look at the source code, and I have the suspicion that this
is related to `TIME_QUANTUM_USEC`. In the 2.1 changelog, `When running
on uniprocessor systems, every th.kill needs TIME_QUANTUM_USEC time`
is mentioned (I'm not sure if "processor" refers to CPU or core).

Right, this seems to be a problem if you have 3 or more threads
running.  If you have only 2 threads (one doing work, the other
doing th.kill+th.join), it's OK.

In other words, I think your other threads are hogging up time
from the background thread you want to kill, as well as the
thread joining the to-be-killed background thread.

You can try lowering thread priorities to shorten timeslices,
but I'm not sure how effective it'd be...

Post by Sav Erio
I've tried a very simplified version of the project, and on
JRuby/Rubinius the time slice is significantly smaller (at least one
order of magnitude smaller).

They don't have GVL; totally different threading implementations.

Post by Sav Erio
Is this behavior expected, or I'm unintentionally causing it? In the
latter case, which conditions could potentially cause it?

Expected, I think so :<  Ideal?  No...

GVL is round-robin, and we cannot control runqueue order easily
because it's in the kernel.  Killed threads should probably jump
to the head of the queue...

Unsubscribe: <mailto:ruby-talk-***@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

[Eric Wong]

Post by Sarkar Chinmoy
Please DO NOT send any more e-mails. I want to UNSUBSCRIBE

You need to unsubscribe yourself, nobody can do it for you. See:

Unsubscribe: <mailto:ruby-talk-***@ruby-lang.org?subject=unsubscribe>

Unsubscribe: <mailto:ruby-talk-***@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

[Saverio M.]

Unsubscribe: <mailto:ruby-talk-***@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>