Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian Development (http://www.linux-archive.org/debian-development/)
-   -   Threads and Gtk (http://www.linux-archive.org/debian-development/711455-threads-gtk.html)

Martin Sivak 10-11-2012 12:23 PM

Threads and Gtk
 
Hi,

we have got a lot of information while debugging #862801 so this email is just a quick summary about how we are supposed to use Gtk in threaded environment. As you might know - GLib itself is threadsafe but Gtk is not.

According to the Gtk team, the gdk_threads_enter/leave pair should not be used at all (and they have apparently discouraged usage of it since early releases of Gtk2). Moreover in the current Gdk docs (http://developer.gnome.org/gdk3/stable/gdk3-Threads.html#GDK-THREADS-ENTER:CAPS) those functions are now marked as deprecated.

The preferred way (and now the only way) is to use g_idle_add (GLib.idle_add) with a callback method to schedule GUI changes. The callback method will then get called by the Gtk main loop so no locking is needed (and GLib.idle_add performs none). But that is also the reason why everything Gtk related must be done from the mainloop thread either directly or via idle_add.

If we need to wait for the callback's response (Dialog?), we should probably use Queue.Queue's .put (in the callback) and .get (at the place where we need the response) as Queue properly blocks while waiting for data. I have written two decorators which will make any function decorated with it to be executed in the main loop regardless of the thread it was called in.

This all means that gdk_threaded context manager should not be used anywhere anymore and I am currently trying to update the code to follow this rule. You can check the decorators (gui/utils.py) and changes (still work in progress) in my personal threading branch at:

http://fedorapeople.org/cgit/msivak/public_git/anaconda.git/?h=threading

--
Martin Sivák
msivak@redhat.com
Red Hat Czech
Anaconda team / Brno, CZ


_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list

Chris Lumens 10-11-2012 03:07 PM

Threads and Gtk
 
> we have got a lot of information while debugging #862801 so this email
> is just a quick summary about how we are supposed to use Gtk in
> threaded environment. As you might know - GLib itself is threadsafe
> but Gtk is not.

There's very conflicting information on the internet regarding the
thread safety of GTK.

> According to the Gtk team, the gdk_threads_enter/leave pair should not
> be used at all (and they have apparently discouraged usage of it since
> early releases of Gtk2). Moreover in the current Gdk docs
> (http://developer.gnome.org/gdk3/stable/gdk3-Threads.html#GDK-THREADS-ENTER:CAPS)
> those functions are now marked as deprecated.

They're going to have to work pretty hard to change public perception,
then.

> The preferred way (and now the only way) is to use g_idle_add
> (GLib.idle_add) with a callback method to schedule GUI changes. The
> callback method will then get called by the Gtk main loop so no
> locking is needed (and GLib.idle_add performs none). But that is also
> the reason why everything Gtk related must be done from the mainloop
> thread either directly or via idle_add.

We'll just need to make sure that the added callbacks only get run once
to prevent those 100% CPU usage bugs from coming back.

> If we need to wait for the callback's response (Dialog?), we should
> probably use Queue.Queue's .put (in the callback) and .get (at the
> place where we need the response) as Queue properly blocks while
> waiting for data. I have written two decorators which will make any
> function decorated with it to be executed in the main loop regardless
> of the thread it was called in.

We will definitely need to wait for a callback response in several
places, mostly related to error handling around packaging and storage.
So yeah, we'll need a mechanism for that.

> This all means that gdk_threaded context manager should not be used
> anywhere anymore and I am currently trying to update the code to
> follow this rule. You can check the decorators (gui/utils.py) and
> changes (still work in progress) in my personal threading branch at:
>
> http://fedorapeople.org/cgit/msivak/public_git/anaconda.git/?h=threading

In commit 085cd2d3002efc7b0bbf80b98d1bf383cb48875b, last chunk, that was
added for a specific case. You should be able to figure out which one
via "git blame". Please verify you don't cause that issue to come back
by removing that line.

Thanks for taking a look into this.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list

Martin Sivak 10-11-2012 03:28 PM

Threads and Gtk
 
Hi,

> GLib itself is threadsafe, but Gtk is not.

This was actually always part of the documentation. Gtk was considered thread aware, but not thread safe.

> > those functions are now marked as deprecated.
>
> They're going to have to work pretty hard to change public
> perception,
> then.

Yep, I told them. They are telling everybody that we are using the Gtk in a wrong way, but the correct way is not really documented anywhere. And all the tutorials on the web have this either wrong or partially wrong... And there are almost none for Gtk3..

> We'll just need to make sure that the added callbacks only get run
> once
> to prevent those 100% CPU usage bugs from coming back.

The decorator returns False, so this should be taken care of. We just have to look at all plain idle_adds and check the return codes.

> We will definitely need to wait for a callback response in several
> places, mostly related to error handling around packaging and
> storage.
> So yeah, we'll need a mechanism for that.

I just tested the @gtk_thread_wait and it does exactly this. So we should be OK.

> In commit 085cd2d3002efc7b0bbf80b98d1bf383cb48875b, last chunk, that
> was
> added for a specific case. You should be able to figure out which
> one
> via "git blame". Please verify you don't cause that issue to come
> back
> by removing that line.

I did, it was needed because we had with gdk_threaded() in the code. As my branch is not using Gdk locks at all, it is meaningless now. I actually used @gtk_thread_wait for the two dialogs from UserInterface and exception.py uses idle_add so it should be OK as well.

I am testing those changes (including the infamous lockup when removing last keyboard layout :), but it takes long time as you can imagine. We also have to take a look at python-meh with Vrata to make sure everything plays well together.

Martin

----- Original Message -----
> > we have got a lot of information while debugging #862801 so this
> > email
> > is just a quick summary about how we are supposed to use Gtk in
> > threaded environment. As you might know - GLib itself is threadsafe
> > but Gtk is not.
>
> There's very conflicting information on the internet regarding the
> thread safety of GTK.
>
> > According to the Gtk team, the gdk_threads_enter/leave pair should
> > not
> > be used at all (and they have apparently discouraged usage of it
> > since
> > early releases of Gtk2). Moreover in the current Gdk docs
> > (http://developer.gnome.org/gdk3/stable/gdk3-Threads.html#GDK-THREADS-ENTER:CAPS)
> > those functions are now marked as deprecated.
>
> They're going to have to work pretty hard to change public
> perception,
> then.
>
> > The preferred way (and now the only way) is to use g_idle_add
> > (GLib.idle_add) with a callback method to schedule GUI changes. The
> > callback method will then get called by the Gtk main loop so no
> > locking is needed (and GLib.idle_add performs none). But that is
> > also
> > the reason why everything Gtk related must be done from the
> > mainloop
> > thread either directly or via idle_add.
>
> We'll just need to make sure that the added callbacks only get run
> once
> to prevent those 100% CPU usage bugs from coming back.
>
> > If we need to wait for the callback's response (Dialog?), we should
> > probably use Queue.Queue's .put (in the callback) and .get (at the
> > place where we need the response) as Queue properly blocks while
> > waiting for data. I have written two decorators which will make any
> > function decorated with it to be executed in the main loop
> > regardless
> > of the thread it was called in.
>
> We will definitely need to wait for a callback response in several
> places, mostly related to error handling around packaging and
> storage.
> So yeah, we'll need a mechanism for that.
>
> > This all means that gdk_threaded context manager should not be used
> > anywhere anymore and I am currently trying to update the code to
> > follow this rule. You can check the decorators (gui/utils.py) and
> > changes (still work in progress) in my personal threading branch
> > at:
> >
> > http://fedorapeople.org/cgit/msivak/public_git/anaconda.git/?h=threading
>
> In commit 085cd2d3002efc7b0bbf80b98d1bf383cb48875b, last chunk, that
> was
> added for a specific case. You should be able to figure out which
> one
> via "git blame". Please verify you don't cause that issue to come
> back
> by removing that line.
>
> Thanks for taking a look into this.
>
> - Chris
>
> _______________________________________________
> Anaconda-devel-list mailing list
> Anaconda-devel-list@redhat.com
> https://www.redhat.com/mailman/listinfo/anaconda-devel-list
>

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list

Vratislav Podzimek 10-11-2012 07:55 PM

Threads and Gtk
 
On Thu, 2012-10-11 at 11:28 -0400, Martin Sivak wrote:
> Hi,
>
> > GLib itself is threadsafe, but Gtk is not.
>
> This was actually always part of the documentation. Gtk was considered thread aware, but not thread safe.
>
> > > those functions are now marked as deprecated.
> >
> > They're going to have to work pretty hard to change public
> > perception,
> > then.
>
> Yep, I told them. They are telling everybody that we are using the Gtk in a wrong way, but the correct way is not really documented anywhere. And all the tutorials on the web have this either wrong or partially wrong... And there are almost none for Gtk3..
>
> > We'll just need to make sure that the added callbacks only get run
> > once
> > to prevent those 100% CPU usage bugs from coming back.
>
> The decorator returns False, so this should be taken care of. We just have to look at all plain idle_adds and check the return codes.
>
> > We will definitely need to wait for a callback response in several
> > places, mostly related to error handling around packaging and
> > storage.
> > So yeah, we'll need a mechanism for that.
>
> I just tested the @gtk_thread_wait and it does exactly this. So we should be OK.
>
> > In commit 085cd2d3002efc7b0bbf80b98d1bf383cb48875b, last chunk, that
> > was
> > added for a specific case. You should be able to figure out which
> > one
> > via "git blame". Please verify you don't cause that issue to come
> > back
> > by removing that line.
>
> I did, it was needed because we had with gdk_threaded() in the code. As my branch is not using Gdk locks at all, it is meaningless now. I actually used @gtk_thread_wait for the two dialogs from UserInterface and exception.py uses idle_add so it should be OK as well.
>
> I am testing those changes (including the infamous lockup when removing last keyboard layout :), but it takes long time as you can imagine. We also have to take a look at python-meh with Vrata to make sure everything plays well together.
Running 'kill -USR1 `cat /var/run/anaconda.pid`' should be good for
testing as it raises an exception from a separate thread.

--
Vratislav Podzimek

Anaconda Rider | Red Hat, Inc. | Brno - Czech Republic

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list

Chris Lumens 10-11-2012 09:30 PM

Threads and Gtk
 
> > We will definitely need to wait for a callback response in several
> > places, mostly related to error handling around packaging and
> > storage.
> > So yeah, we'll need a mechanism for that.
>
> I just tested the @gtk_thread_wait and it does exactly this. So we should be OK.

Okay, great. I've got a couple bugs I need to fix by adding additional
error handling dialogs, but I'm going to hold off on those until you get
this stuff in and I can see what format to follow.

> I am testing those changes (including the infamous lockup when
> removing last keyboard layout :), but it takes long time as you can
> imagine. We also have to take a look at python-meh with Vrata to make
> sure everything plays well together.

Heh, okay. Make sure to check the keyboard deadlock in both ways it
could come up. And yes, I understand testing this stuff takes an awful
long time.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list

Vratislav Podzimek 10-12-2012 01:43 PM

Threads and Gtk
 
On Thu, 2012-10-11 at 08:23 -0400, Martin Sivak wrote:
> Hi,
>
> we have got a lot of information while debugging #862801 so this email is just a quick summary about how we are supposed to use Gtk in threaded environment. As you might know - GLib itself is threadsafe but Gtk is not.
>
> According to the Gtk team, the gdk_threads_enter/leave pair should not be used at all (and they have apparently discouraged usage of it since early releases of Gtk2). Moreover in the current Gdk docs (http://developer.gnome.org/gdk3/stable/gdk3-Threads.html#GDK-THREADS-ENTER:CAPS) those functions are now marked as deprecated.
>
> The preferred way (and now the only way) is to use g_idle_add (GLib.idle_add) with a callback method to schedule GUI changes. The callback method will then get called by the Gtk main loop so no locking is needed (and GLib.idle_add performs none). But that is also the reason why everything Gtk related must be done from the mainloop thread either directly or via idle_add.
>
> If we need to wait for the callback's response (Dialog?), we should probably use Queue.Queue's .put (in the callback) and .get (at the place where we need the response) as Queue properly blocks while waiting for data. I have written two decorators which will make any function decorated with it to be executed in the main loop regardless of the thread it was called in.
>
> This all means that gdk_threaded context manager should not be used anywhere anymore and I am currently trying to update the code to follow this rule. You can check the decorators (gui/utils.py) and changes (still work in progress) in my personal threading branch at:
>
> http://fedorapeople.org/cgit/msivak/public_git/anaconda.git/?h=threading
I've tested the patches from this branch in various use cases that are
known to be problematic and hit no hang and no unexpected termination or
traceback. I believe these patches are worth sending to
anaconda-patches, reviewing, more testing and getting them to F18.

--
Vratislav Podzimek

Anaconda Rider | Red Hat, Inc. | Brno - Czech Republic

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list


All times are GMT. The time now is 12:48 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.