FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 05-16-2011, 11:45 AM
Ales Kozumplik
 
Default Reworking dispatch

Hello,

A patchset finishing the feature
https://fedoraproject.org/wiki/Anaconda/Features/Rework_dispatch
follows. Take a look at
https://github.com/akozumpl/anaconda/commits/dispatch for a more
pleasant browsing.


The main changes are:

:: stricter semantics of the dispatch interface: once a step is skipped
it is skipped forever and can not be unskipped. Similarly, a requested
step is guaranteed to run. In practice this means that if something
calls dispatch.request_step("filter") then "filter" will be run no
matter what (or the call throws an exception right at that moment)
:: making step into a proper class so it can have different attributes
which can come in handy (especially with the multithreaded UI I'm
working on). Step scheduling state (scheduled, skipped, etc.) is one of
those attributes now.
:: creating a specialized dict class IndexedDict to have a clean way of
accessing steps. It is just a mapping from step names to steps but
ordered, imposing a total ordering on all the steps added to dispatching
(the order being the order in which steps are run in).
::cleaning up the dispatcher and step initialization in the main
anaconda file
:: related to the previous: moving sshd initialization and the rescue
mode into steps (rescue mode never returns once started)
:: changing this and that so nothing gets broken and everything runs
like before
:: making sure that going Back in the installer makes the dispatcher
reverts step scheduling that happened in that step (this is the only
exception to "no re-scheduling what was skipped already" but the concept
makes sense in this case)

:: unit tests for many of those things

I made a small diagram showing the allowed step transitions here:
http://akozumpl.fedorapeople.org/scheduling.png
The dispatcher runs steps that are green and does not run steps that are
bright orange in the diagram. The point is: a scheduled step is run if
nothing tries to force it or disable it. Requested step is always run,
skipped step is never run. Unscheduled skip is not run (but can be
scheduled).


It's possible there's going to be few problems after I push this but I
tested it (text/kickstart/commandline) and I will fix the remaining
issues as they are found.


Ales

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-16-2011, 10:04 PM
"Brian C. Lane"
 
Default Reworking dispatch

On Mon, May 16, 2011 at 01:45:59PM +0200, Ales Kozumplik wrote:
> Hello,
>
> A patchset finishing the feature
> https://fedoraproject.org/wiki/Anaconda/Features/Rework_dispatch
> follows. Take a look at
> https://github.com/akozumpl/anaconda/commits/dispatch for a more
> pleasant browsing.
>

Overall these changes look pretty good. A few questions and comments:

Does sshd really belong as a step in dispatcher? It seems to me that it
isn't related to anaconda at all other than looking for the flag and
starting it. It really isn't part of the installer itself.

In the Dispatcher object I'd rather see the big list of steps as a
separate method. Think of the case where something wants to modify the
class and use a different list of steps. Having them all set in init
makes it harder to do something like that.

Related to that, why is _add_step prefixed with '_'? I could see cases
where either a subclass or something using the class would want to add
steps.

This could make livecd/image support in anaconda easier (I'm speculating
here, but my current plan is a combination of subclasses and plugin hook
points in anaconda to allow extending it easily).


--
Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT)
_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-17-2011, 01:02 PM
Chris Lumens
 
Default Reworking dispatch

> Does sshd really belong as a step in dispatcher? It seems to me that it
> isn't related to anaconda at all other than looking for the flag and
> starting it. It really isn't part of the installer itself.

Yeah, I would still like to see the approach where sshd is a systemd
service that anaconda just starts.

> In the Dispatcher object I'd rather see the big list of steps as a
> separate method. Think of the case where something wants to modify the
> class and use a different list of steps. Having them all set in init
> makes it harder to do something like that.

Agreed - good idea.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-17-2011, 03:34 PM
Ales Kozumplik
 
Default Reworking dispatch

On 05/17/2011 12:04 AM, Brian C. Lane wrote:

On Mon, May 16, 2011 at 01:45:59PM +0200, Ales Kozumplik wrote:

Hello,

A patchset finishing the feature
https://fedoraproject.org/wiki/Anaconda/Features/Rework_dispatch
follows. Take a look at
https://github.com/akozumpl/anaconda/commits/dispatch for a more
pleasant browsing.



Overall these changes look pretty good. A few questions and comments:

Does sshd really belong as a step in dispatcher? It seems to me that it
isn't related to anaconda at all other than looking for the flag and
starting it. It really isn't part of the installer itself.


Yes, I think: if it fits the dispatching model then why keep it as
something special in the main file?


Also, image that some installs (image install) for instance will never
want to schedule it while some might schedule it depending on a flag
(sshd, debug..). So the installclass should have a say.



In the Dispatcher object I'd rather see the big list of steps as a
separate method. Think of the case where something wants to modify the
class and use a different list of steps. Having them all set in init
makes it harder to do something like that.


Will do that.



Related to that, why is _add_step prefixed with '_'? I could see cases
where either a subclass or something using the class would want to add
steps.


I'll make it public.

Ales

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-17-2011, 03:36 PM
Ales Kozumplik
 
Default Reworking dispatch

On 05/17/2011 03:02 PM, Chris Lumens wrote:

Does sshd really belong as a step in dispatcher? It seems to me that it
isn't related to anaconda at all other than looking for the flag and
starting it. It really isn't part of the installer itself.


Yeah, I would still like to see the approach where sshd is a systemd
service that anaconda just starts.


It's on my todo but I am still pondering whether it's not best to wait
for the sshd package itself to come up with its own systemd
configuration: either for inspiration or even take the whole thing from
there.


Ales





In the Dispatcher object I'd rather see the big list of steps as a
separate method. Think of the case where something wants to modify the
class and use a different list of steps. Having them all set in init
makes it harder to do something like that.


Agreed - good idea.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list


_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-17-2011, 06:27 PM
Chris Lumens
 
Default Reworking dispatch

I see a couple things that concern me.

First, Dispatch.dispatch still exists and with it, all the "if dir == DISPATCH_"
crud in the UI steps. At this past FUDCon, I thought the plan we
discussed was that each step would keep its own state so we didn't have
to check for the direction we were moving through anaconda, and that
steps would add each other to a sort of queue instead of having the big
master list.

I can only speak for myself here, but I know I have always hated the
tests at the beginning of the various steps. They have led to a variety
of bugs over the years. By keeping this kind of stuff, I don't see how
this is a big improvement.

Second, there's still the concept of steps that take a function to call
and steps that look up the function to call in a stepToClass dict in the
UI. This really needs to get unified.

Third, I was really hoping that any control flow rework would also
provide us with a proper main loop. Peter's been talking about wanting
that for years, and we'll need one if we ever want anaconda to be a dbus
message producer in addition to consumer. And I've already got at least
one reason for wanting that.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-18-2011, 11:27 AM
Ales Kozumplik
 
Default Reworking dispatch

On 05/17/2011 08:27 PM, Chris Lumens wrote:

I see a couple things that concern me.


I will try to do something about those where possible, see below:



First, Dispatch.dispatch still exists and with it, all the "if dir == DISPATCH_"
crud in the UI steps. At this past FUDCon, I thought the plan we
discussed was that each step would keep its own state so we didn't have
to check for the direction we were moving through anaconda, and that


When user keeps hitting back in the gui, steps should revert what they
did previously so they can be reexecuted later(at least in theory, it's
not always necessary). Tell me if you think otherwise, but we need to
keep this ability. To do that right now, yes, they check the
dispatch.dir variable and react accordingly.


My patchset allows for an alternative where there would be one method
for going forward and one for going back, like 'storageinit' and
'storageinit_revert'. The steps that don't need it would not implement
the '_revert' method.



steps would add each other to a sort of queue instead of having the big
master list.


This just depends on how you think about what we have now. The 'one big
list' is like a registry of all steps existing, a superset of all the
steps running. An additional thing it provides is a total ordering on
all the steps: given two steps you can decide which one should run
sooner. Then steps can dynamically schedule what other steps will or
will not run, just like adding stuff to a queue.


I don't think we need different structure at the moment. How would we
for instance benefit from a queue? Simplifying the main loop, perhaps.
But since it is not possible to determine all the steps that need to be
taken ahead of the dispatch start, there would need to be a mechanism
that can remove stuff from the queue without running them (like user
chooses upgrade and we are no longer going to do bootloader). There
still is a problem of ordering, suppose there are steps A, B and C
where each following step depends on some information that can be
affected by the previous step. If A adds C to the queue before B the two
steps would just run in reverse order and the information from B would
never be seen by C.



I can only speak for myself here, but I know I have always hated the
tests at the beginning of the various steps. They have led to a variety
of bugs over the years. By keeping this kind of stuff, I don't see how
this is a big improvement.


Improvement is in splitting the concepts of Dispatcher and Step: Step is
a class now and can have attributes, for instance a separate attribute
to tell it what to do on revert. This way we can get rid of the checks.




Second, there's still the concept of steps that take a function to call
and steps that look up the function to call in a stepToClass dict in the
UI. This really needs to get unified.


I am totally open to suggestions here. The problem is that executing the
step 'bootloader' in GUI does something completely different then in
text. IOW, what a step with a screen does is given by what interface it
is running under. So this information should be kept in the interface
anyway (or we need interface-agnostic steps, i.e. no calls to gtk/snack
from there: a whole different and bigger job).



Third, I was really hoping that any control flow rework would also
provide us with a proper main loop. Peter's been talking about wanting
that for years, and we'll need one if we ever want anaconda to be a dbus
message producer in addition to consumer. And I've already got at least
one reason for wanting that.


Yeah, this is a pain in the ass. A 'proper' main loop what do you mean
by that?


The Dispatch.dispatch should act as the main loop and I tried to make it
that way as possible, yet it still has to exit and be reentered. This is
because we have another loop, gtk.main and one of the loops needs to be
running the other loop (one of them needs to be on the stack above the
other one): in our case it must be gtk.main, because we don't want to
quit it and start it again over and over. And we can't just keep
gtk.main sitting above either, the gui would never do anything.


The solution to this is having gtk.main run in a separate thread. But I
had to start on this feature first and having the steps with attributes
is the first, huh, step.


So all in all, this set enables a lot of things we couldn't do before
and breaks (hopefully) none. Plus I remember from fudcon the main
complain was it was never possible to tell if a step is going to be run
or not and I think I helped that by making the scheduling interface more
strict.


Ales

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-18-2011, 07:18 PM
Chris Lumens
 
Default Reworking dispatch

> When user keeps hitting back in the gui, steps should revert what
> they did previously so they can be reexecuted later(at least in
> theory, it's not always necessary). Tell me if you think otherwise,
> but we need to keep this ability. To do that right now, yes, they
> check the dispatch.dir variable and react accordingly.

Our relatively short term plan is to come up with a whole new UI. Given
that, I think we can afford to throw out some of our old behavior where
it makes sense to. Perhaps here is one of those places.

I think first off, we should consider breaking our steps down into more
discrete chunks. I am positive we've got some non-interactive steps
doing way more than they really ought to.

Then, I think we can get rid of the concept of steps resetting. I think
preserving the data makes more sense - especially in a model where we
have everyone enter their selections beforehand, and then press the big
"GO!" button. I think it's the less surprising behavior. If you think
about it, there are really only three interactive steps that are
sufficiently complex to require resetting: filter, partitioning, and
packaging. Partitioning even has a reset button.

I haven't thought this completely out, but I think a combination of the
non-resetting approach, making finer grained steps, and perhaps thinking
of how some steps should be one-shot and some should execute every time
allows us to kill a lot of this nonsense.

> My patchset allows for an alternative where there would be one
> method for going forward and one for going back, like 'storageinit'
> and 'storageinit_revert'. The steps that don't need it would not
> implement the '_revert' method.

Just thinking out loud here - if a step were a class, perhaps we could
have some sort of class decorator where actions were executed on
entering and leaving.

> >Second, there's still the concept of steps that take a function to call
> >and steps that look up the function to call in a stepToClass dict in the
> >UI. This really needs to get unified.
>
> I am totally open to suggestions here. The problem is that executing
> the step 'bootloader' in GUI does something completely different
> then in text. IOW, what a step with a screen does is given by what
> interface it is running under. So this information should be kept in
> the interface anyway (or we need interface-agnostic steps, i.e. no
> calls to gtk/snack from there: a whole different and bigger job).

Having interactive and non-interactive steps comingled just bugs me. We
may have to put this off until the UI rework ramps up, though. Assuming
we go with the plan of getting everything from the user, then applying
it all, that would separate out interactive steps into one list and
non-interactive into another. They wouldn't be interspersed like they
are now.

So, I guess I'm fine with what's going on here for the time being.

> >Third, I was really hoping that any control flow rework would also
> >provide us with a proper main loop. Peter's been talking about wanting
> >that for years, and we'll need one if we ever want anaconda to be a dbus
> >message producer in addition to consumer. And I've already got at least
> >one reason for wanting that.
>
> Yeah, this is a pain in the ass. A 'proper' main loop what do you
> mean by that?
>
> The Dispatch.dispatch should act as the main loop and I tried to
> make it that way as possible, yet it still has to exit and be
> reentered. This is because we have another loop, gtk.main and one of
> the loops needs to be running the other loop (one of them needs to
> be on the stack above the other one): in our case it must be
> gtk.main, because we don't want to quit it and start it again over
> and over. And we can't just keep gtk.main sitting above either, the
> gui would never do anything.

In a main loop approach, anaconda would be event driven. We'd set a
bunch of stuff up, call g_main_loop_run (for instance - could be
something else), and then all the step moving would be driven by events
like button presses.

> The solution to this is having gtk.main run in a separate thread.
> But I had to start on this feature first and having the steps with
> attributes is the first, huh, step.
>
> So all in all, this set enables a lot of things we couldn't do
> before and breaks (hopefully) none. Plus I remember from fudcon the
> main complain was it was never possible to tell if a step is going
> to be run
> or not and I think I helped that by making the scheduling interface
> more strict.

Hm, I am curious why you are going with the threading approach vs.
completely separate processes and a communications protocol between
them. Seems to me that'd be an even more strict separation between the
logic and presentation layers.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-19-2011, 08:39 AM
Ales Kozumplik
 
Default Reworking dispatch

On 05/18/2011 09:18 PM, Chris Lumens wrote:

I haven't thought this completely out, but I think a combination of the
non-resetting approach, making finer grained steps, and perhaps thinking
of how some steps should be one-shot and some should execute every time
allows us to kill a lot of this nonsense.


I agree, though we are going to have to twist and turn things to fit the
select-and-go model. I'll start looking into that.





My patchset allows for an alternative where there would be one
method for going forward and one for going back, like 'storageinit'
and 'storageinit_revert'. The steps that don't need it would not
implement the '_revert' method.


Just thinking out loud here - if a step were a class, perhaps we could
have some sort of class decorator where actions were executed on
entering and leaving.


Step is a class already, so we have those options.


Third, I was really hoping that any control flow rework would also
provide us with a proper main loop. Peter's been talking about wanting
that for years, and we'll need one if we ever want anaconda to be a dbus
message producer in addition to consumer. And I've already got at least
one reason for wanting that.


Yeah, this is a pain in the ass. A 'proper' main loop what do you
mean by that?

The Dispatch.dispatch should act as the main loop and I tried to
make it that way as possible, yet it still has to exit and be
reentered. This is because we have another loop, gtk.main and one of
the loops needs to be running the other loop (one of them needs to
be on the stack above the other one): in our case it must be
gtk.main, because we don't want to quit it and start it again over
and over. And we can't just keep gtk.main sitting above either, the
gui would never do anything.


In a main loop approach, anaconda would be event driven. We'd set a
bunch of stuff up, call g_main_loop_run (for instance - could be
something else), and then all the step moving would be driven by events
like button presses.


I can not really imagine this yet, ditching dispatch() and have a
generic main loop transition between steps, it's something to look into.



Hm, I am curious why you are going with the threading approach vs.
completely separate processes and a communications protocol between
them. Seems to me that'd be an even more strict separation between the
logic and presentation layers.


This is exactly where I am headed! It's only an implementation thing
that it's easier to start with threading+Queue, but once the separation
is good enough it's straightforward to switch over to processes+IPC.


But that's the same problem as with the other high level goals: the
large body of code does things in a wrong way utilizing some undesirable
mechanism A. We can not simply replace A for B without rewriting all the
code to use B too. This is true for steps granularity, controller/view
separation, the select-and-go model etc. Maybe we should start allowing
both A and B to exist at the same time, deprecating the former and
encouraging everyone to use the latter. In this way those big changes
can happen gradually and everyone can benefit from the new interfaces
sooner. I am thinking about using this for the threading feature.


Ales

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 05-19-2011, 02:02 PM
Peter Jones
 
Default Reworking dispatch

On 05/18/2011 03:18 PM, Chris Lumens wrote:
>> My patchset allows for an alternative where there would be one
>> method for going forward and one for going back, like 'storageinit'
>> and 'storageinit_revert'. The steps that don't need it would not
>> implement the '_revert' method.
>
> Just thinking out loud here - if a step were a class, perhaps we could
> have some sort of class decorator where actions were executed on
> entering and leaving.

I think this is on the right path, but I might take it a step further - we
could, with relative ease, have a class hierarchy like:

Step
One-shot Step
Resetting Step (if we decide we need to)

>>> Second, there's still the concept of steps that take a function to call
>>> and steps that look up the function to call in a stepToClass dict in the
>>> UI. This really needs to get unified.
>>
>> I am totally open to suggestions here. The problem is that executing
>> the step 'bootloader' in GUI does something completely different
>> then in text. IOW, what a step with a screen does is given by what
>> interface it is running under. So this information should be kept in
>> the interface anyway (or we need interface-agnostic steps, i.e. no
>> calls to gtk/snack from there: a whole different and bigger job).
>
> Having interactive and non-interactive steps comingled just bugs me. We
> may have to put this off until the UI rework ramps up, though. Assuming
> we go with the plan of getting everything from the user, then applying
> it all, that would separate out interactive steps into one list and
> non-interactive into another. They wouldn't be interspersed like they
> are now.

I do really think we need to consider making the UI into an application
that writes a kickstart file. Then when the user mashes the big "GO"
button, we're basically only executing it.

> Hm, I am curious why you are going with the threading approach vs.
> completely separate processes and a communications protocol between
> them. Seems to me that'd be an even more strict separation between the
> logic and presentation layers.

This would work well with what I said above.

--
Peter

In computing, turning the obvious into the useful is a living
definition of the word "frustration"
-- Alan Perlis

01234567890123456789012345678901234567890123456789 012345678901234567890123456789

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 

Thread Tools




All times are GMT. The time now is 10:03 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org