FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Infrastructure

 
 
LinkBack Thread Tools
 
Old 04-19-2012, 07:06 PM
Kevin Fenzi
 
Default Summary/Minutes from today's Fedora infrastructure meeting (2012-04-19) (new time)

============================================
#fedora-meeting: Infrastructure (2012-04-19)
============================================


Meeting started by nirik at 18:00:00 UTC. The full logs are available at
http://meetbot.fedoraproject.org/fedora-meeting/2012-04-19/infrastructure.2012-04-19-18.00.log.html
.



Meeting summary
---------------
* Robot Roll Call (nirik, 18:00:01)

* New folks introductions and Apprentice tasks. (nirik, 18:02:13)

* two factor auth status (nirik, 18:04:02)

* Staging re-work status (nirik, 18:04:29)

* Applications status / discussion (nirik, 18:06:38)
* reviews needed to move fedmsg ahead. (nirik, 18:10:09)
* ACTION: threebean to post list (nirik, 18:10:20)
* LINK: http://gitorious.org/sticky-notes (nirik, 18:12:46)
* looking at sticky-notes for a pastebin server (nirik, 18:14:03)

* Meeting time (nirik, 18:17:44)

* Upcoming Tasks/Items (nirik, 18:19:34)
* 2012-05-01 to 2012-05-15 - F17 Final Freeze. (nirik, 18:19:43)
* 2012-05-01 - nag fi-apprentices. (nirik, 18:19:43)
* 2011-05-03 - gitweb-cache removal day. (nirik, 18:19:43)
* 2012-05-09 - Check if puppet works on f17 yet. (nirik, 18:19:43)
* 2012-05-10 - drop inactive fi-apprentices (nirik, 18:19:43)
* 2012-05-15 - F17 release (nirik, 18:19:44)

* Private Cloud (nirik, 18:22:06)
* waiting for hardware, will move forward once thats done. (nirik,
18:27:23)

* Open floor (nirik, 18:29:28)
* LINK: http://fedorahosted.org/koji and
http://koji.fedoraproject.org/koji/ (nirik, 18:35:40)
* need a solution to find builds that need lots of memory. (nirik,
18:41:12)
* IDEA: ask koji developers upstream for a solution (nirik, 18:41:22)
* ACTION: dgilmore to talk to mikes about koji options. (nirik,
18:48:22)
* LINK: http://fedoraproject.org/wiki/Fails_to_build_from_source
(nirik, 18:53:06)
* will setup initial ansible ssh keys for testing with. (nirik,
19:01:06)
* LINK: http://images.srv1.elrod.me/stickynotes-fedora.png
(CodeBlock, 19:02:33)
* provide feedback on sticky-notes theme:
http://images.srv1.elrod.me/stickynotes-fedora.png (nirik,
19:04:00)

Meeting ended at 19:06:02 UTC.




Action Items
------------
* threebean to post list
* dgilmore to talk to mikes about koji options.




Action Items, by person
-----------------------
* dgilmore
* dgilmore to talk to mikes about koji options.
* threebean
* threebean to post list
* **UNASSIGNED**
* (none)




People Present (lines said)
---------------------------
* nirik (145)
* skvidal (75)
* dgilmore (35)
* herlo (32)
* codemaniac (11)
* misc (9)
* ctria (7)
* CodeBlock (7)
* abadger1999 (5)
* zodbot (4)
* threebean (4)
* ianweller (3)
* pingou (3)
* mmello (3)
* lmacken (2)
* athmane (1)
* smooge (0)
* ricky (0)
* mdomsch (0)
--
18:00:00 <nirik> #startmeeting Infrastructure (2012-04-19)
18:00:00 <zodbot> Meeting started Thu Apr 19 18:00:00 2012 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:01 <nirik> #meetingname infrastructure
18:00:01 <zodbot> The meeting name has been set to 'infrastructure'
18:00:01 <nirik> #topic Robot Roll Call
18:00:01 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean
18:00:01 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean
18:00:11 <nirik> who all is around for a infrastructure meeting?
18:00:15 * lmacken
18:00:19 * skvidal is here
18:00:23 * threebean is here
18:00:27 * athmane is here
18:00:31 * ianweller is last-minuting a physics lab report but will pay attention
18:01:18 * pingou
18:01:24 <dgilmore> buenos
18:02:06 <nirik> ok, lets go ahead and dive in.
18:02:13 <nirik> #topic New folks introductions and Apprentice tasks.
18:02:14 <nirik> If any new folks want to give a quick one line bio or any apprentices
18:02:14 <nirik> would like to ask general questions, they can do so here.
18:02:32 <nirik> any new folks? or questions on easyfix tickets or general getting started questions?
18:03:54 <nirik> ok, moving on then...
18:04:02 <nirik> #topic two factor auth status
18:04:03 <skvidal> no changes on 2fa - pto the beginning of this week.
18:04:18 <nirik> ok. We can ping it next week...
18:04:29 <nirik> #topic Staging re-work status
18:04:34 <nirik> I've worked on this some.
18:04:40 <nirik> I've commited docs on the new setup.
18:04:56 <nirik> please read http://infrastructure.fedoraproject.org/infra/docs/staging.txt and let me know if anything needs fixing.
18:05:18 <nirik> I'm going to try and get the bapp01->02 move done, and easyfix moved to production, then we can do the staging branch killing.
18:05:53 <nirik> any questions on that ?
18:06:35 <nirik> ok, I'll schedule the actual staging killing after those other 2 things are done.
18:06:38 <nirik> #topic Applications status / discussion
18:06:44 <nirik> any news on applications this week?
18:07:08 <nirik> abadger1999 / threebean / lmacken / pingou / CodeBlock / anyone else who works on apps.
18:07:33 <abadger1999> New python-fedora this week that should resolve some issues clients were seeing.
18:07:45 <nirik> yeah, thanks for that fix.
18:07:54 <lmacken> abadger1999++
18:08:12 * nirik still doesn't fully understand it, but thats ok
18:08:16 <abadger1999> I'm not aware of any of our scripts that were running into this but it won't hurt them to update on the infra boxes.
18:08:58 <threebean> fedmsg progress - got a mediawiki plugin done and tested locally. waiting still on lots of package reviews.
18:09:13 <ianweller> he's alive!
18:09:44 <nirik> threebean: could you perhaps drop a list to the infra list? or a wiki page link or something? then we can try and divide them up...
18:09:54 <threebean> sure thing
18:10:08 * threebean writes that now
18:10:09 <nirik> #info reviews needed to move fedmsg ahead.
18:10:20 <nirik> #action threebean to post list
18:11:08 <nirik> ok, any other app news?
18:11:22 <dgilmore> where did we ever get with re-writing fpaste?
18:12:15 <nirik> dgilmore: oh yeah, have some news on that...
18:12:32 <nirik> we are looking at possibly using sticky-notes instead for the server side.
18:12:46 <nirik> http://gitorious.org/sticky-notes
18:12:47 <abadger1999> mmello, I believe was interested in helping out with reviews, too.
18:13:01 <nirik> it's under review, pending some fixes from upstream.
18:13:13 <nirik> athmane has found some issues and submitted patches to it.
18:13:24 <mmello> abadger1999: yes..I'm
18:13:32 <dgilmore> nirik: cool
18:13:44 <dgilmore> mmello: get to work
18:13:50 <mmello> abadger1999: threebean is helping to understand the process and I'm currently review my first package
18:13:53 <mmello> dgilmore:
18:14:03 <nirik> #info looking at sticky-notes for a pastebin server
18:14:11 <dgilmore> nirik: would the current fpaste client work with the new server? or need rewritten?
18:14:18 <nirik> there's a command line 'pastebinit' already in fedora.
18:14:25 <dgilmore> cool
18:14:35 <nirik> so, we could look at making it obsolete fpaste and perhaps provide a compat link.
18:14:50 <nirik> it misses a few things like the --sysinfo tho.
18:15:18 <nirik> anyhow, it looks promising.
18:15:34 <codemaniac> hello nirik and everyone
18:15:44 <nirik> morning codemaniac. welcome.
18:16:01 <codemaniac> good morning to you too nirik
18:16:03 <dgilmore> nirik: figured id ask
18:16:09 <pingou> but php
18:16:15 <ctria> hi all. did i get lost timezones again?
18:16:17 <nirik> pingou: yeah, its php.
18:16:27 <nirik> ctria: meeting just started 15m ago.
18:16:35 <nirik> any further news on applications?
18:16:35 <ctria>
18:16:49 <codemaniac> sorry for joining late team
18:16:56 <nirik> no problem at all.
18:17:24 <codemaniac> CodeBlock: you around ?
18:17:39 <nirik> ok, will move along then...
18:17:44 <nirik> #topic Meeting time
18:17:54 <nirik> is everyone ok with this meeting time? 18UTC?
18:17:59 <skvidal> +1
18:18:00 <pingou> +1
18:18:03 <ianweller> +1
18:18:07 <abadger1999> +1
18:18:25 <nirik> ok then... seems popular.
18:18:47 <codemaniac> yea niriki , i am very much comfortable with this time , its +5:30 UTC here
18:19:07 <codemaniac> +1024
18:19:13 <nirik> cool.
18:19:34 <nirik> #topic Upcoming Tasks/Items
18:19:43 <nirik> #info 2012-05-01 to 2012-05-15 - F17 Final Freeze.
18:19:43 <nirik> #info 2012-05-01 - nag fi-apprentices.
18:19:43 <nirik> #info 2011-05-03 - gitweb-cache removal day.
18:19:43 <nirik> #info 2012-05-09 - Check if puppet works on f17 yet.
18:19:43 <nirik> #info 2012-05-10 - drop inactive fi-apprentices
18:19:44 <nirik> #info 2012-05-15 - F17 release
18:19:53 <nirik> so final freeze is coming up.
18:20:25 <nirik> One thing I will probibly schedule soon is a migration of our hosted machine... hosted03 -> hosted01/02. Should be a pretty easy migration.
18:20:37 <nirik> anyone have other items they are working on they would like to schedule or shout out?
18:21:46 <nirik> ok, I'll take that as a no.
18:21:55 <dgilmore> nothing here
18:22:06 <nirik> #topic Private Cloud
18:22:14 <nirik> just a short mention of our cloud plans....
18:22:14 <skvidal> we have new progress?
18:22:35 <nirik> not really... but I made a wiki page: https://fedoraproject.org/wiki/Infrastructure_private_cloud
18:22:58 <nirik> so, if people have suggestions/ideas, post them on list.
18:23:01 * skvidal counts backward from 5
18:23:16 <nirik> we are currently waiting for hardware still, but I think we have close to the config we want speced out.
18:23:32 <codemaniac> nirik: you have already finalized the budget and initial designs right , t
18:24:02 <codemaniac> what exactly the cloud is going to host ?
18:24:05 <nirik> codemaniac: yeah, for the initial part of it. We can hopefully expand it next year if it works out.
18:24:11 <skvidal> codemaniac: read the page - it explains
18:24:12 <nirik> see the wiki page...
18:24:52 <nirik> so, I think it will be very handy once we have it in place.
18:24:56 <ctria> just curiosity, can't we have any cloud solution from red hat or this is what red hat proposes?
18:25:13 <nirik> as we get closer to deploying it, we will no doubt want lots of people to help test it and such.
18:25:22 <herlo> nirik: who's doing the work? I'd love to help. I should have some time in the next couple of weeks to ramp up
18:25:31 <nirik> ctria: we evaluated a number of cloud tech, and found eucalyptus to meet our needs.
18:25:52 <nirik> herlo: skvidal is point on it... we are kinda holding on hardware tho.
18:25:59 <skvidal> hi
18:26:13 <herlo> nirik: skvidal: okay, just ping me when you are ready
18:26:14 <skvidal> we have a test cluster running. it doesn't do a lot b/c of the lack of ips to use w/it
18:26:27 <herlo> nod
18:26:44 <nirik> yeah. hard to use when you can't get to it from outside easily.
18:26:51 <codemaniac> me too in , would like to offer whatever i can
18:27:23 <nirik> #info waiting for hardware, will move forward once thats done.
18:27:24 <herlo> what will it take to get more IPs? Or do we need to free some up elsewhere?
18:27:53 <nirik> the new hardware will have a pile of external ips for us to use.
18:28:01 <herlo> okay, great
18:28:14 * herlo is done asking questions now
18:29:25 <nirik> cool...
18:29:28 <nirik> #topic Open floor
18:29:42 <nirik> anyone have items for open floor? general questions? etc
18:29:59 <skvidal> global updates
18:30:14 <skvidal> should I go ahead and just shoot a yum update on most everything?
18:30:59 <nirik> yeah, I think so.
18:31:02 <abadger1999> +1
18:31:08 <nirik> catch back up after the freeze.
18:31:10 <skvidal> ok
18:31:37 <nirik> I don't think we need reboots tho. The last update kernel fixes several security issues, but not sure how much they apply to our setup
18:32:04 <skvidal> second thing
18:32:12 <skvidal> builders and koji
18:32:25 <skvidal> I'd like to take buildvm01 and 02 out of koji for a while
18:32:31 <dgilmore> skvidal: ok
18:32:31 <skvidal> so I can mess with them
18:32:44 <nirik> sounds fine to me. Not much build activity right now anyhow.
18:32:56 <skvidal> and I'd like to find out about more migration to having more builders be vm-based
18:33:32 <nirik> what were the next steps in that plan?
18:33:53 <skvidal> 1. we need a way of telling koji that some pkgs need A LOT of ram
18:34:13 <skvidal> 2. we need to convert more of the x86-## boxes to buildvmhost
18:34:33 <dgilmore> skvidal: we have no way today of knowing that
18:34:50 <codemaniac> A lame question , what is koji ?
18:34:52 <nirik> dgilmore: won't we need that for arm anyhow? and seed with their list?
18:34:58 <misc> codemaniac: the build hub
18:35:00 <nirik> codemaniac: not lame at all. Our buildsystem...
18:35:11 <dgilmore> skvidal: for arm we have a seperate channel called heavybuilder, and it has a list of packages in the koji hub policy that it sends to that channel
18:35:13 <ctria> is there a list of packages to blame for LOT ram?
18:35:16 <dgilmore> but its not an automatic thing
18:35:23 <dgilmore> its a manually maintained list
18:35:25 <skvidal> dgilmore: oh - so we can't do that?
18:35:28 <skvidal> ctria: more or less
18:35:40 <nirik> http://fedorahosted.org/koji and http://koji.fedoraproject.org/koji/
18:35:41 <ctria> then we could probably create a new channel and redirect them there
18:35:44 <misc> codemaniac: https://fedoraproject.org/wiki/Using_the_Koji_build_system should answer the questions do not hesitate to ask for clarification
18:35:48 <dgilmore> skvidal: we can but its prone to failure and will need constant tweaking
18:35:56 <ctria> in the new channel to have better specs for builders
18:36:07 <skvidal> dgilmore: having all of our hosts have 8GB+ ram seems.... like poor resource use
18:36:13 <nirik> dgilmore: does changing that need a hub restart?
18:36:16 <codemaniac> thanks misc, nirik .Sure i will ask
18:36:20 <dgilmore> nirik: it does
18:36:20 <skvidal> dgilmore: since 95% of our pkgs don't need even 2GB of ram
18:36:25 <nirik> thats a pain.
18:36:25 <dgilmore> skvidal: right
18:36:39 <herlo> skvidal: +1 to not using a hammer for all builders
18:36:41 <skvidal> I suspect the largermem pkgs are probably under 50
18:36:46 <nirik> wonder if we could ask upstream koji for a better fix... some on the fly thing.
18:37:06 <misc> and I think the list would not grow much over time
18:37:19 <dgilmore> nirik: well we could probably write the policy so that if the task load is above say 4 where 6 is the max it goes to a bigger builder
18:37:28 <herlo> I've run into similar issues, I think it's possible to have a koji setting for ram requirements.
18:37:43 <dgilmore> nirik: but there could be something under a load of 4 that uses lots of ram to say link
18:37:53 <nirik> yeah.
18:37:59 <dgilmore> the last load is dynamically worked out based on avaergae build completion time
18:38:05 <dgilmore> the lask load
18:38:10 <dgilmore> task
18:38:11 <herlo> lol
18:38:14 <misc> couldn't cgroup be used to track the ram used by a set of process ?
18:38:25 <dgilmore> misc: builders are all rhel6
18:38:53 <nirik> so, I guess we all ponder on the problem and try and come up with a good solution...
18:39:02 <skvidal> nirik: I think your suggestion is a good place to start
18:39:07 <skvidal> ask koji upstream for help
18:39:24 <misc> ok so can we track the memory on vm ?
18:39:25 <nirik> it would be nice to have a better solution for arm too...
18:39:33 <misc> I think that's how balooning does, no ?
18:39:36 <dgilmore> nirik: probably the best solution would be to have koji keep track of ram usage during builds and store that. then we could do things nicely
18:39:43 <dgilmore> but thats likely alot of work
18:39:43 <nirik> misc: the problem is that we get a package to build and we don't know how much mem it will need to build.
18:39:45 <skvidal> misc: you mean on the fly allocation of ram?
18:39:52 <misc> ( or maybe polling memory in the vm every minutes, to have some stats )
18:39:53 <herlo> +1 nirik's suggestion
18:40:00 <skvidal> misc: normally the event which tells us it is out of memory is that it OOMs
18:40:01 <nirik> dgilmore: hum... yeah.
18:40:26 <dgilmore> we can somewhat safely make assumptions based on previous builds
18:40:30 <nirik> or perhaps a FTBFS run could also tell us which packages need more mem and we set the list based on that?
18:40:46 <nirik> at least it would be a good seed.
18:41:07 <dgilmore> i suspect that having a task load of more than 4 would cover 95% of cases
18:41:11 <dgilmore> maybe 100%
18:41:12 <nirik> #info need a solution to find builds that need lots of memory.
18:41:13 <ctria> there is definition for package name in koji policies http://fedoraproject.org/wiki/Koji/Policies
18:41:22 <nirik> #idea ask koji developers upstream for a solution
18:41:44 <skvidal> nirik: hmm
18:42:04 <skvidal> nirik: so here's an idea of what i could do on buildvm##
18:42:14 <skvidal> it would take a while, though...
18:42:30 <skvidal> iteratively build every pkg on the 4gb they have available
18:42:37 <skvidal> and see which ones cause it to oom
18:42:53 <nirik> that would take quite a long while...
18:42:56 <dgilmore> channel = method createrepo :: use createrepo has req_channel :: req is_child_task :: parent source */gcc* :: use heavybuilder source */glibc* :: use heavybuilder source */kernel* :: use heavybuilder
18:42:58 <skvidal> I know
18:43:07 <skvidal> dgilmore: -ENOCONTEXT
18:43:09 <dgilmore> thats a snippet of the policy use on arm
18:43:28 <nirik> once we have the cloud setup, we could fire up N builders with 4GB and mass rebuild...
18:43:28 <dgilmore> it came aross bas
18:43:44 <skvidal> nirik: so...
18:43:49 <nirik> ceph is apparently another one.
18:43:58 <nirik> webkitgtk is also a pig
18:44:01 <skvidal> nirik: I could convert an existing x86-## box
18:44:05 <skvidal> nirik: to buildvmhost
18:44:06 <dgilmore> skvidal: http://fpaste.org/QfFj/ thats the full current policy used on arm
18:44:16 <skvidal> nirik: and then i'd have 8+ builders to test with
18:45:00 <nirik> yeah, still will take a while, but less while.
18:45:53 <skvidal> I'm open to other options...
18:45:56 <skvidal> considering our build load
18:46:03 <skvidal> I could probably rob 2 more machines w/o us losing out muchj
18:46:24 <nirik> I'm fine with that, but perhaps we should see if koji dev's have any brilliant ideas we are missing first?
18:46:24 <dgilmore> skvidal: sure. we really only hit the builders really hard for mass rebuilds
18:46:25 <skvidal> our peak afaict has been like 23% use
18:46:36 <skvidal> nirik: you're no fun at all !
18:46:46 <nirik> I know, right?
18:46:52 <skvidal> nirik: I can bounce an email to buildsys list
18:46:58 <misc> dgilmore: technically, that's some form of mass rebuild, so maybe this can be done at the same time
18:47:20 <misc> ( or, if someone is gonna rebuild everything, this can produce a list of FBTS and a list of package that need more ram )
18:47:40 <dgilmore> nirik: ill talk with the mikes
18:48:07 <skvidal> dgilmore: even better
18:48:12 <nirik> works for me.
18:48:22 <nirik> #action dgilmore to talk to mikes about koji options.
18:48:39 <nirik> misc: we typically only do mass rebuilds when there's a good reason... rpm payload change, gcc change, etc.
18:48:40 <CodeBlock> agh, late, sorry
18:48:41 * CodeBlock here
18:49:08 <nirik> misc: but we hope to use the new cloud setup to mass rebuild to check the FTBFS stuff.
18:49:11 <skvidal> nirik: I think I'd like the opportunity to see if I can do a sensible port of ftbfs over to mockchain, etc
18:49:38 <nirik> yeah, would be good. keeping in mind hopefully that it would be portable to the privatecloud.
18:49:52 <skvidal> it should just be ssh-based
18:50:08 <skvidal> so the only difference is how you get the instances started
18:50:13 <skvidal> not how you spawn off jobs
18:50:26 <nirik> yeah.
18:50:45 <skvidal> question about ftbfs
18:50:55 <skvidal> does that build all rawhide srpms against rawhide?
18:51:20 <nirik> yep.
18:51:34 <nirik> we can ask mdomsch for his scripts.
18:51:55 <dgilmore> skvidal: i believe thats how mdomsch did it
18:51:58 <nirik> I think it was just basically: build everything once, get list of fails, rebuild, cycle until you have no changes.
18:52:19 <skvidal> nirik: nod
18:52:26 <skvidal> nirik: I was checking on which base it built from
18:52:30 <nirik> but there's issues with that too I guess.
18:52:32 <herlo> can someone expand ftbfs for me plz
18:52:32 <skvidal> ie: did he take a single rawhide snapshot
18:52:40 <skvidal> Fails To Build From Source
18:52:42 <nirik> herlo: fails to build from source.
18:52:45 <herlo> tx
18:52:49 <herlo> skvidal wins!
18:52:51 <nirik> ie, you get the src.rpm we have in git and it doesn't work.
18:53:06 <nirik> http://fedoraproject.org/wiki/Fails_to_build_from_source
18:53:25 <herlo> yeah, I have done something very similar with my koji project
18:53:44 <nirik> rawhide can be a challenge, since there's so many moving packages. ;(
18:53:50 * dgilmore needs to run
18:54:08 <skvidal> nirik: which is why I was thinking about taking a snapshot of rawhide into a local mirror
18:54:13 <skvidal> nirik: that all the builders can see
18:54:47 <nirik> you mean for the build repo/binary rpms? or the git repos building from?
18:54:51 <nirik> or both.
18:55:25 <skvidal> build repo/binary
18:55:37 <skvidal> so that what it is building against for the build time doesn't change
18:55:54 <nirik> yeah.
18:56:04 <nirik> although if we can get it down to under a day, it would be the same rawhide.
18:56:18 <skvidal> I cannot fathom how many machines we would need for that
18:56:28 <nirik> large number...
18:56:32 <skvidal> dgilmore: can I convert x86-16,17, 18 over to buildvmhosts?
18:57:20 <nirik> skvidal: I think he had to run...
18:57:27 <skvidal> okay
18:57:34 <nirik> lets continue out of meeting on this...
18:57:36 <skvidal> ok
18:57:39 <skvidal> sorryt for running on
18:57:41 <nirik> anything else for meeting?
18:57:47 <nirik> no worries at all. Good to discuss.
18:58:06 <skvidal> one more item
18:58:26 <skvidal> does anyone else have any interest in ansible and/or tryting to write playbooks?
18:58:33 * herlo does
18:58:38 <herlo> just haven't had time yet
18:58:42 <skvidal> understood.
18:58:47 <herlo> will next week or two for sure
18:58:54 <skvidal> one last thing
18:58:58 <herlo> depending on what gets done with euca, I'd be up for it
18:59:01 <nirik> yeah, I'd like to look at ansible more too.
18:59:09 <skvidal> I didn't hear any horrible screaming about the ssh key thing last week
18:59:15 <skvidal> just some minor grumbling
18:59:27 <skvidal> so.... I'm going to go ahead and make a magic key and put it on lockbox01
18:59:44 <skvidal> and start playing with it on the builder installs I've been working on
18:59:51 <nirik> fair enough.
19:00:03 <nirik> it would only be those instances for now, right? until we expand ansible use?
19:00:09 <skvidal> exactly
19:00:12 <skvidal> nothing else will have it
19:00:23 <nirik> seems sane. Shake out any issues now
19:01:06 <nirik> #info will setup initial ansible ssh keys for testing with.
19:01:19 <skvidal> okay, I'm done with all my openfloor things
19:01:20 <nirik> ok, shall we call it a meeting?
19:01:23 <nirik>
19:01:36 <herlo> yup
19:01:40 <herlo> call it
19:01:43 <CodeBlock> I have one quick thing
19:02:00 <nirik> CodeBlock: go for it.
19:02:33 <CodeBlock> we talked about Stickynotes as a pastebin last week, I quickly made a Fedora theme mockup (showed nirik last week, but wanted to get some more voices about it)
19:02:33 <CodeBlock> http://images.srv1.elrod.me/stickynotes-fedora.png
19:02:44 <nirik> CodeBlock: oh, codemaniac was looking to see if he could help with dpsearch. If you catch him on-line, see if he can assist any
19:03:13 <nirik> I think it's a pretty good start... might need more blue.
19:03:14 <CodeBlock> nirik: cool, he's been trying to ping me and our schedules have been totally missing each other
19:03:48 <herlo> CodeBlock: oh, cool
19:03:52 * herlo looks
19:04:00 <nirik> #info provide feedback on sticky-notes theme: http://images.srv1.elrod.me/stickynotes-fedora.png
19:04:03 <herlo> oh, very nice!
19:04:30 <herlo> CodeBlock: I like the theme, I bet we could package that up pretty quickly
19:04:48 <herlo> ping me when you have a package and I'll do a review
19:05:02 <herlo> we can get that into the infra repo soon I hope.
19:05:02 <nirik> herlo: it's already up. dcr226 has it up and I am reviewing.
19:05:07 <CodeBlock> herlo: Cool. It needs some coding work, I just wanted to get a working prototype, but I will continue working on it
19:05:09 <herlo> nirik: with the theme?
19:05:19 <herlo> I'm specifically talking about the theme
19:05:23 <nirik> herlo: oh, sorry, no, not the theme...
19:05:27 <nirik> yeah, would be good to get that in too.
19:05:28 <herlo> nw
19:05:42 <nirik> ok, calling it a meeting as we are over.
19:05:45 <herlo> nirik: yeah, knew about the stickynotes review
19:05:54 <herlo> CodeBlock: awesome.
19:05:54 <nirik> Lets all continue over on #fedora-admin and #fedora-noc and #fedora-apps.
19:05:58 <herlo> indeed
19:05:58 <nirik> thanks for coming everyone!
19:06:02 <nirik> #endmeeting
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 04-20-2012, 03:53 AM
Arijit Dutta
 
Default Summary/Minutes from today's Fedora infrastructure meeting (2012-04-19) (new time)

Dear Kevin ,
Last night i was knid of on the fly travelling in a train to reach my home , so was getting disconnected frequently .Thanks for all the resources you have shared earlier with me , i just gone through some starter documentations about git , puppet and nagios .The Fedora Infrastructure arch is also getting clearer to me day by day .Have not seen earlier such a vast network architechture where hundreds of hosts getting configurations from a central CM server(lockbox01) ,and also hundreds of people doing config changes and all are in sync with GIT .

I was also trying to contact CodeBlock on IRC to check if i can help him in his wiki search project .Please let me know if any course of action you have already planned to me .

Thanks and Best Regards ,Arijit Dutta .
On Fri, Apr 20, 2012 at 12:36 AM, Kevin Fenzi <kevin@scrye.com> wrote:

============================================

#fedora-meeting: Infrastructure (2012-04-19)

============================================





Meeting started by nirik at 18:00:00 UTC. The full logs are available at

http://meetbot.fedoraproject.org/fedora-meeting/2012-04-19/infrastructure.2012-04-19-18.00.log.html


.







Meeting summary

---------------

* Robot Roll Call *(nirik, 18:00:01)



* New folks introductions and Apprentice tasks. *(nirik, 18:02:13)



* two factor auth status *(nirik, 18:04:02)



* Staging re-work status *(nirik, 18:04:29)



* Applications status / discussion *(nirik, 18:06:38)

** reviews needed to move fedmsg ahead. *(nirik, 18:10:09)

** ACTION: threebean to post list *(nirik, 18:10:20)

** LINK: http://gitorious.org/sticky-notes * (nirik, 18:12:46)

** looking at sticky-notes for a pastebin server *(nirik, 18:14:03)



* Meeting time *(nirik, 18:17:44)



* Upcoming Tasks/Items *(nirik, 18:19:34)

** 2012-05-01 to 2012-05-15 - F17 Final Freeze. *(nirik, 18:19:43)

** 2012-05-01 - nag fi-apprentices. *(nirik, 18:19:43)

** 2011-05-03 - gitweb-cache removal day. *(nirik, 18:19:43)

** 2012-05-09 - Check if puppet works on f17 yet. *(nirik, 18:19:43)

** 2012-05-10 - drop inactive fi-apprentices *(nirik, 18:19:43)

** 2012-05-15 - F17 release *(nirik, 18:19:44)



* Private Cloud *(nirik, 18:22:06)

** waiting for hardware, will move forward once thats done. *(nirik,

* *18:27:23)



* Open floor *(nirik, 18:29:28)

** LINK: http://fedorahosted.org/koji and

* *http://koji.fedoraproject.org/koji/ *(nirik, 18:35:40)

** need a solution to find builds that need lots of memory. *(nirik,

* *18:41:12)

** IDEA: ask koji developers upstream for a solution *(nirik, 18:41:22)

** ACTION: dgilmore to talk to mikes about koji options. *(nirik,

* *18:48:22)

** LINK: http://fedoraproject.org/wiki/Fails_to_build_from_source

* *(nirik, 18:53:06)

** will setup initial ansible ssh keys for testing with. *(nirik,

* *19:01:06)

** LINK: http://images.srv1.elrod.me/stickynotes-fedora.png

* *(CodeBlock, 19:02:33)

** provide feedback on sticky-notes theme:

* *http://images.srv1.elrod.me/stickynotes-fedora.png *(nirik,

* *19:04:00)



Meeting ended at 19:06:02 UTC.









Action Items

------------

* threebean to post list

* dgilmore to talk to mikes about koji options.









Action Items, by person

-----------------------

* dgilmore

** dgilmore to talk to mikes about koji options.

* threebean

** threebean to post list

* **UNASSIGNED**

** (none)









People Present (lines said)

---------------------------

* nirik (145)

* skvidal (75)

* dgilmore (35)

* herlo (32)

* codemaniac (11)

* misc (9)

* ctria (7)

* CodeBlock (7)

* abadger1999 (5)

* zodbot (4)

* threebean (4)

* ianweller (3)

* pingou (3)

* mmello (3)

* lmacken (2)

* athmane (1)

* smooge (0)

* ricky (0)

* mdomsch (0)

--

18:00:00 <nirik> #startmeeting Infrastructure (2012-04-19)

18:00:00 <zodbot> Meeting started Thu Apr 19 18:00:00 2012 UTC. *The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.

18:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.

18:00:01 <nirik> #meetingname infrastructure

18:00:01 <zodbot> The meeting name has been set to 'infrastructure'

18:00:01 <nirik> #topic Robot Roll Call

18:00:01 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean

18:00:01 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean

18:00:11 <nirik> who all is around for a infrastructure meeting?

18:00:15 * lmacken

18:00:19 * skvidal is here

18:00:23 * threebean is here

18:00:27 * athmane is here

18:00:31 * ianweller is last-minuting a physics lab report but will pay attention

18:01:18 * pingou

18:01:24 <dgilmore> buenos

18:02:06 <nirik> ok, lets go ahead and dive in.

18:02:13 <nirik> #topic New folks introductions and Apprentice tasks.

18:02:14 <nirik> If any new folks want to give a quick one line bio or any apprentices

18:02:14 <nirik> would like to ask general questions, they can do so here.

18:02:32 <nirik> any new folks? or questions on easyfix tickets or general getting started questions?

18:03:54 <nirik> ok, moving on then...

18:04:02 <nirik> #topic two factor auth status

18:04:03 <skvidal> no changes on 2fa - pto the beginning of this week.

18:04:18 <nirik> ok. We can ping it next week...

18:04:29 <nirik> #topic Staging re-work status

18:04:34 <nirik> I've worked on this some.

18:04:40 <nirik> I've commited docs on the new setup.

18:04:56 <nirik> please read http://infrastructure.fedoraproject.org/infra/docs/staging.txt and let me know if anything needs fixing.


18:05:18 <nirik> I'm going to try and get the bapp01->02 move done, and easyfix moved to production, then we can do the staging branch killing.

18:05:53 <nirik> any questions on that ?

18:06:35 <nirik> ok, I'll schedule the actual staging killing after those other 2 things are done.

18:06:38 <nirik> #topic Applications status / discussion

18:06:44 <nirik> any news on applications this week?

18:07:08 <nirik> abadger1999 / threebean / lmacken / pingou / CodeBlock / anyone else who works on apps.

18:07:33 <abadger1999> New python-fedora this week that should resolve some issues clients were seeing.

18:07:45 <nirik> yeah, thanks for that fix.

18:07:54 <lmacken> abadger1999++

18:08:12 * nirik still doesn't fully understand it, but thats ok

18:08:16 <abadger1999> I'm not aware of any of our scripts that were running into this but it won't hurt them to update on the infra boxes.

18:08:58 <threebean> fedmsg progress - got a mediawiki plugin done and tested locally. *waiting still on lots of package reviews.

18:09:13 <ianweller> he's alive!

18:09:44 <nirik> threebean: could you perhaps drop a list to the infra list? or a wiki page link or something? then we can try and divide them up...

18:09:54 <threebean> sure thing

18:10:08 * threebean writes that now

18:10:09 <nirik> #info reviews needed to move fedmsg ahead.

18:10:20 <nirik> #action threebean to post list

18:11:08 <nirik> ok, any other app news?

18:11:22 <dgilmore> where did we ever get with re-writing fpaste?

18:12:15 <nirik> dgilmore: oh yeah, have some news on that...

18:12:32 <nirik> we are looking at possibly using sticky-notes instead for the server side.

18:12:46 <nirik> http://gitorious.org/sticky-notes

18:12:47 <abadger1999> mmello, I believe was interested in helping out with reviews, too.

18:13:01 <nirik> it's under review, pending some fixes from upstream.

18:13:13 <nirik> athmane has found some issues and submitted patches to it.

18:13:24 <mmello> abadger1999: yes..I'm

18:13:32 <dgilmore> nirik: cool

18:13:44 <dgilmore> mmello: get to work

18:13:50 <mmello> abadger1999: threebean is helping to understand the process and I'm currently review my first package

18:13:53 <mmello> dgilmore:

18:14:03 <nirik> #info looking at sticky-notes for a pastebin server

18:14:11 <dgilmore> nirik: would the current fpaste client work with the new server? or need rewritten?

18:14:18 <nirik> there's a command line 'pastebinit' already in fedora.

18:14:25 <dgilmore> cool

18:14:35 <nirik> so, we could look at making it obsolete fpaste and perhaps provide a compat link.

18:14:50 <nirik> it misses a few things like the --sysinfo tho.

18:15:18 <nirik> anyhow, it looks promising.

18:15:34 <codemaniac> hello nirik and everyone

18:15:44 <nirik> morning codemaniac. welcome.

18:16:01 <codemaniac> good morning to you too nirik

18:16:03 <dgilmore> nirik: figured id ask

18:16:09 <pingou> but php

18:16:15 <ctria> hi all. did i get lost timezones again?

18:16:17 <nirik> pingou: yeah, its php.

18:16:27 <nirik> ctria: meeting just started 15m ago.

18:16:35 <nirik> any further news on applications?

18:16:35 <ctria>

18:16:49 <codemaniac> sorry for joining late team

18:16:56 <nirik> no problem at all.

18:17:24 <codemaniac> CodeBlock: you around ?

18:17:39 <nirik> ok, will move along then...

18:17:44 <nirik> #topic Meeting time

18:17:54 <nirik> is everyone ok with this meeting time? 18UTC?

18:17:59 <skvidal> +1

18:18:00 <pingou> +1

18:18:03 <ianweller> +1

18:18:07 <abadger1999> +1

18:18:25 <nirik> ok then... seems popular.

18:18:47 <codemaniac> yea niriki , i am very much comfortable with this time , its +5:30 UTC here

18:19:07 <codemaniac> +1024

18:19:13 <nirik> cool.

18:19:34 <nirik> #topic Upcoming Tasks/Items

18:19:43 <nirik> #info 2012-05-01 to 2012-05-15 - F17 Final Freeze.

18:19:43 <nirik> #info 2012-05-01 - nag fi-apprentices.

18:19:43 <nirik> #info 2011-05-03 - gitweb-cache removal day.

18:19:43 <nirik> #info 2012-05-09 - Check if puppet works on f17 yet.

18:19:43 <nirik> #info 2012-05-10 - drop inactive fi-apprentices

18:19:44 <nirik> #info 2012-05-15 - F17 release

18:19:53 <nirik> so final freeze is coming up.

18:20:25 <nirik> One thing I will probibly schedule soon is a migration of our hosted machine... hosted03 -> hosted01/02. Should be a pretty easy migration.

18:20:37 <nirik> anyone have other items they are working on they would like to schedule or shout out?

18:21:46 <nirik> ok, I'll take that as a no.

18:21:55 <dgilmore> nothing here

18:22:06 <nirik> #topic Private Cloud

18:22:14 <nirik> just a short mention of our cloud plans....

18:22:14 <skvidal> we have new progress?

18:22:35 <nirik> not really... but I made a wiki page: https://fedoraproject.org/wiki/Infrastructure_private_cloud

18:22:58 <nirik> so, if people have suggestions/ideas, post them on list.

18:23:01 * skvidal counts backward from 5

18:23:16 <nirik> we are currently waiting for hardware still, but I think we have close to the config we want speced out.

18:23:32 <codemaniac> nirik: you have already finalized the budget and initial designs right , t

18:24:02 <codemaniac> what exactly the cloud is going to host ?

18:24:05 <nirik> codemaniac: yeah, for the initial part of it. We can hopefully expand it next year if it works out.

18:24:11 <skvidal> codemaniac: read the page - it explains

18:24:12 <nirik> see the wiki page...

18:24:52 <nirik> so, I think it will be very handy once we have it in place.

18:24:56 <ctria> just curiosity, can't we have any cloud solution from red hat or this is what red hat proposes?

18:25:13 <nirik> as we get closer to deploying it, we will no doubt want lots of people to help test it and such.

18:25:22 <herlo> nirik: who's doing the work? I'd love to help. I should have some time in the next couple of weeks to ramp up

18:25:31 <nirik> ctria: we evaluated a number of cloud tech, and found eucalyptus to meet our needs.

18:25:52 <nirik> herlo: skvidal is point on it... we are kinda holding on hardware tho.

18:25:59 <skvidal> hi

18:26:13 <herlo> nirik: skvidal: okay, just ping me when you are ready

18:26:14 <skvidal> we have a test cluster running. it doesn't do a lot b/c of the lack of ips to use w/it

18:26:27 <herlo> nod

18:26:44 <nirik> yeah. hard to use when you can't get to it from outside easily.

18:26:51 <codemaniac> me too in , would like to offer whatever i can

18:27:23 <nirik> #info waiting for hardware, will move forward once thats done.

18:27:24 <herlo> what will it take to get more IPs? Or do we need to free some up elsewhere?

18:27:53 <nirik> the new hardware will have a pile of external ips for us to use.

18:28:01 <herlo> okay, great

18:28:14 * herlo is done asking questions now

18:29:25 <nirik> cool...

18:29:28 <nirik> #topic Open floor

18:29:42 <nirik> anyone have items for open floor? general questions? etc

18:29:59 <skvidal> global updates

18:30:14 <skvidal> should I go ahead and just shoot a yum update on most everything?

18:30:59 <nirik> yeah, I think so.

18:31:02 <abadger1999> +1

18:31:08 <nirik> catch back up after the freeze.

18:31:10 <skvidal> ok

18:31:37 <nirik> I don't think we need reboots tho. The last update kernel fixes several security issues, but not sure how much they apply to our setup

18:32:04 <skvidal> second thing

18:32:12 <skvidal> builders and koji

18:32:25 <skvidal> I'd like to take buildvm01 and 02 out of koji for a while

18:32:31 <dgilmore> skvidal: ok

18:32:31 <skvidal> so I can mess with them

18:32:44 <nirik> sounds fine to me. Not much build activity right now anyhow.

18:32:56 <skvidal> and I'd like to find out about more migration to having more builders be vm-based

18:33:32 <nirik> what were the next steps in that plan?

18:33:53 <skvidal> 1. we need a way of telling koji that some pkgs need A LOT of ram

18:34:13 <skvidal> 2. we need to convert more of the x86-## boxes to buildvmhost

18:34:33 <dgilmore> skvidal: we have no way today of knowing that

18:34:50 <codemaniac> A lame question , what is koji ?

18:34:52 <nirik> dgilmore: won't we need that for arm anyhow? and seed with their list?

18:34:58 <misc> codemaniac: the build hub

18:35:00 <nirik> codemaniac: not lame at all. Our buildsystem...

18:35:11 <dgilmore> skvidal: for arm we have a seperate channel called heavybuilder, and it has a list of packages in the koji hub policy that it sends to that channel

18:35:13 <ctria> is there a list of packages to blame for LOT ram?

18:35:16 <dgilmore> but its not an automatic thing

18:35:23 <dgilmore> its a manually maintained list

18:35:25 <skvidal> dgilmore: oh - so we can't do that?

18:35:28 <skvidal> ctria: more or less

18:35:40 <nirik> http://fedorahosted.org/koji and http://koji.fedoraproject.org/koji/


18:35:41 <ctria> then we could probably create a new channel and redirect them there

18:35:44 <misc> codemaniac: https://fedoraproject.org/wiki/Using_the_Koji_build_system should answer the questions do not hesitate to ask for clarification


18:35:48 <dgilmore> skvidal: we can but its prone to failure *and will need constant tweaking

18:35:56 <ctria> in the new channel to have better specs for builders

18:36:07 <skvidal> dgilmore: having all of our hosts have 8GB+ ram seems.... like poor resource use

18:36:13 <nirik> dgilmore: does changing that need a hub restart?

18:36:16 <codemaniac> thanks misc, nirik .Sure i will ask

18:36:20 <dgilmore> nirik: it does

18:36:20 <skvidal> dgilmore: since 95% of our pkgs don't need even 2GB of ram

18:36:25 <nirik> thats a pain.

18:36:25 <dgilmore> skvidal: right

18:36:39 <herlo> skvidal: +1 to not using a hammer for all builders

18:36:41 <skvidal> I suspect the largermem pkgs are probably under 50

18:36:46 <nirik> wonder if we could ask upstream koji for a better fix... some on the fly thing.

18:37:06 <misc> and I think the list would not grow much over time

18:37:19 <dgilmore> nirik: well we could probably write the policy so that if the task load is above say 4 *where *6 is the max it goes to a bigger builder

18:37:28 <herlo> I've run into similar issues, I think it's possible to have a koji setting for ram requirements.

18:37:43 <dgilmore> nirik: but there could be something under a load of 4 that uses lots of ram to say link

18:37:53 <nirik> yeah.

18:37:59 <dgilmore> the last load is dynamically worked out based on avaergae build completion time

18:38:05 <dgilmore> the lask load

18:38:10 <dgilmore> task

18:38:11 <herlo> lol

18:38:14 <misc> couldn't cgroup be used to track the ram used by a set of process ?

18:38:25 <dgilmore> misc: builders are all rhel6

18:38:53 <nirik> so, I guess we all ponder on the problem and try and come up with a good solution...

18:39:02 <skvidal> nirik: I think your suggestion is a good place to start

18:39:07 <skvidal> ask koji upstream for help

18:39:24 <misc> ok so can we track the memory on vm ?

18:39:25 <nirik> it would be nice to have a better solution for arm too...

18:39:33 <misc> I think that's how balooning does, no ?

18:39:36 <dgilmore> nirik: probably the best solution would be to have koji keep track of ram usage during builds and store that. then we could do things nicely

18:39:43 <dgilmore> but thats likely alot of work

18:39:43 <nirik> misc: the problem is that we get a package to build and we don't know how much mem it will need to build.

18:39:45 <skvidal> misc: you mean on the fly allocation of ram?

18:39:52 <misc> ( or maybe polling memory in the vm every minutes, to have some stats )

18:39:53 <herlo> +1 nirik's suggestion

18:40:00 <skvidal> misc: normally the event which tells us it is out of memory is that it OOMs

18:40:01 <nirik> dgilmore: hum... yeah.

18:40:26 <dgilmore> we can somewhat safely make assumptions based on previous builds

18:40:30 <nirik> or perhaps a FTBFS run could also tell us which packages need more mem and we set the list based on that?

18:40:46 <nirik> at least it would be a good seed.

18:41:07 <dgilmore> i suspect that having a task load of more than 4 would cover 95% of cases

18:41:11 <dgilmore> maybe 100%

18:41:12 <nirik> #info need a solution to find builds that need lots of memory.

18:41:13 <ctria> there is definition for package name in koji policies http://fedoraproject.org/wiki/Koji/Policies

18:41:22 <nirik> #idea ask koji developers upstream for a solution

18:41:44 <skvidal> nirik: hmm

18:42:04 <skvidal> nirik: so here's an idea of what i could do on buildvm##

18:42:14 <skvidal> it would take a while, though...

18:42:30 <skvidal> iteratively build every pkg on the 4gb they have available

18:42:37 <skvidal> and see which ones cause it to oom

18:42:53 <nirik> that would take quite a long while...

18:42:56 <dgilmore> channel = method createrepo :: use createrepo has req_channel :: req is_child_task :: parent source */gcc* *:: use heavybuilder source */glibc* :: use heavybuilder source */kernel* :: use heavybuilder


18:42:58 <skvidal> I know

18:43:07 <skvidal> dgilmore: -ENOCONTEXT

18:43:09 <dgilmore> thats a snippet of the policy use on arm

18:43:28 <nirik> once we have the cloud setup, we could fire up N builders with 4GB and mass rebuild...

18:43:28 <dgilmore> it came aross bas

18:43:44 <skvidal> nirik: so...

18:43:49 <nirik> ceph is apparently another one.

18:43:58 <nirik> webkitgtk is also a pig

18:44:01 <skvidal> nirik: I could convert an existing x86-## box

18:44:05 <skvidal> nirik: to *buildvmhost

18:44:06 <dgilmore> skvidal: http://fpaste.org/QfFj/ thats the full current policy used on arm

18:44:16 <skvidal> nirik: and then i'd have 8+ builders to test with

18:45:00 <nirik> yeah, still will take a while, but less while.

18:45:53 <skvidal> I'm open to other options...

18:45:56 <skvidal> considering our build load

18:46:03 <skvidal> I could probably rob 2 more machines w/o us losing out muchj

18:46:24 <nirik> I'm fine with that, but perhaps we should see if koji dev's have any brilliant ideas we are missing first?

18:46:24 <dgilmore> skvidal: sure. we really only hit the builders really hard for mass rebuilds

18:46:25 <skvidal> our peak afaict has been like 23% use

18:46:36 <skvidal> nirik: you're no fun at all !

18:46:46 <nirik> I know, right?

18:46:52 <skvidal> nirik: I can bounce an email to buildsys list

18:46:58 <misc> dgilmore: technically, that's some form of mass rebuild, so maybe this can be done at the same time

18:47:20 <misc> ( or, if someone is gonna rebuild everything, this can produce a list of FBTS and a list of package that need more ram )

18:47:40 <dgilmore> nirik: ill talk with the mikes

18:48:07 <skvidal> dgilmore: even better

18:48:12 <nirik> works for me.

18:48:22 <nirik> #action dgilmore to talk to mikes about koji options.

18:48:39 <nirik> misc: we typically only do mass rebuilds when there's a good reason... rpm payload change, gcc change, etc.

18:48:40 <CodeBlock> agh, late, sorry

18:48:41 * CodeBlock here

18:49:08 <nirik> misc: but we hope to use the new cloud setup to mass rebuild to check the FTBFS stuff.

18:49:11 <skvidal> nirik: I think I'd like the opportunity to see if I can do a sensible port of ftbfs over to mockchain, etc

18:49:38 <nirik> yeah, would be good. keeping in mind hopefully that it would be portable to the privatecloud.

18:49:52 <skvidal> it should just be ssh-based

18:50:08 <skvidal> so the only difference is how you get the instances started

18:50:13 <skvidal> not how you spawn off jobs

18:50:26 <nirik> yeah.

18:50:45 <skvidal> question about ftbfs

18:50:55 <skvidal> does that build all rawhide srpms against rawhide?

18:51:20 <nirik> yep.

18:51:34 <nirik> we can ask mdomsch for his scripts.

18:51:55 <dgilmore> skvidal: i believe thats how mdomsch did it

18:51:58 <nirik> I think it was just basically: build everything once, get list of fails, rebuild, cycle until you have no changes.

18:52:19 <skvidal> nirik: nod

18:52:26 <skvidal> nirik: I was checking on which base it built from

18:52:30 <nirik> but there's issues with that too I guess.

18:52:32 <herlo> can someone expand ftbfs for me plz

18:52:32 <skvidal> ie: did he take a single rawhide snapshot

18:52:40 <skvidal> Fails To Build From Source

18:52:42 <nirik> herlo: fails to build from source.

18:52:45 <herlo> tx

18:52:49 <herlo> skvidal wins!

18:52:51 <nirik> ie, you get the src.rpm we have in git and it doesn't work.

18:53:06 <nirik> http://fedoraproject.org/wiki/Fails_to_build_from_source

18:53:25 <herlo> yeah, I have done something very similar with my koji project

18:53:44 <nirik> rawhide can be a challenge, since there's so many moving packages. ;(

18:53:50 * dgilmore needs to run

18:54:08 <skvidal> nirik: which is why I was thinking about taking a snapshot of rawhide into a local mirror

18:54:13 <skvidal> nirik: that all the builders can see

18:54:47 <nirik> you mean for the build repo/binary rpms? or the git repos building from?

18:54:51 <nirik> or both.

18:55:25 <skvidal> build repo/binary

18:55:37 <skvidal> so that what it is building against for the build time doesn't change

18:55:54 <nirik> yeah.

18:56:04 <nirik> although if we can get it down to under a day, it would be the same rawhide.

18:56:18 <skvidal> I cannot fathom how many machines we would need for that

18:56:28 <nirik> large number...

18:56:32 <skvidal> dgilmore: can I convert x86-16,17, 18 over to buildvmhosts?

18:57:20 <nirik> skvidal: I think he had to run...

18:57:27 <skvidal> okay

18:57:34 <nirik> lets continue out of meeting on this...

18:57:36 <skvidal> ok

18:57:39 <skvidal> sorryt for running on

18:57:41 <nirik> anything else for meeting?

18:57:47 <nirik> no worries at all. Good to discuss.

18:58:06 <skvidal> one more item

18:58:26 <skvidal> does anyone else have any interest in ansible and/or tryting to write playbooks?

18:58:33 * herlo does

18:58:38 <herlo> just haven't had time yet

18:58:42 <skvidal> understood.

18:58:47 <herlo> will next week or two for sure

18:58:54 <skvidal> one last thing

18:58:58 <herlo> depending on what gets done with euca, I'd be up for it

18:59:01 <nirik> yeah, I'd like to look at ansible more too.

18:59:09 <skvidal> I didn't hear any horrible screaming about the ssh key thing last week

18:59:15 <skvidal> just some minor grumbling

18:59:27 <skvidal> so.... I'm going to go ahead and make a magic key and put it on lockbox01

18:59:44 <skvidal> and start playing with it on the builder installs I've been working on

18:59:51 <nirik> fair enough.

19:00:03 <nirik> it would only be those instances for now, right? until we expand ansible use?

19:00:09 <skvidal> exactly

19:00:12 <skvidal> nothing else will have it

19:00:23 <nirik> seems sane. Shake out any issues now

19:01:06 <nirik> #info will setup initial ansible ssh keys for testing with.

19:01:19 <skvidal> okay, I'm done with all my openfloor things

19:01:20 <nirik> ok, shall we call it a meeting?

19:01:23 <nirik>

19:01:36 <herlo> yup

19:01:40 <herlo> call it

19:01:43 <CodeBlock> I have one quick thing

19:02:00 <nirik> CodeBlock: go for it.

19:02:33 <CodeBlock> we talked about Stickynotes as a pastebin last week, I quickly made a Fedora theme mockup (showed nirik last week, but wanted to get some more voices about it)

19:02:33 <CodeBlock> http://images.srv1.elrod.me/stickynotes-fedora.png

19:02:44 <nirik> CodeBlock: oh, codemaniac was looking to see if he could help with dpsearch. If you catch him on-line, see if he can assist any

19:03:13 <nirik> I think it's a pretty good start... might need more blue.

19:03:14 <CodeBlock> nirik: cool, he's been trying to ping me and our schedules have been totally missing each other

19:03:48 <herlo> CodeBlock: oh, cool

19:03:52 * herlo looks

19:04:00 <nirik> #info provide feedback on sticky-notes theme: http://images.srv1.elrod.me/stickynotes-fedora.png

19:04:03 <herlo> oh, very nice!

19:04:30 <herlo> CodeBlock: I like the theme, I bet we could package that up pretty quickly

19:04:48 <herlo> ping me when you have a package and I'll do a review

19:05:02 <herlo> we can get that into the infra repo soon I hope.

19:05:02 <nirik> herlo: it's already up. dcr226 has it up and I am reviewing.

19:05:07 <CodeBlock> herlo: Cool. It needs some coding work, I just wanted to get a working prototype, but I will continue working on it

19:05:09 <herlo> nirik: with the theme?

19:05:19 <herlo> I'm specifically talking about the theme

19:05:23 <nirik> herlo: oh, sorry, no, not the theme...

19:05:27 <nirik> yeah, would be good to get that in too.

19:05:28 <herlo> nw

19:05:42 <nirik> ok, calling it a meeting as we are over.

19:05:45 <herlo> nirik: yeah, knew about the stickynotes review

19:05:54 <herlo> CodeBlock: awesome.

19:05:54 <nirik> Lets all continue over on #fedora-admin and #fedora-noc and #fedora-apps.

19:05:58 <herlo> indeed

19:05:58 <nirik> thanks for coming everyone!

19:06:02 <nirik> #endmeeting


_______________________________________________

infrastructure mailing list

infrastructure@lists.fedoraproject.org

https://admin.fedoraproject.org/mailman/listinfo/infrastructure


_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 04-20-2012, 04:40 PM
Kevin Fenzi
 
Default Summary/Minutes from today's Fedora infrastructure meeting (2012-04-19) (new time)

On Fri, 20 Apr 2012 09:23:23 +0530
Arijit Dutta <arijitdutta23@gmail.com> wrote:

> Dear Kevin ,
>
> Last night i was knid of on the fly travelling in a train to reach my
> home , so was getting disconnected frequently .

No problem.

> Thanks for all the resources you have shared earlier with me , i just
> gone through some starter documentations about git , puppet and
> nagios .The Fedora Infrastructure arch is also getting clearer to me
> day by day .Have not seen earlier such a vast network architechture
> where hundreds of hosts getting configurations from a central CM
> server(lockbox01) ,and also hundreds of people doing config changes
> and all are in sync with GIT .

yep. It works out pretty well most of the time.

> I was also trying to contact CodeBlock on IRC to check if i can help
> him in his wiki search project .Please let me know if any course of
> action you have already planned to me .

Yeah, I mentioned that to him. Hopefully your two schedules can sync up
at some point here.

kevin

_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 

Thread Tools




All times are GMT. The time now is 04:04 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org