FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 04-16-2008, 07:55 AM
Tiziano Müller
 
Default PostgreSQL Status

Since my blog doesn't get syndicated on planet.gentoo.org I'm posting the
entry here again, but it's probably a good idea anyway:

First I want to apologize for the current situation. I know we're lagging
behind and I know that bugs are piling up.
As a small excuse I can only say that my time is limited and PostgreSQL
isn't the only thing in Gentoo I (have to) maintain.

But there are good news:
Thanks to the help of Michael Krelin (also known as polyonymous on IRC) I
was able to commit a whole new set of ebuilds yesterday.
Now, you're probably going to ask why we just didn't go on with the current
ebuilds in the tree.

There are a couple of reasons:
a) dev-db/libpq is slotted but with collisions. Fixing this wasn't really an
option since slotting this was wrong from the beginning.
b) The split-up in two packages dev-db/{libpq,postgresql} was wrong too:
There are a couple of packages which do not only need libpq, but also one
of the other PostgreSQL libraries (like libecpg, libpgtypes, libecpg_compat
or libpgport).
c) Upgrading between major versions of PostgreSQL requires the DB admin to
bump the database using the old version, moving the database away and to
reload the dump into a new database cluster using the new version of
PostgreSQL. Having to take down the old server and purging the old version
of PostgreSQL before being able to try out the new one is more than
cumbersome. Therefore a slotted postgresql-server is needed to make the
upgrade easier.
d) A lot of things were broken in the old postgresql ebuilds as you can see
when going through the bug list and we needed to give them a general
overhaul.

What do the new ebuilds offer:
a) A split into dev-db/postgresql-{base,server,docs}. Now, I know that
splitting up packages isn't the Gentoo way. I know we could have done it
using USE flags but this approach gives more flexibility due to the current
way how binary packages are being generated and distributed.
b) New virtuals: virtual/postgresql-{base,server} to be able to install
pgcluster as your postgresql-base in a next step.
c) Slotting: It is now possible to have more than one major version of
PostgreSQL installed and running on the same machine.
d) A lot of other improvements, in detail, the following bugs will be fixed:
42894 Suggestion of use of slots for PostgreSQL
49864 dev-db/postgresql - pkg_config should allow passing optio...
55073 PostgreSQL client versus server installation
154620 PostgreSQL 8.1 server lacks instrumentation functions for...
158509 dev-db/libpq-8.0.9 fails configure with segmentation faul...
159223 postgresql threads USE flag required for ecpg
160178 dev-db/libpq : using (almost deprecated) gnuconfig_update
160181 dev-db/postgresql : using (almost deprecated) gnuconfig_u...
161803 dev-db/postgresql - /etc/conf.d/postgresql PG_OPTS variab...
161980 QA Notice: USE Flag 'kernel_linux' not in IUSE for dev-db...
177555 Postgresql/libpg upgrade from 8.1.8 to libpg-8.2.4 and po...
194350 >=dev-db/libpq-8.2.4 creates broken symlinks
194701 dev-db/postgresql - wrong elog into about configuration f...
198434 dev-db/postgresql - add ldap use flag
206725 dev-db/{libpq,postgresql} 8.2.7/8.3.1 version bump
206820 dev-db/postgresql default conf.d should not override max_...
209826 dev-db/postgresql - 2 issues with the current init script
210938 dev-db/postgresql - disable strict permission check on ss...
213699 dev-db/libpq-8.2.6 failed to configure w/ USE="threads"
214438 The dev-db/postgresql-8.2.6 ebuild does not respect confi...
215940 dev-db/postgresql provides bad init script for baselayout...


What you have to do to switch:
I'll put together more documentation as soon as the ebuilds get unmasked (in
1-2 weeks).
In general the only thing you have to then do is to uninstall dev-db/libpq
and dev-db/postgresql and install the same version of postgresql-base and
postgresql-server. No revdep-rebuild is needed.
For early adopters: It's best to wait until we changed the dependencies,
afterwards you can unmask the dev-db/postgresql-{docs,base,server}
packages...

What is needed from the ebuild developers side:
We have to change every dependency on dev-db/libpq to
virtual/postgresql-base and dev-db/postgresql to virtual/postgresql-server.
As soon as the old ebuilds are gone, we have to go through the ebuilds
depending on virtual/postgresql-server whether they can be built with
dev-db/postgresql-base only. Do not switch from dev-db/postgresql to
virtual/postgresql-base directly as long as you can't assure that your
package builds with libpq only.

Well, that's it basically. Don't hesitate to contact me in case of problems
or suggestions. You can answer to this post, leave comments on
http://dev-zero.ch/blog/ or send me a mail of course :-).
(But don't expect me to read forum entries since I simply do not have the
time to do that regularly.)

Again, many thanks to Michael Krelin and all the others who helped with
testing and sending patches.

Cheers,
Tiziano


--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-16-2008, 08:18 AM
Donnie Berkholz
 
Default PostgreSQL Status

On 09:55 Wed 16 Apr , Tiziano Müller wrote:
> What do the new ebuilds offer:
> a) A split into dev-db/postgresql-{base,server,docs}. Now, I know that
> splitting up packages isn't the Gentoo way. I know we could have done it
> using USE flags but this approach gives more flexibility due to the current
> way how binary packages are being generated and distributed.

I'd like to hear some more info on this point.

> In general the only thing you have to then do is to uninstall dev-db/libpq
> and dev-db/postgresql and install the same version of postgresql-base and
> postgresql-server. No revdep-rebuild is needed.
> For early adopters: It's best to wait until we changed the dependencies,
> afterwards you can unmask the dev-db/postgresql-{docs,base,server}
> packages...

People want `emerge postgresql` to do something. Otherwise it's not
always obvious which random hyphenated packages you're supposed to
install, and it's just like you're digging around some huge subpackage
list in Ubuntu or Fedora.

Thanks,
Donnie
--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-16-2008, 09:13 AM
Tiziano Müller
 
Default PostgreSQL Status

Donnie Berkholz wrote:

> On 09:55 Wed 16 Apr , Tiziano Müller wrote:
>> What do the new ebuilds offer:
>> a) A split into dev-db/postgresql-{base,server,docs}. Now, I know that
>> splitting up packages isn't the Gentoo way. I know we could have done it
>> using USE flags but this approach gives more flexibility due to the
>> current way how binary packages are being generated and distributed.
>
> I'd like to hear some more info on this point.

Consider this use case:
30 machines with one staging machine and binary deployment.
On 28 machines you want the libraries only, on one you also need the server
and on one you want the docs.
Easy done with (sanely) splitted packages.
Please note that the only additional package compared to the split
dev-db/{libpq,postgresql} is dev-db/postgresql-docs.

>
>> In general the only thing you have to then do is to uninstall
>> dev-db/libpq and dev-db/postgresql and install the same version of
>> postgresql-base and postgresql-server. No revdep-rebuild is needed.
>> For early adopters: It's best to wait until we changed the dependencies,
>> afterwards you can unmask the dev-db/postgresql-{docs,base,server}
>> packages...
>
> People want `emerge postgresql` to do something. Otherwise it's not
> always obvious which random hyphenated packages you're supposed to
> install, and it's just like you're digging around some huge subpackage
> list in Ubuntu or Fedora.

Well, we can re-introduce a virtual/postgresql (or dev-db/postgresql) after
the old ebuilds are gone.

Cheers,
Tiziano

--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-16-2008, 02:46 PM
Carsten Lohrke
 
Default PostgreSQL Status

> c) Upgrading between major versions of PostgreSQL requires the DB admin to
> bump the database using the old version, moving the database away and to
> reload the dump into a new database cluster using the new version of
> PostgreSQL. Having to take down the old server and purging the old version
> of PostgreSQL before being able to try out the new one is more than
> cumbersome. Therefore a slotted postgresql-server is needed to make the
> upgrade easier.

As I read upstream's documentation¹, this is incorrect:

# It is recommended that you use the pg_dump and pg_dumpall programs from the
# newer version of PostgreSQL, to take advantage of any enhancements that may
# have been made in these programs. Current releases of the dump programs can
# read data from any server version back to 7.0.

This has also been pointed out in a bug report, duped by some overzealous bug
wrangler a few months ago.

> What do the new ebuilds offer:
> a) A split into dev-db/postgresql-{base,server,docs}. Now, I know that
> splitting up packages isn't the Gentoo way. I know we could have done it
> using USE flags but this approach gives more flexibility due to the current
> way how binary packages are being generated and distributed.

I wouldn't say so. Needlessly growing the forest of odd named, badly
documented and sometimes needlessly added use flags is something I'm not fond
of, looking at the development of Gentoo.


Thanks to everyone taking care of our PostgreSQL packages.


Carsten


[1] http://www.postgresql.org/docs/8.2/interactive/migration.html
 
Old 04-16-2008, 04:16 PM
Tiziano Müller
 
Default PostgreSQL Status

Carsten Lohrke wrote:

>> c) Upgrading between major versions of PostgreSQL requires the DB admin
>> to bump the database using the old version, moving the database away and
>> to reload the dump into a new database cluster using the new version of
>> PostgreSQL. Having to take down the old server and purging the old
>> version of PostgreSQL before being able to try out the new one is more
>> than cumbersome. Therefore a slotted postgresql-server is needed to make
>> the upgrade easier.
>
> As I read upstream's documentation¹, this is incorrect:
>
> # It is recommended that you use the pg_dump and pg_dumpall programs from
> # the newer version of PostgreSQL, to take advantage of any enhancements
> # that may have been made in these programs. Current releases of the dump
> # programs can read data from any server version back to 7.0.

While the dump command can read clusters created by an older version it is
still necessary to dump and reload your data on version bumps between major
versions as written in
http://www.postgresql.org/docs/8.3/static/install-upgrading.html:
[...]
The internal data storage format typically changes in every major release of
PostgreSQL. Therefore, if you are upgrading an existing installation that
does not have a version number of "8.3.x", you must back up and restore
your data. If you are upgrading from PostgreSQL "8.3.x", the new version
can use your current data files so you should skip the backup and restore
steps below because they are unnecessary.
[...]

Cheers,
Tiziano


--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-17-2008, 08:59 AM
Luca Barbato
 
Default PostgreSQL Status

Tiziano Müller wrote:

What do the new ebuilds offer:
a) A split into dev-db/postgresql-{base,server,docs}.


WRONG we aren't debian.

we have a nice use for documentation...


Now, I know that
splitting up packages isn't the Gentoo way. I know we could have done it
using USE flags but this approach gives more flexibility due to the current
way how binary packages are being generated and distributed.


It gives an annoyance please reconsider.


b) New virtuals: virtual/postgresql-{base,server} to be able to install
pgcluster as your postgresql-base in a next step.


see before.


c) Slotting: It is now possible to have more than one major version of
PostgreSQL installed and running on the same machine.


Great =)


d) A lot of other improvements, in detail, the following bugs will be fixed:


Wonderful =)

lu

--

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-17-2008, 09:58 AM
Tiziano Müller
 
Default PostgreSQL Status

Luca Barbato wrote:

> Tiziano Müller wrote:
>> What do the new ebuilds offer:
>> a) A split into dev-db/postgresql-{base,server,docs}.
>
> WRONG we aren't debian.
This is why we decided not to split out headers, clients and contrib.

>
>> Now, I know that
>> splitting up packages isn't the Gentoo way. I know we could have done it
>> using USE flags but this approach gives more flexibility due to the
>> current way how binary packages are being generated and distributed.
>
> It gives an annoyance please reconsider.
Done that. Won't change. See my answer to dberkholz's message.


--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-17-2008, 04:30 PM
Enrico Weigelt
 
Default PostgreSQL Status

* Tiziano Müller <dev-zero@gentoo.org> schrieb:

Hi,

> Since my blog doesn't get syndicated on planet.gentoo.org I'm posting the
> entry here again, but it's probably a good idea anyway:

Actually, I'd clearly prefer lists like this one for those discussions.
Blogs are okay for writing journal articles and let people comment them,
but bad for real discussions, IMHO.

> First I want to apologize for the current situation. I know we're lagging
> behind and I know that bugs are piling up.
> As a small excuse I can only say that my time is limited and PostgreSQL
> isn't the only thing in Gentoo I (have to) maintain.

Well, my time is limited too (hmm, an contract for Gentoo development
would be fine ;-)), otherwise I'd already reworked the pg ebuilds ;-o

> There are a couple of reasons:
> a) dev-db/libpq is slotted but with collisions. Fixing this wasn't really an
> option since slotting this was wrong from the beginning.

ACK. It should be done by *real* MVCC. But I doubt that portage is
actually capable of this yet. For the vast majority of the cases I
see slots just as a dirty hack ;-p

The main problem for MVCC (and also what often makes revdep-rebuild
necessary) is the point that source and binary packages have to be
completely different: the mapping from source to binary happens at
only build time and all the rest of the dependency handling derives
from that. I'm currently implementing this approach in my own build/
packaging system, but it's going to tricky.

> b) The split-up in two packages dev-db/{libpq,postgresql} was wrong too:
> There are a couple of packages which do not only need libpq, but also one
> of the other PostgreSQL libraries (like libecpg, libpgtypes, libecpg_compat
> or libpgport).

NAK! Each of these libraries are their own entities. Some clients even may
depend on one of them, but not libpq. So they clearly belong to their
own packages.

I do *NOT* want *ANY* unneeded stuff in my systems. You'll force me to fork.

> c) Upgrading between major versions of PostgreSQL requires the DB admin to
> bump the database using the old version, moving the database away and to
> reload the dump into a new database cluster using the new version of
> PostgreSQL. Having to take down the old server and purging the old version
> of PostgreSQL before being able to try out the new one is more than
> cumbersome. Therefore a slotted postgresql-server is needed to make the
> upgrade easier.

That's the point where we need *real* MVCC. An virtual package could
coordinate the update process (eg. pulling in new versions and providing
update utils, maybe with some additional refcounting and automatic cleanup
based on that)

Yes, MVCC is not trivial ;-P

> What do the new ebuilds offer:
> a) A split into dev-db/postgresql-{base,server,docs}. Now, I know that
> splitting up packages isn't the Gentoo way. I know we could have done it
> using USE flags but this approach gives more flexibility due to the current
> way how binary packages are being generated and distributed.

I actually think packages should be split whenever suitable, useflags should
only be used for *conditional* building (where features are switched that
do *not* reflect separate modules) or for virtual (frontend) packages.

> b) New virtuals: virtual/postgresql-{base,server} to be able to install
> pgcluster as your postgresql-base in a next step.

Yes, frontend virtuals for convenience are an good idea for many users.
I, personally, probably won't use them, since my setups would be too
complex for them. Other folks with simpler setups might be perfectly
fine with them.

> c) Slotting: It is now possible to have more than one major version of
> PostgreSQL installed and running on the same machine.

Great. This makes smooth update processes easier (reduces the need of
custom ebuilds). But I, pesonally, prefer *separate* packages instead
of slots.

> 159223 postgresql threads USE flag required for ecpg

BTW: does portage meanwhile handle feature dependencies ?
It's always a big mess when an whole install/update fails in the middle
just because some package wants some other built with certain useflag

> 161980 QA Notice: USE Flag 'kernel_linux' not in IUSE for dev-db...

Naive question: why does this useflag appear in pg ?


cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-17-2008, 04:40 PM
Enrico Weigelt
 
Default PostgreSQL Status

* Luca Barbato <lu_zero@gentoo.org> schrieb:
> Tiziano Müller wrote:
> >What do the new ebuilds offer:
> >a) A split into dev-db/postgresql-{base,server,docs}.
>
> WRONG we aren't debian.

It's bad, just because Debian does it ?!
Sounds quite religions to me. I don't like religions interfering with
technical designs.


cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
--
gentoo-dev@lists.gentoo.org mailing list
 
Old 04-17-2008, 04:42 PM
Enrico Weigelt
 
Default PostgreSQL Status

* Tiziano Müller <dev-zero@gentoo.org> schrieb:

> > WRONG we aren't debian.
> This is why we decided not to split out headers, clients and contrib.

Actually, I'd like to see them all split out. But this sooner or
later requires the upstream (or an intermediate layer, like OSS-QM)
to support this - individual distros shouldn't do this by themselves
(too much unnecessary duplicate work).


cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service - http://www.metux.de/
---------------------------------------------------------------------
Please visit the OpenSource QM Taskforce:
http://wiki.metux.de/public/OpenSource_QM_Taskforce
Patches / Fixes for a lot dozens of packages in dozens of versions:
http://patches.metux.de/
---------------------------------------------------------------------
--
gentoo-dev@lists.gentoo.org mailing list
 

Thread Tools




All times are GMT. The time now is 01:17 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org