FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 08-20-2008, 02:06 PM
Loc Minier
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

clone 481134 -1
reassign 481134 xpdf-japanese
retitle 481134 xpdf-japanese Should register fonts in fontconfig and/or defoma and/or provide poppler specific symlinks
retitle -1 poppler doesn't support xpdf config anymore
severity -1 important
stop

Heya,

Executive summary: downgrading RC-ness against poppler, the burden of
the fix is probably on xpdf-japanese to start with. Request for
comments/help on fontconfig/defoma topics.

On Wed, May 14, 2008, Junichi Uekawa wrote:
> xpdf-japanese installs CMAP files in
> /usr/share/fonts/cmap/adobe-japan1 etc, but libpoppler looks at
> /usr/share/poppler.
> I need to install the following symlinks in order to use the adobe
> CMAP files.
> $ ls -l /usr/share/poppler/cidToUnicode/Adobe-Japan1
> lrwxrwxrwx 1 root root 34 2008-01-18 19:53 Adobe-Japan1 -> /usr/share/fonts/cmap/adobe-japan1
> $ ls -l /usr/share/poppler/cMap/
> lrwxrwxrwx 1 root root 50 2008-01-18 19:53 /usr/share/poppler/cidToUnicode/Adobe-Japan1 -> /usr/share/xpdf/japanese/Adobe-Japan1.cidToUnicode

(the symlinks are flipped)

I confirmed that adding similar symlinks allows display of Japanese PDF
files with evince (poppler). I don't think adding the symlinks in
poppler is a good idea.

What happens here is that xpdf-japanese installs a xpdfrc config file
snippet which will properly configure unicodeMap and cMapDir with
various files for xpdf, see /etc/xpdf/xpdfrc-japanese:
#----- begin Japanese support package (2004-jul-27)
cidToUnicode Adobe-Japan1 /usr/share/xpdf/japanese/Adobe-Japan1.cidToUnicode
unicodeMap ISO-2022-JP /usr/share/xpdf/japanese/ISO-2022-JP.unicodeMap
unicodeMap EUC-JP /usr/share/xpdf/japanese/EUC-JP.unicodeMap
unicodeMap Shift-JIS /usr/share/xpdf/japanese/Shift-JIS.unicodeMap
cMapDir Adobe-Japan1 /usr/share/fonts/cmap/adobe-japan1
toUnicodeDir /usr/share/fonts/cmap/adobe-japan1
...

poppler probably used to parse xpdf's config as it was forked from
xpdf.

Nowadays, poppler has these /usr/share/poppler/cidToUnicode and /cMap
dirs (along others) to allow for these mappings; these rely on the
filenames for the mappings to work, so e.g. if poppler needs
Adobe-Japan1's mapping, it will look for Adobe-Japan1 in there. The
actual data for the Adobe-Japan1 font is available in
/usr/share/fonts/cmap/adobe-japan1 and
/usr/share/xpdf/japanese/Adobe-Japan1.cidToUnicode, but there isn't any
place with the data available with the expected file names
"Adobe-Japan1".

My understanding is that xpdf-japanese is very xpdf specific; it does
install some files below /usr/share/fonts/cmap, but I'm not sure this
is used by anything except xpdf. It only calls update-xpdfrc in
postinst.

As I see it, the modern way of dealing with fonts is to use fontconfig
which poppler supports, but I'm not sure fontconfig provides support
for cMap and cidToUnicode information. A Debian alternative for
dealing with fonts is to use defoma which allows for package specific
scripts to be run when fonts are registered/removed via defoma. Some
defoma scripts mention cmap, but I saw no cidToUnicode references there
either.

So because the data is not currently available in any place I could
find / think of or isn't in the proper format, I see the following
solutions to fix this bug:
a) make poppler parse xpdf's config again; I'm not too hot on this as
it makes poppler leak that it's xpdf based, diverges from upstream,
and isn't a sustainable long-term option; I'm not even sure it's
still possible
b) make xpdf-japanese register fonts properly in fontconfig; I don't
have enough fontcp,fog foo to understand whether fontconfig can
carry the relevant information/mappings though; help to write a
/etc/fonts/conf.d/*xpdf-japanese*.conf is welcome; it's not clear to
me whether such mappings are reaching too far for fontconfig's goal;
this might also require poppler patching to lookup the relevant
fontconfig data if poppler doesn't use this part of fontconfig, or
if we had new API for this stuff to fontconfig; probably a good
solution for other packages in the same solution as well
c) make xpdf-japanese register fonts with defoma and make poppler run a
defoma script to generate appropriate symlinks to the cidToUnicode
and cMap files/dirs; I don't have enough defoma foo to understand
whether it's already possible in defoma, but I would expect it is;
this has the benefit of making it easy to handle other packages in
the same case; help welcome to answer the open issues here
d) make xpdf-japanese generate /usr/share/poppler/{cMap,cidToUnicode}/*
symlinks; probably dead easy, but makes the solution poppler
specific: other fontconfig or defoma using apps wont benefit from
the fonts/mappings; also leaves open the question of integration:
should it be a new poppler-japanese package? who should be pulling
this package?

For the long term, I'd recommend either b), or c) if b) isn't possible;
for lenny, either the long term approach or the d) approach would work,
but d) would only solve the problem for this particular package.

Because b, c), and d) will almost certainly require changes to
xpdf-japanese, I'm reassigning the RC counterpart of this bug to this
package. I'm keeping an important bug open against poppler because it
can be considered a regression of poppler, but I don't think it should
be considered RC for poppler because:
* xpdf-japanese is non-free while poppler is free; perhaps this is a
fallacy, but I don't think support for a non-free package should
affect the Debian release; if this is considered >= serious
severity, it shouldn't be release-blocking
* it's an upstream design decision that poppler doesn't support xpdf
config/data and poppler doesn't aim to be compatible with xpdf's
config/data: xpdf-japanese needs to grow compatibility with poppler
instead


Cheers,
--
Loc Minier


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-21-2008, 04:18 AM
Hamish Moffatt
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

On Wed, Aug 20, 2008 at 04:06:40PM +0200, Loc Minier wrote:
> For the long term, I'd recommend either b), or c) if b) isn't possible;
> for lenny, either the long term approach or the d) approach would work,
> but d) would only solve the problem for this particular package.
>
> Because b, c), and d) will almost certainly require changes to
> xpdf-japanese, I'm reassigning the RC counterpart of this bug to this
> package. I'm keeping an important bug open against poppler because it
> can be considered a regression of poppler, but I don't think it should
> be considered RC for poppler because:
> * xpdf-japanese is non-free while poppler is free; perhaps this is a
> fallacy, but I don't think support for a non-free package should
> affect the Debian release; if this is considered >= serious
> severity, it shouldn't be release-blocking
> * it's an upstream design decision that poppler doesn't support xpdf
> config/data and poppler doesn't aim to be compatible with xpdf's
> config/data: xpdf-japanese needs to grow compatibility with poppler
> instead

Patches for xpdf-japanese would be most welcome. I'm not familiar with
the details of fontconfig, defoma etc.

Maybe the relevant data in xpdf-japanese should be moved into
cmap-adobe-japan1 instead. Probably xpdf-japanese's additional CMaps
should move too; I thought I submitted a bug about this years ago but I
don't see it now.

Hamish
--
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-21-2008, 04:42 AM
Junichi Uekawa
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

Hi,

I'm a bit worried that this needs to be fixed for
xpdf-chinese-simplified
xpdf-chinese-traditional
xpdf-korean
xpdf-japanese

packages. The fix would probably be simple. It should probably be
hardlinks rather than symlinks considering dpkg behavior wrt symlinked
dirs.

I'd like this fix included in lenny so that we don't see a regression
in Debian for Japanese users.


There was a package in the NEW queue 'poppler-data' which should
really have fixed this bug.
http://ftp-master.debian.org/new/poppler-data_0.2.0-1.html

That going into Debian sid would fix this bug also, but not quite sure
if it's reasonable to expect a new package to enter Debian lenny.


regards,
junichi


> clone 481134 -1
> reassign 481134 xpdf-japanese
> retitle 481134 xpdf-japanese Should register fonts in fontconfig and/or defoma and/or provide poppler specific symlinks
> retitle -1 poppler doesn't support xpdf config anymore
> severity -1 important
> stop
>
> Heya,
>
> Executive summary: downgrading RC-ness against poppler, the burden of
> the fix is probably on xpdf-japanese to start with. Request for
> comments/help on fontconfig/defoma topics.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-21-2008, 05:18 AM
Hamish Moffatt
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

On Wed, Aug 20, 2008 at 09:42:21PM -0700, Junichi Uekawa wrote:
> I'm a bit worried that this needs to be fixed for
> xpdf-chinese-simplified
> xpdf-chinese-traditional
> xpdf-korean
> xpdf-japanese
>
> packages. The fix would probably be simple. It should probably be
> hardlinks rather than symlinks considering dpkg behavior wrt symlinked
> dirs.

You can't hardlink directories though, so we might as well symlink the
individual files.

> I'd like this fix included in lenny so that we don't see a regression
> in Debian for Japanese users.
>
>
> There was a package in the NEW queue 'poppler-data' which should
> really have fixed this bug.
> http://ftp-master.debian.org/new/poppler-data_0.2.0-1.html
>
> That going into Debian sid would fix this bug also, but not quite sure
> if it's reasonable to expect a new package to enter Debian lenny.

I kinda think it's in poppler's domain to fix this; logically
xpdf-japanese (etc) exists only to enhance xpdf. poppler-data appears to
be a good solution.

thanks,
Hamish
--
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-22-2008, 02:51 PM
Junichi Uekawa
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

Hi,


> > I'm a bit worried that this needs to be fixed for
> > xpdf-chinese-simplified
> > xpdf-chinese-traditional
> > xpdf-korean
> > xpdf-japanese
> >
> > packages. The fix would probably be simple. It should probably be
> > hardlinks rather than symlinks considering dpkg behavior wrt symlinked
> > dirs.
>
> You can't hardlink directories though, so we might as well symlink the
> individual files.
>
> > I'd like this fix included in lenny so that we don't see a regression
> > in Debian for Japanese users.
> >
> >
> > There was a package in the NEW queue 'poppler-data' which should
> > really have fixed this bug.
> > http://ftp-master.debian.org/new/poppler-data_0.2.0-1.html
> >
> > That going into Debian sid would fix this bug also, but not quite sure
> > if it's reasonable to expect a new package to enter Debian lenny.
>
> I kinda think it's in poppler's domain to fix this; logically
> xpdf-japanese (etc) exists only to enhance xpdf. poppler-data appears to
> be a good solution.

1. poppler was theoretically self-contained, poppler-data was REJECTed
once and not yet part of non-free as of today.

2. Japanese users upgrading from etch would have xpdf-japanese
installed because evince (poppler) needed and used xpdf-japanese, and
natural upgrade path would be xpdf-japanese supporting poppler.

These factors make adding support in poppler somewhat reasonable.


diff -ur xpdf-japanese-20040727-orig/debian/changelog xpdf-japanese-20040727/debian/changelog
--- xpdf-japanese-20040727-orig/debian/changelog 2008-08-22 07:13:09.000000000 -0700
+++ xpdf-japanese-20040727/debian/changelog 2008-08-22 07:24:30.000000000 -0700
@@ -1,3 +1,10 @@
+xpdf-japanese (1:20040727-1.1) unstable; urgency=low
+
+ * add symlinks to facilitate poppler transition for lenny.
+ In the longer term, users should migrate to poppler-data.
+
+ -- Junichi Uekawa <dancer@debian.org> Fri, 22 Aug 2008 07:24:30 -0700
+
xpdf-japanese (1:20040727-1) unstable; urgency=low

* New upstream version
diff -ur xpdf-japanese-20040727-orig/debian/rules xpdf-japanese-20040727/debian/rules
--- xpdf-japanese-20040727-orig/debian/rules 2008-08-22 07:13:09.000000000 -0700
+++ xpdf-japanese-20040727/debian/rules 2008-08-22 07:45:47.000000000 -0700
@@ -2,6 +2,9 @@
TARGET=japanese
CMAP=/usr/share/fonts/cmap/adobe-japan1

+# datadir for poppler
+POPPLER_DATA=/usr/share/poppler
+
POSTRM=debian/xpdf-$(TARGET).postrm
PREINST=debian/xpdf-$(TARGET).preinst

@@ -29,6 +32,11 @@
cp -r *.cidToUnicode *.unicodeMap debian/xpdf-$(TARGET)/usr/share/xpdf/$(TARGET)/
cp add-to-xpdfrc debian/xpdf-$(TARGET)/etc/xpdf/xpdfrc-$(TARGET)

+ # install poppler symlinks
+ mkdir -p debian/xpdf-$(TARGET)${POPPLER_DATA}/cMap/
+ dh_link ${CMAP} ${POPPLER_DATA}/cMap/Adobe-Japan1
+ dh_link /usr/share/xpdf/japanese/Adobe-Japan1.cidToUnicode ${POPPLER_DATA}/cidToUnicode/Adobe-Japan1/Adobe-Japan1.cidToUnicode
+
cp debian/preinst-template $(PREINST)
cp debian/postrm-template $(POSTRM)



regards,
junichi
--
dancer@{netfort.gr.jp,debian.org}


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-22-2008, 05:33 PM
Osamu Aoki
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

On Fri, Aug 22, 2008 at 11:26:53PM +0800, Deng Xiyue wrote:
> 在 2008-08-22五的 07:51 -0700,Junichi Uekawa写道:
> > 1. poppler was theoretically self-contained, poppler-data was REJECTed
> > once and not yet part of non-free as of today.
>
> Which is now sitting in NEW queue again for approval[1]. IMHO, It
> doesn't worth to put effort in something to solve problem that is
> already handled by others, which makes it unnecessary once accepted.
> Better to push ftp-masters to get it reviewed and ask for a freeze
> exception ASAP.
>
> [1] http://ftp-master.debian.org/new/poppler-data_0.2.0-1.html

Time is short...

* Does Junichi's fix cause any side effects?
If no, then just do it and ask release team to give hint to include
this in release. (We all know release team acts fast.)

* Do we know how to ask FTP master? My impression is they ar too busy
with many things so they may not be able to act soon enough. Why take
chance.

(Deng: did you send request? Arguing here for FTP mater seems not so
effective.)

Osamu


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-23-2008, 01:39 AM
Hamish Moffatt
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

Hi,

On Fri, Aug 22, 2008 at 07:51:36AM -0700, Junichi Uekawa wrote:
> > > I'm a bit worried that this needs to be fixed for
> > > xpdf-chinese-simplified
> > > xpdf-chinese-traditional
> > > xpdf-korean
> > > xpdf-japanese
> > >
> > > packages. The fix would probably be simple. It should probably be
> > > hardlinks rather than symlinks considering dpkg behavior wrt symlinked
> > > dirs.
> >
> > You can't hardlink directories though, so we might as well symlink the
> > individual files.
> >
> > > I'd like this fix included in lenny so that we don't see a regression
> > > in Debian for Japanese users.
> > >
> > >
> > > There was a package in the NEW queue 'poppler-data' which should
> > > really have fixed this bug.
> > > http://ftp-master.debian.org/new/poppler-data_0.2.0-1.html
> > >
> > > That going into Debian sid would fix this bug also, but not quite sure
> > > if it's reasonable to expect a new package to enter Debian lenny.
> >
> > I kinda think it's in poppler's domain to fix this; logically
> > xpdf-japanese (etc) exists only to enhance xpdf. poppler-data appears to
> > be a good solution.
>
> 1. poppler was theoretically self-contained, poppler-data was REJECTed
> once and not yet part of non-free as of today.
>
> 2. Japanese users upgrading from etch would have xpdf-japanese
> installed because evince (poppler) needed and used xpdf-japanese, and
> natural upgrade path would be xpdf-japanese supporting poppler.
>
> These factors make adding support in poppler somewhat reasonable.

If the release managers would approve an update to xpdf-japanese I am
happy to upload a new version including your patch, or for you to NMU
the package.


Hamish
--
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-23-2008, 06:40 AM
Deng Xiyue
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

在 2008-08-23*的 11:39 +1000,Hamish Moffatt写道:
> Hi,
>
> On Fri, Aug 22, 2008 at 07:51:36AM -0700, Junichi Uekawa wrote:
> > > > I'm a bit worried that this needs to be fixed for
> > > > xpdf-chinese-simplified
> > > > xpdf-chinese-traditional
> > > > xpdf-korean
> > > > xpdf-japanese
> > > >
> > > > packages. The fix would probably be simple. It should probably be
> > > > hardlinks rather than symlinks considering dpkg behavior wrt symlinked
> > > > dirs.
> > >
> > > You can't hardlink directories though, so we might as well symlink the
> > > individual files.
> > >
> > > > I'd like this fix included in lenny so that we don't see a regression
> > > > in Debian for Japanese users.
> > > >
> > > >
> > > > There was a package in the NEW queue 'poppler-data' which should
> > > > really have fixed this bug.
> > > > http://ftp-master.debian.org/new/poppler-data_0.2.0-1.html
> > > >
> > > > That going into Debian sid would fix this bug also, but not quite sure
> > > > if it's reasonable to expect a new package to enter Debian lenny.
> > >
> > > I kinda think it's in poppler's domain to fix this; logically
> > > xpdf-japanese (etc) exists only to enhance xpdf. poppler-data appears to
> > > be a good solution.
> >
> > 1. poppler was theoretically self-contained, poppler-data was REJECTed
> > once and not yet part of non-free as of today.
> >
> > 2. Japanese users upgrading from etch would have xpdf-japanese
> > installed because evince (poppler) needed and used xpdf-japanese, and
> > natural upgrade path would be xpdf-japanese supporting poppler.
> >
> > These factors make adding support in poppler somewhat reasonable.
>
> If the release managers would approve an update to xpdf-japanese I am
> happy to upload a new version including your patch, or for you to NMU
> the package.
>

Which means the same should be done in
xpdf-{chinese-{simplified,tranditional},korean} as well, and all will be
unnecessary once poppler-data get accepted. I kind of think that's not
a good way to deal with it. But well, just my 2 cents.

--
Regards,
Deng Xiyue, a.k.a. manphiz



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 08-23-2008, 07:10 AM
Deng Xiyue
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

在 2008-08-23*的 02:33 +0900,Osamu Aoki写道:
> On Fri, Aug 22, 2008 at 11:26:53PM +0800, Deng Xiyue wrote:
> > 在 2008-08-22五的 07:51 -0700,Junichi Uekawa写道:
> > > 1. poppler was theoretically self-contained, poppler-data was REJECTed
> > > once and not yet part of non-free as of today.
> >
> > Which is now sitting in NEW queue again for approval[1]. IMHO, It
> > doesn't worth to put effort in something to solve problem that is
> > already handled by others, which makes it unnecessary once accepted.
> > Better to push ftp-masters to get it reviewed and ask for a freeze
> > exception ASAP.
> >
> > [1] http://ftp-master.debian.org/new/poppler-data_0.2.0-1.html
>
> Time is short...
>
> * Does Junichi's fix cause any side effects?
> If no, then just do it and ask release team to give hint to include
> this in release. (We all know release team acts fast.)
>
> * Do we know how to ask FTP master? My impression is they ar too busy
> with many things so they may not be able to act soon enough. Why take
> chance.
>
> (Deng: did you send request? Arguing here for FTP mater seems not so
> effective.)
>
>
> Osamu
>


Don't know whether it is orthodox, but as the urgency of the issue, I
got the the mail addres of ftp-masters from pabs on IRC (thanks pabs)
and added to CC.

Ftp-masters, sorry to break the protocol, but due to the current
situation, it'll be very helpful if the poppler-data package sitting in
NEW will be reviewed soonish so that it can be decided whether the NMU
of xpdf-CJK packages should be done. Obviously poppler-data is the best
way to solve the issue. Thanks very much.

--
Regards,
Deng Xiyue, a.k.a. manphiz
 
Old 08-23-2008, 08:09 AM
Hamish Moffatt
 
Default Bug#481134: libpoppler does not use cmap files from xpdf-{japanese,...}, and fails to parse Japanese PDF files.

On Sat, Aug 23, 2008 at 03:10:22PM +0800, Deng Xiyue wrote:
> Ftp-masters, sorry to break the protocol, but due to the current
> situation, it'll be very helpful if the poppler-data package sitting in
> NEW will be reviewed soonish so that it can be decided whether the NMU
> of xpdf-CJK packages should be done. Obviously poppler-data is the best
> way to solve the issue. Thanks very much.

You also need the release managers (debian-release@lists.debian.org) to
accept the new package into lenny, and they appear reluctant to allow
new packages in general.

Hamish
--
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 04:02 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org