FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Launchpad User

 
 
LinkBack Thread Tools
 
Old 11-21-2007, 10:41 AM
Carlos Perelló Marín
 
Default Call for testing new Launchpad Translations code performance

Hi,

As part of this development cycle in Launchpad, we are deploying a
performance improvement in Launchpad Translations that should fix most
of the timeouts that you have been experiencing.

Our current testing is showing a wonderful improvement. However we would
like to ask you for some input on this new performance improvement to be
sure it's as good as it should be.

The procedure to help us test this before we deploy it tomorrow is
simple: every time you get a timeout error while working with
launchpad.net please, visit the same url but adding 'staging' to it. So
for instance, translations.launchpad.net should be changed with
translations.staging.launchpad.net in the URL. Please try to do the same
translation and see whether it keeps giving you a timeout error or,
instead, if it's saved and allows you to keep working with the next
batch. If you get an OOPs error, please, send an email to
rosetta@launchpad.net with its code so we can see whether an easy fix
could be accomplished before deploying this change later this week.

Please note that all changes made in translations.staging.launchpad.net
will not be used at all and will be lost in a couple of days so please,
DO NOT use it as a substitute of launchpad.net.

Thanks for your help.


The Launchpad Translations Team.



--
launchpad-users mailing list
launchpad-users@lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/launchpad-users
 
Old 11-21-2007, 12:55 PM
"Philippe Verdy"
 
Default Call for testing new Launchpad Translations code performance

> -----Message d'origine-----
> De*: launchpad-users-bounces@lists.canonical.com [mailto:launchpad-users-
> bounces@lists.canonical.com] De la part de Carlos Perelló Marín
> Envoyé*: mercredi 21 novembre 2007 12:41
> À*: Ubuntu translators
> Cc*: launchpad-users
> Objet*: Call for testing new Launchpad Translations code performance
>
> Hi,
>
> As part of this development cycle in Launchpad, we are deploying a
> performance improvement in Launchpad Translations that should fix most
> of the timeouts that you have been experiencing.
>
> Our current testing is showing a wonderful improvement. However we would
> like to ask you for some input on this new performance improvement to be
> sure it's as good as it should be.
>
> The procedure to help us test this before we deploy it tomorrow is
> simple: every time you get a timeout error while working with
> launchpad.net please, visit the same url but adding 'staging' to it.


The security certificate for connecting on the "staging" server is invalid:
it tries to use an address for which the certificate was not emitted!

Note that the certificate allows ONLY:
Domain=launchpad.net.
Domain=*.launchpad.net.

The latest indicates that only a single HOST label is permitted within the
main domain launchpad.net, but the certificate does not allow these labels
to be subdomains (it would require an additional permission to permit the
DNS server that hosts the subdomain "staging.lauchpad.net.").

In other words, the site certificate tries to use the certificate emitted
for HOSTS in the domain "lauchpad.net." for hosts in a DISTINCT domain
"staging.launchpad.net." and that necessarily uses a distinct set of primary
DNS servers with distinct administration.

Note that the identity of the owner of "launchpad.net." is asserted from the
".net" top-level registry where it was registered; on the opposite
"staging.launchpad.net" is a private entry managed in your own DNS server
without any reference to the ".net." registry authority, so the source of
identity is different. This explains why the certificate is rejected.

For a certificate to be usable in a subdomain, you need to emit your own
certificate for the subdomain, and sign it using an authority hosted on your
main domain "launchpad.net": this new certificate becomes usable if it is
signed using the certificate emitted for the main site. This means that you
need to create a chained certificate.

More generally, chaining SOAs for delegating the management of security in a
subdomain requires a complex setting, plus some extensive verification by
your existing certificate SOA for the main domain. For this reason, managing
secure sites (HTTPS) in subdomains should be avoided, especially if the
subdomain and its hosts is not registered within the same DNS servers as
your main domain.

Another solution would be to avoid using a subdomain for your staging site:
replace the dot "." before ".staging.launchpad.net" by a hyphen "-", and
register the staging hosts directly with the "-staging" label extension
within your main domain.

So I suggest you register the "translations-staging.launchpad.net" host in
your main domain and use it instead of "translations.staging.launchpad.net"
(using an unsecure subdomain), because this won't require installing another
chained certificate (managed by your own local source of authority), if you
don't have a local source of authority (note that your site certificate does
not declare that the certificate is usable as a valid SOA for a subdomain,
as this would become a security risk by permitting you to host sites
considered "secured" but managed by others than you at launchpad.net: they
would create SOAs at will if it was allowed, so this kind of certificate is
expensive to get and verify by your existing certificate emitter)...




--
launchpad-users mailing list
launchpad-users@lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/launchpad-users
 
Old 11-21-2007, 04:43 PM
"Philippe Verdy"
 
Default Call for testing new Launchpad Translations code performance

Carlos Perelló Marín [mailto:carlos.perello@canonical.com]
> El mié, 21-11-2007 a las 15:22 +0100, Philippe Verdy escribió:
> > Could I request a missing item for translations: currently, there's
> > absolutely NO ways of entering non-breaking spaces in translations,
> despite
> > some translations require them.
>
> That's not completely true, we should expose that feature more but we do
> allow you to introduce non-breaking spaces:
>
> https://bugs.launchpad.net/rosetta/+bug/81281

No this occurs as well in IE7, not just Mozilla-based browsers. There's
apparently something wrong in the way the content is encoded/interpreted in
browsers, possibly because it is not correctly encoded to support
"alternate" spaces, or because of the encoding used to return the submitted
form.

I see only one way to solve it: don't send to browsers a pure text, send
escaped texts (using a trick like "uNNNN" encoding). Then:
* if Javascript is not supported or disabled, the users will see the text
using these escapes, and will submit the data using a "safe" encoding that
is preserved.
* If Javascript is enabled, your Javascript can PRESENT the decoded text to
the user, and process back the input from the user so that the text is shown
in WYSIWYG mode. There should then be some checkbox allowing the text input
form to show the decoded (WYSIWIG) or encoded (uNNNN) form.

Only the encoded form (with escapes) will be used to talk to the webserver.
Having the possibility to switch the content of an input form between the
two forms will help seeing the otherwise invisible characters that may cause
problems, or will help users entering these characters.

Note this:

Browsers don't let users enter for example a NBSP character, EVEN IF this
character is mapped on the keyboard, because this is generally mapped on
Ctrl+Space or Strl+Shift+Space and browsers are modifying the keymap and
disabling the Control key, or transforming the input as soon as it is
entered, even if this input comes from a copy/paste operation...).

Instead, they are assuming that the language entered will match with the
language used in the web page, and forcing the input to adopt the encoding
and character subsets used in the language determined from the web page
(they uses various tricks to do that, including not only the page encoding,
but also metadata sent in the HTTP headers, or some other elements in the
page, but generally they ignore the xml:lang or lang attribute set in the
web input elements, and this becomes even more complex with stylesheets in
actions).

The encodings interactins are really complex to handle over the HTTP
interface and with the interaction of HTML syntax. One way to prevent this
is to simplify the encoding at this interface, and then let some Javascript
make the work locally in the browser, to render the text back to normal
without the intermediate encoding used. Such loal Javascript will perform
the input decoding/encoding, validation and reformatting dynamically.

If Javascript is not enabled, users will still be able to interact with a
normal browser, but using only "safe" characters and an escaping syntax.
Note also that it is not clear what Launchpad is doing with translations
that contain text containing something that looks like literal HTML or
literal named character references, I've seen them changing after just
changing one character in the resource, despite it was not expected that
this would affect the encoding of the rest of the text. So when I download
back the translation results, I can see that they have been transformed
without any warning set to the user (no visible difference) when submitting
the data.

For this reason, I have reverted from using Launchpad: it cannot handle
international text properly and really breaks existing resources that were
working properly before and were already properly encoded. (For the projet
I'm interested in, the resources are to be converted into Java properties
files, and really contain Unicode text; Unicode being used as the central
encoding, even if it is then automatically converted into ASCII only using
Java-specific resource format for Unicode escapes in a ASCII only file):


very large files with thousands of resources that were completed and
reviewed by many persons since several years have suddenly been degraded to
become almost unusable, and everything needs to be rechecked manually (the
project counts more than 400,000 resources in various languages and scripts,
the whole set of texts occupying several megabytes if not compressed), and
the translation status was suddenly degraded so that many existing languages
were no longer usable and would have been removed from the distribution
(this included very common languages that had resources translated at nearly
100%, with just a few ones to maintain from time to time, and whose
translation level suddenly came to below 50% in the needed core resources).

Note also that in your site,

* Please don't let input box force their width so that they require
scrolling horizontally (even on a display with a large resolution), just
because the text to translate is a single paragraph (without any newline).
The text in that case is supposed to be displayed with automatic
line-wrapping, and your interface allows seeing the position where newlines
are effectively encoded in the resource text.

* the stylesheet is nearly unusable for proper text input: the text is
really TOO SMALL for entering anything else than just Basic Latin (i.e.
English and a few other languages, but most other languages use non ASCII
characters, and they are really hard to see and correct; for languages with
complex scripts or with subtle glyph distinctions, like Chinese, pointed
Arabic, Indian scripts, but also Korean using the regular Hangul alphabet,
it's almost impossible to read the text properly).

* the fonts specified are forced, but do not allow proper input of
international text. Please remove the font assignment at least in the input
box, or in the resource display (let the user specify its own visual font
from the browser settings, or at least make sure that the language being
worked on has its localized text styled using fonts that DO WORK with the
language):

Test a list of fonts working for each language/script pair, i.e. each one of
the supported locales, then mark the text displayed with a style specific to
that language, and then map a list of fonts for this language in a CSS
"font-family:", and make sure that the font are styled with a sufficient
point size !

And please let users grow the visual font size, both for the display of the
resources, and/or the input element.

My message is on topic, because this topic is speaking about a new Launchpad
Translations site, and it is testing some new features (primarily to speak
about the current performance problem, but this should also include the
problems of usability).




--
launchpad-users mailing list
launchpad-users@lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/launchpad-users
 
Old 11-21-2007, 09:19 PM
"Philippe Verdy"
 
Default Call for testing new Launchpad Translations code performance

Bruno Patri [mailto:bruno.patri@gmail.com]
> Envoyé*: mercredi 21 novembre 2007 19:37
> À*: verdy_p@wanadoo.fr
> Objet*: Re: Call for testing new Launchpad Translations code performance
>
> > No this occurs as well in IE7, not just Mozilla-based browsers. There's
> > apparently something wrong in the way the content is encoded/interpreted
> in
> > browsers, possibly because it is not correctly encoded to support
> > "alternate" spaces, or because of the encoding used to return the
> submitted
> > form.
>
> How many translators are using IE ? IE even version 7 is very buggy...

This is a completely out-of-topic (and in fact false) assertion. Any way, I
use both (and I have tried both) IE and Firefox (I don't use Opera).

All browsers have bugs (IE7 is not exempt, but Mozilla Gecko is not exempt
of them as well, as shown in the case of NBSP within Web input forms, that
Mozilla/Firefox is sending as regular space without notice! We really need
to enter "[nbsp]" and not regular NBSP characters, and when correcting a
resource, care must be taken to change back MANUALLY the existing NBSP into
"[nbsp]" before submitting, otherwise they are changed immediately into
normal SPACE's within the input form coming from the web server).

Really, the web-server should simply avoid sending any NBSP in non encoded
form, and in fact the server should escape all characters that are known to
cause problems (they can be decoded by the client using local Javascript,
and Javascript can be used as well during the data validation made by the
submit button so that these characters are reencoded into their escaped
form);

> > Note this:
> >
> > Browsers don't let users enter for example a NBSP character, EVEN IF
> this
> > character is mapped on the keyboard, because this is generally mapped on
> > Ctrl+Space or Strl+Shift+Space and browsers are modifying the keymap and
> > disabling the Control key, or transforming the input as soon as it is
> > entered, even if this input comes from a copy/paste operation...).
>
> The non-braking space is mapped on Alt+Shift+Space on my keyboard (oss
> fr keymap, it was Alt+Space on former keyboard layouts), and it works
> like a charm with Konqueror. I only need to type [nbsp] when I'm using
> a Gecko based browser.

Note: I don't want a non-BRAKING space, as such spaces are not supposed to
slow down my browser. I hope you spoke about non-BREAKING space (i.e. a
space that prohibits line wraps).

My keyboard layout also has a mapping for NBSP (in fact it has several ones,
including Ctrl+SPACE, or Ctrl+Shift+SPACE, or Ctrl+Alt+SPACE = AltGr+SPACE,
or Shift+Ctrl+Alt+SPACE = Shift+AltGr+SPACE), but none of the two browsers
allow entering it, because of the way it uses some internal key bindings
that are overriding the default keyboard layout: either IE rejects the
keystrokes and generates NO character, Firefox converts the keystroke
silently into a regular space within the HTML input form, and both use one
of these mappings to activate the system menu of the current window.

I may even try to enter Alt+0160, but Mozilla takes the digit 6 to perform
another action when it is pressed along with Alt, despite I am composing a
character, that it interrupts in the middle of the composition (and I have
not release the Alt key!)

Such bugs occur in both IE and Mozilla, that make false assumptions about
"unused" key bindings that they think are safe to use for their own purpose
(but both browsers are violating the ISO standard by not reusing keys that
are normally reserved for locale-specific keyboard layouts).

> > My message is on topic, because this topic is speaking about a new
> Launchpad
> > Translations site, and it is testing some new features (primarily to
> speak
> > about the current performance problem, but this should also include the
> > problems of usability).
>
> It's actually offtopic. Carlos was asking about future Launchpad
> improvements to solve the timeouts issues in translations.

Well, not so much. His message was mostly about the recent launch of a
staging server (still bogous, because its security certificate is completely
invalid, and does not pass the HTTPS validation performed in browsers, as it
attempts to reuse a certificate made only for the main server, and not
suitable for the new specific "staging.*" subdomain.)

He wanted to announce it so that we use his message to report problems. But
this should include all problems with the staging server, otherwise, the
test would not be so much useful.

As the new server was recently installed, these things must be corrected for
allowing further tests, otherwise the tests will not be enough complete, and
things will be released without being really tested.

For now, I have found a way to solve the timeout issues:
* I have stopped using the interface that allows translating entries by
groups of 10: it does not work, most of the time, due to timeouts (or too
much memory used) when performing its internal (certainly badly optimized)
SQL queries.
* I have stopped using any filters (such as trying to filter only the
resources with the "untranslated" or "need review" status). The filters are
actually taking too much resources on the server, certainly because of a bad
design of the database (incorrect indexes, or malformed SQL queries that
can't benefit of those indexes, or invalid selectivity statistics in some
index, so that the internal SQL optimizer is computing the wrong query
execution plan, and requires performing multiple full table scans requiring
too much server resources...)

In addition, the current design of the web pages and input forms has lots of
problems, as well as the data submission protocol which does not convert the
text with correct escaping to pass the HTTP and/or HTML limitations in
browsers.

My feeling is that escaping NBSP in the "[NBSP]" form is the wrong approach,
a more general escaping mechanism should have been designed (and it becomes
impossible to use the 6 literal "[NBSP]" characters in the translation, as
there's no way to escape the "[" character itself to avoid this
reinterpretation).




--
launchpad-users mailing list
launchpad-users@lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/launchpad-users
 
Old 11-21-2007, 10:31 PM
Matthew Paul Thomas
 
Default Call for testing new Launchpad Translations code performance

On Nov 22, 2007, at 11:19 AM, Philippe Verdy wrote:


Bruno Patri [mailto:bruno.patri@gmail.com]
...

It's actually offtopic. Carlos was asking about future Launchpad
improvements to solve the timeouts issues in translations.


Well, not so much. His message was mostly about the recent launch of a
staging server


That is incorrect: the staging server has existed for over a year.
Carlos asked specifically for help in "testing new Translations code
performance".



(still bogous, because its security certificate is completely
invalid, and does not pass the HTTPS validation performed in browsers,
as it attempts to reuse a certificate made only for the main server,
and not suitable for the new specific "staging.*" subdomain.)


I have seen this problem before, but I am unable to reproduce it now.
If you have reliable steps to reproduce the problem (i.e. steps that
work the second time someone follows them, not just the first time),
please report it as a bug
<https://bugs.launchpad.net/launchpad/+filebug>.


He wanted to announce it so that we use his message to report
problems. But this should include all problems with the staging
server, otherwise, the test would not be so much useful.


As the new server was recently installed, these things must be
corrected for allowing further tests, otherwise the tests will not be
enough complete, and things will be released without being really
tested.


The opposite is true. It will be helpful for people to verify the
performance improvements this week, but the other problems you
mentioned are completely unrelated and will be fixed later.


Cheers
--
Matthew Paul Thomas
http://mpt.net.nz/
--
launchpad-users mailing list
launchpad-users@lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/launchpad-users
 
Old 11-22-2007, 05:16 AM
"Philippe Verdy"
 
Default Call for testing new Launchpad Translations code performance

> De*: launchpad-users-bounces@lists.canonical.com [mailto:launchpad-users-
> > (still bogous, because its security certificate is completely
> > invalid, and does not pass the HTTPS validation performed in browsers,
> > as it attempts to reuse a certificate made only for the main server,
> > and not suitable for the new specific "staging.*" subdomain.)
>
> I have seen this problem before, but I am unable to reproduce it now.
> If you have reliable steps to reproduce the problem (i.e. steps that
> work the second time someone follows them, not just the first time),
> please report it as a bug
> <https://bugs.launchpad.net/launchpad/+filebug>.

Then try with IE7. The certificate is rejected ALWAYS, and the HTTPS URL
displays on red in the address bar if you accept to use it, with a warning
about it saying that the certificate was not issued for the subdomain, but
only for the main domain.

You can retry it again and again over all URLs, the certificate remains
invalid.
Only Mozilla/Firefox seems to accept it, but I think that it is wrong, and
should not reuse the certificate for a subdomain, for security reasons (if
not convinced, consider detailing the process of validating a certificate
and see how domain identity is asserted and verified. The certicifate has
been validated only according to the policy of the .net TLD, assuming that
the site identity is verified by the .net registry with the info provided by
you at the registrar. On the opposite, the subdomain is not authenticated
but is created by you only within your own DNS, without the registry being
able to verify anything.

So if your DNS get hacked, there's no other SOA available to assert that the
certificate is valid for the new domain. This is dangerous because you are
hosting websites (hosts) for other projects, and at anytime, some malicious
hosted project, created for a short time before you discover it, could be
used to perform "secured" authentication by reusing your site certificate.
This could turn your site into a malicious source of authentication for
performing transactions considered "secure", despite the subsite may be
malicious and not really authenticated. And you're placing your certificate
at risk of being exploited for other unintended use.

There exists exploits of such things, used by worms or phishers. My opinion
is that this is a severe security bug of Mozilla/Firefox that does not
respect the certificate contract.




--
launchpad-users mailing list
launchpad-users@lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/launchpad-users
 

Thread Tools




All times are GMT. The time now is 06:22 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org