FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Java

 
 
LinkBack Thread Tools
 
Old 03-08-2010, 07:09 PM
Sylvestre Ledru
 
Default Reasonable values for the -Xmx parameter ?

Le lundi 08 mars 2010 à 20:11 +0100, Vincent Fourmond a écrit :
> Hello,
>
> I'm wondering really what could be a decent value for -Xmx parameter.
> I used to think that the lowest parameter that seem to let the program
> run for every arch is good, but I don't think anymore this is a good idea:
>
> * the program might be clever enough to not allocate more memory than
> it can, while still being able to use significantly more to speedup (for
> instance through the use of caches)
> * the tightest the memory, the more often the GC has to run, which
> could lead to performance penalties.
>
> My question then is: is there a problem setting it to a value big
> enough (say, -Xmx1G) for a standard app ? (I'm thinking about freecol,
> that takes up quite a lot of memory) After all, most of the other
> programming languages don't limit memory by default, and the use of the
> ulimit shell builtin permits some fine tuning for this parameter (and
> others).
>
> What do you think ?
I had previous bad experiences on setting more memory than available. It
leaded to unexpected crashes and I had to come back to 256m.
It is a really pity that it is still mandatory to specify it...

Sylvestre



--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1268078941.6366.10940.camel@zlarin">http://lists.debian.org/1268078941.6366.10940.camel@zlarin
 
Old 03-08-2010, 07:34 PM
Vincent Fourmond
 
Default Reasonable values for the -Xmx parameter ?

Sylvestre Ledru wrote:
> Le lundi 08 mars 2010 à 20:11 +0100, Vincent Fourmond a écrit :
>> I'm wondering really what could be a decent value for -Xmx parameter.
>> I used to think that the lowest parameter that seem to let the program
>> run for every arch is good, but I don't think anymore this is a good idea:
>>
>> * the program might be clever enough to not allocate more memory than
>> it can, while still being able to use significantly more to speedup (for
>> instance through the use of caches)
>> * the tightest the memory, the more often the GC has to run, which
>> could lead to performance penalties.
>>
>> My question then is: is there a problem setting it to a value big
>> enough (say, -Xmx1G) for a standard app ? (I'm thinking about freecol,
>> that takes up quite a lot of memory) After all, most of the other
>> programming languages don't limit memory by default, and the use of the
>> ulimit shell builtin permits some fine tuning for this parameter (and
>> others).
>>
>> What do you think ?
> I had previous bad experiences on setting more memory than available. It
> leaded to unexpected crashes and I had to come back to 256m.
> It is a really pity that it is still mandatory to specify it...

All right...

Then, maybe I could add a function to java-wrappers that would find
out what is a 'good default' for that parameter, getting something more
than the memory required but still reasonably less than the memory
present on the machine ?

Would that be useful for anyone else than me ?

Cheers,

Vincent

--
The moon was high now, in a sky as black as a cup of coffee that
wasn't very black at all.
-- Terry Pratchet, Men at arms

Vincent, listening to Roads (Portishead)


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4B955F5C.3070004@gmail.com">http://lists.debian.org/4B955F5C.3070004@gmail.com
 
Old 03-08-2010, 08:20 PM
Pablo Duboue
 
Default Reasonable values for the -Xmx parameter ?

On Mon, Mar 8, 2010 at 3:34 PM, Vincent Fourmond <fourmond@gmail.com> wrote:
> Sylvestre Ledru wrote:
>> Le lundi 08 mars 2010 à 20:11 +0100, Vincent Fourmond a écrit :
>>> * I'm wondering really what could be a decent value for -Xmx parameter.
>>> I used to think that the lowest parameter that seem to let the program
>>> run for every arch is good, but I don't think anymore this is a good idea:
>>>
>>> * * the program might be clever enough to not allocate more memory than
>>> it can, while still being able to use significantly more to speedup (for
>>> instance through the use of caches)
>>> * * the tightest the memory, the more often the GC has to run, which
>>> could lead to performance penalties.
>>>
>>> * My question then is: is there a problem setting it to a value big
>>> enough (say, -Xmx1G) for a standard app ? (I'm thinking about freecol,
>>> that takes up quite a lot of memory) After all, most of the other
>>> programming languages don't limit memory by default, and the use of the
>>> ulimit shell builtin permits some fine tuning for this parameter (and
>>> others).
>>>
>>> * What do you think ?
>> I had previous bad experiences on setting more memory than available. It
>> leaded to unexpected crashes and I had to come back to 256m.
>> It is a really pity that it is still mandatory to specify it...

I never had that problem. The only problem I had was some old code
that was leaking file descriptors and that went undiagnosed because
with a smaller heap the garbage collected File instances had their OS
resources freed up correctly. Now, that was a bug and needed to be
fixed :-) But aside from that, more heap had always made for a
happier JVM, but my workloads are batch processing of large corpora
and the like.

> *All right...
>
> *Then, maybe I could add a function to java-wrappers that would find
> out what is a 'good default' for that parameter, getting something more
> than the memory required but still reasonably less than the memory
> present on the machine ?

If you want to go in that direction, you might want to add also
pointer compression in AMD64 (-XX:+UseCompressedOops), which makes a
huge difference with respect to minimum heap sizes in AMD64. Otherwise
what it's OK for 32-bits might be way too little for 64 bits.

http://wikis.sun.com/display/HotSpotInternals/CompressedOops
http://java.sun.com/javase/6/webnotes/6u14.html

> *Would that be useful for anyone else than me ?

I find it difficult to imagine how would you automatically determine
the heap size based on the ideas you sketched in your earlier
comments. If what you propose actually solves the problem, then you
can actually write a patch for upstream to do that within the JVM :-)
:-)

In general, I would vouch for having some functionality that allows
the maintainers specify something in the lines of "this is the bare
minimum heap" and "this is the recommended heap" and let the
java-wrapper use one or the other depending on physical RAM and/or
some global configuration parameter in /etc (which would be a fine
place to add the compressed pointers flag). That of course is just a
wish for an ideal world ;-)

P.


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: d5adecf91003081320y45ebf03cu1975d5e153724e42@mail. gmail.com">http://lists.debian.org/d5adecf91003081320y45ebf03cu1975d5e153724e42@mail. gmail.com
 
Old 03-08-2010, 10:11 PM
Vincent Fourmond
 
Default Reasonable values for the -Xmx parameter ?

Pablo Duboue wrote:
>> All right...
>>
>> Then, maybe I could add a function to java-wrappers that would find
>> out what is a 'good default' for that parameter, getting something more
>> than the memory required but still reasonably less than the memory
>> present on the machine ?
>
> If you want to go in that direction, you might want to add also
> pointer compression in AMD64 (-XX:+UseCompressedOops), which makes a
> huge difference with respect to minimum heap sizes in AMD64. Otherwise
> what it's OK for 32-bits might be way too little for 64 bits.
>
> http://wikis.sun.com/display/HotSpotInternals/CompressedOops
> http://java.sun.com/javase/6/webnotes/6u14.html
>
>> Would that be useful for anyone else than me ?
>
> I find it difficult to imagine how would you automatically determine
> the heap size based on the ideas you sketched in your earlier
> comments. If what you propose actually solves the problem, then you
> can actually write a patch for upstream to do that within the JVM :-)
> :-)

My idea would be something just like "all the memory minus a bit",
which would make java apps behave somehow like the other programming
languages with respect to memory allocation.

The reason why this isn't done by default in the JVM (and why it
definitely shouldn't be done that way) is that the JVM puts a strong
emphasis on security: setting a limit on the resources a web applet can
take seems like a good idea.

> In general, I would vouch for having some functionality that allows
> the maintainers specify something in the lines of "this is the bare
> minimum heap" and "this is the recommended heap" and let the
> java-wrapper use one or the other depending on physical RAM

That's something easy to do, and it seems like a good idea too... I
can implement that.

> and/or
> some global configuration parameter in /etc (which would be a fine
> place to add the compressed pointers flag). That of course is just a
> wish for an ideal world ;-)

This is a really interesting idea, since other flags can come in
useful too globally, such as graphic parameters (I'm thinking for
instance about Dsun.java2d.pmoffscreen=false that seems to significantly
improve the rendering speed on some hardware - never experienced myself).

Isn't there already a way to do that ? (configuration files setting
various parameters for the JVM ?)

Cheers,

Vincent

--
Vincent Fourmond, Debian Developer
http://vince-debian.blogspot.com/

Give a man a fish and you feed him for a day.
Give him a poisoned fish and you feed him for the rest of his life !
-- Slightly twisted chinese proverb

Vincent, listening to Learning To Live (Dream Theater)


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4B958418.9010309@debian.org">http://lists.debian.org/4B958418.9010309@debian.org
 
Old 03-09-2010, 08:33 AM
Andrew Haley
 
Default Reasonable values for the -Xmx parameter ?

On 03/08/2010 11:11 PM, Vincent Fourmond wrote:
> Pablo Duboue wrote:
>>> All right...
>>>
>>> Then, maybe I could add a function to java-wrappers that would find
>>> out what is a 'good default' for that parameter, getting something more
>>> than the memory required but still reasonably less than the memory
>>> present on the machine ?
>>
>> If you want to go in that direction, you might want to add also
>> pointer compression in AMD64 (-XX:+UseCompressedOops), which makes a
>> huge difference with respect to minimum heap sizes in AMD64. Otherwise
>> what it's OK for 32-bits might be way too little for 64 bits.
>>
>> http://wikis.sun.com/display/HotSpotInternals/CompressedOops
>> http://java.sun.com/javase/6/webnotes/6u14.html
>>
>>> Would that be useful for anyone else than me ?
>>
>> I find it difficult to imagine how would you automatically determine
>> the heap size based on the ideas you sketched in your earlier
>> comments. If what you propose actually solves the problem, then you
>> can actually write a patch for upstream to do that within the JVM :-)
>> :-)
>
> My idea would be something just like "all the memory minus a bit",
> which would make java apps behave somehow like the other programming
> languages with respect to memory allocation.
>
> The reason why this isn't done by default in the JVM (and why it
> definitely shouldn't be done that way) is that the JVM puts a strong
> emphasis on security: setting a limit on the resources a web applet can
> take seems like a good idea.

No, that's not the main reason. It's done that way because the heap is
contiguous, so has to be pre-allocated.

The key issue here is the setting of vm.overcommit_memory and
vm.overcommit_ratio. If you set the ratio large enough you can
allocate a large heap without any problems.

Andrew.


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4B9615E4.1090207@redhat.com">http://lists.debian.org/4B9615E4.1090207@redhat.com
 

Thread Tools




All times are GMT. The time now is 11:45 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org