On 08/19/2010 02:15 PM, Michael Hennebry wrote:
> On Thu, 19 Aug 2010, JD wrote:
>
>> Problem comes as Michael explains, that when a process needs a large
>> "physically contiguous" chunk of memory, it might not be available.
>> That said, usually, requests for physically contiguous memory is only
>> needed when wanting to map very large number of DMA pages for
>> doing direct physical I/O.
>> Otherwise, a process itself does not need to have physically contiguous
>> pages. Only the virtual space allocated to that "malloc" or large buffer
>> declaration in a program, is contiguous.
> Why would malloc or a large buffer declaration
> require physically contiguous memory?
It is done in a driver on the process' behalf when doing direct physical
IO .
typically, such blocks of physically contiguous chunks memory are set
aside during boot.
I have also seen special embedded linux drivers that provide an ioctl
to let the process get a set of physically contiguous pages and map the
space
to user virtual space. This is for performance reasons to reduce copying
from user space to kernel space when large amounts of data need to be
moved.
This is not a new idea. it has been around for many years. I first
saw it in Linux back in 1998/1999.

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/19/2010 10:22 PM, Michael Hennebry wrote:
> On Thu, 19 Aug 2010, Gregory Hosler wrote:
>
>> If the memory gets fragged and the kernel wants to defrag, e.g. for a memory
>> request from an application, in order to defrag any "dirty" data portions (those
>> pages that have been written to), the kernel *requires* there to be swap.
>> Otherwise there is no place to write the dirty pages out, in order to read them
>> in elsewhere.
>
> I didn't realize that memory could get fragged.
> I'd thought that one reason for virtual memory
> was allowing pages to be renumbered at will,
> the kernel's will, of course.

virtually memory, yes. physical memory no.

:-)

- -G


- --
+---------------------------------------------------------------------+

Please also check the log file at "/dev/null" for additional information.
(from /var/log/Xorg.setup.log)

| Greg Hosler ghosler@redhat.com |
+---------------------------------------------------------------------+
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAkxt1hkACgkQ404fl/0CV/TAEQCeOTi11TAxgeCAEd1woyy+guLi
H+gAoI76eU9/0yta9lDaGBtNcPIaxYbm
=5/xt
-----END PGP SIGNATURE-----
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On Thu, 19 Aug 2010, JD wrote:

> On 08/19/2010 02:15 PM, Michael Hennebry wrote:
>> On Thu, 19 Aug 2010, JD wrote:
>>
>>> Problem comes as Michael explains, that when a process needs a large
>>> "physically contiguous" chunk of memory, it might not be available.
>>> That said, usually, requests for physically contiguous memory is only
>>> needed when wanting to map very large number of DMA pages for
>>> doing direct physical I/O.
>>> Otherwise, a process itself does not need to have physically contiguous
>>> pages. Only the virtual space allocated to that "malloc" or large buffer
>>> declaration in a program, is contiguous.
>> Why would malloc or a large buffer declaration
>> require physically contiguous memory?
> It is done in a driver on the process' behalf when doing direct physical
> IO .
> typically, such blocks of physically contiguous chunks memory are set
> aside during boot.
> I have also seen special embedded linux drivers that provide an ioctl
> to let the process get a set of physically contiguous pages and map the
> space
> to user virtual space. This is for performance reasons to reduce copying
> from user space to kernel space when large amounts of data need to be
> moved.
> This is not a new idea. it has been around for many years. I first
> saw it in Linux back in 1998/1999.

Perhaps I misunderstood.
Do both of the following necessarily require physically contiguous memory?
char fred[69000];
char *greg=malloc(96000);
Would they sometimes require physically contiguous memory?

--
Michael hennebry@web.cs.ndsu.NoDak.edu
"Pessimist: The glass is half empty.
Optimist: The glass is half full.
Engineer: The glass is twice as big as it needs to be."
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On Thu, 2010-08-19 at 16:15 -0500, Michael Hennebry wrote:
> On Thu, 19 Aug 2010, JD wrote:
>
> > Problem comes as Michael explains, that when a process needs a large
> > "physically contiguous" chunk of memory, it might not be available.
> > That said, usually, requests for physically contiguous memory is only
> > needed when wanting to map very large number of DMA pages for
> > doing direct physical I/O.
> > Otherwise, a process itself does not need to have physically contiguous
> > pages. Only the virtual space allocated to that "malloc" or large buffer
> > declaration in a program, is contiguous.
>
> Why would malloc or a large buffer declaration
> require physically contiguous memory?
I have te equally interesting question? Why you think malloc allocates
memory blocks in the swap area. Do you have a reference for such a
statement?

--
Aaron Konstam <akonstam@sbcglobal.net>

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On 08/20/2010 03:36 PM, Aaron Konstam wrote:
> On Thu, 2010-08-19 at 16:15 -0500, Michael Hennebry wrote:
>> On Thu, 19 Aug 2010, JD wrote:
>>
>>> Problem comes as Michael explains, that when a process needs a large
>>> "physically contiguous" chunk of memory, it might not be available.
>>> That said, usually, requests for physically contiguous memory is only
>>> needed when wanting to map very large number of DMA pages for
>>> doing direct physical I/O.
>>> Otherwise, a process itself does not need to have physically contiguous
>>> pages. Only the virtual space allocated to that "malloc" or large buffer
>>> declaration in a program, is contiguous.
>>
>> Why would malloc or a large buffer declaration
>> require physically contiguous memory?
> I have te equally interesting question? Why you think malloc allocates
> memory blocks in the swap area.

JD didn't say that.

He said that when a process needs a large physically contiguous chunk of memory,
you may need to use swap space to move other processes out of the way.

Andrew.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On 08/20/2010 06:44 AM, Michael Hennebry wrote:
> On Thu, 19 Aug 2010, JD wrote:
>
>> On 08/19/2010 02:15 PM, Michael Hennebry wrote:
>>> On Thu, 19 Aug 2010, JD wrote:
>>>
>>>> Problem comes as Michael explains, that when a process needs a large
>>>> "physically contiguous" chunk of memory, it might not be available.
>>>> That said, usually, requests for physically contiguous memory is only
>>>> needed when wanting to map very large number of DMA pages for
>>>> doing direct physical I/O.
>>>> Otherwise, a process itself does not need to have physically contiguous
>>>> pages. Only the virtual space allocated to that "malloc" or large buffer
>>>> declaration in a program, is contiguous.
>>> Why would malloc or a large buffer declaration
>>> require physically contiguous memory?
>> It is done in a driver on the process' behalf when doing direct physical
>> IO .
>> typically, such blocks of physically contiguous chunks memory are set
>> aside during boot.
>> I have also seen special embedded linux drivers that provide an ioctl
>> to let the process get a set of physically contiguous pages and map the
>> space
>> to user virtual space. This is for performance reasons to reduce copying
>> from user space to kernel space when large amounts of data need to be
>> moved.
>> This is not a new idea. it has been around for many years. I first
>> saw it in Linux back in 1998/1999.
> Perhaps I misunderstood.
> Do both of the following necessarily require physically contiguous memory?
> char fred[69000];
> char *greg=malloc(96000);
> Would they sometimes require physically contiguous memory?
>
It depends on what you want to achieve.
If the target device you will write that buffer to can handle a
contiguous physical space of, say ... a few pages, then you
would want to ask the special driver of that device, via an ioctl,
to give you those pages, and map them to user virtual space -
i.e. you would not allocate them from the heap.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On 08/20/2010 07:36 AM, Aaron Konstam wrote:
> On Thu, 2010-08-19 at 16:15 -0500, Michael Hennebry wrote:
>> On Thu, 19 Aug 2010, JD wrote:
>>
>>> Problem comes as Michael explains, that when a process needs a large
>>> "physically contiguous" chunk of memory, it might not be available.
>>> That said, usually, requests for physically contiguous memory is only
>>> needed when wanting to map very large number of DMA pages for
>>> doing direct physical I/O.
>>> Otherwise, a process itself does not need to have physically contiguous
>>> pages. Only the virtual space allocated to that "malloc" or large buffer
>>> declaration in a program, is contiguous.
>> Why would malloc or a large buffer declaration
>> require physically contiguous memory?
> I have te equally interesting question? Why you think malloc allocates
> memory blocks in the swap area. Do you have a reference for such a
> statement?
>
Who said what you claim was said?
An OP already posted that you CAN run linux
without swap.
Normally, when you DO have swap space, user-
land data areas (both static and dynamic), will
be backed to swap if and when you run out of
memory and some other process needs memory.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On Fri, 20 Aug 2010, JD wrote:

> On 08/20/2010 06:44 AM, Michael Hennebry wrote:
>> On Thu, 19 Aug 2010, JD wrote:
>>
>>> On 08/19/2010 02:15 PM, Michael Hennebry wrote:
>>>> On Thu, 19 Aug 2010, JD wrote:
>>>>
>>>>> Problem comes as Michael explains, that when a process needs a large
>>>>> "physically contiguous" chunk of memory, it might not be available.
>>>>> That said, usually, requests for physically contiguous memory is only
>>>>> needed when wanting to map very large number of DMA pages for
>>>>> doing direct physical I/O.
>>>>> Otherwise, a process itself does not need to have physically contiguous
>>>>> pages. Only the virtual space allocated to that "malloc" or large buffer
>>>>> declaration in a program, is contiguous.
>>>> Why would malloc or a large buffer declaration
>>>> require physically contiguous memory?
>>> It is done in a driver on the process' behalf when doing direct physical
>>> IO .
>>> typically, such blocks of physically contiguous chunks memory are set
>>> aside during boot.
>>> I have also seen special embedded linux drivers that provide an ioctl
>>> to let the process get a set of physically contiguous pages and map the
>>> space
>>> to user virtual space. This is for performance reasons to reduce copying
>>> from user space to kernel space when large amounts of data need to be
>>> moved.
>>> This is not a new idea. it has been around for many years. I first
>>> saw it in Linux back in 1998/1999.
>> Perhaps I misunderstood.
>> Do both of the following necessarily require physically contiguous memory?
>> char fred[69000];
>> char *greg=malloc(96000);
>> Would they sometimes require physically contiguous memory?
>>
> It depends on what you want to achieve.
> If the target device you will write that buffer to can handle a
> contiguous physical space of, say ... a few pages, then you
> would want to ask the special driver of that device, via an ioctl,
> to give you those pages, and map them to user virtual space -
> i.e. you would not allocate them from the heap.

It makes sense that if a process insists on physically
contiguous memory and can't get it, the process would die,
but the above code does not tell the compiler what is to be achieved.

In the following, would fred or greg necessarily
refer to physically contiguous memory?

#include <stdlib.h>
extern void hank(char *);

int main(*args[], int argsNum)
{
char fred[69000];
char *greg=malloc(96000);
char *greg=malloc(96000);
hank(fred);
hank(greg);
return 0;
}

--
Michael hennebry@web.cs.ndsu.NoDak.edu
"Pessimist: The glass is half empty.
Optimist: The glass is half full.
Engineer: The glass is twice as big as it needs to be."
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On Fri, 20 Aug 2010 13:22:33 -0500 (CDT)
Michael Hennebry <hennebry@web.cs.ndsu.nodak.edu> wrote:
> It makes sense that if a process insists on physically
> contiguous memory and can't get it, the process would die,
> but the above code does not tell the compiler what is to be achieved.
>
> In the following, would fred or greg necessarily
> refer to physically contiguous memory?
>
> #include <stdlib.h>
> extern void hank(char *);
>
> int main(*args[], int argsNum)
> {
> char fred[69000];
> char *greg=malloc(96000);
> hank(fred);
> hank(greg);
> return 0;
> }

If I remember my Kerningham-Ritchie correctly, the answer is yes, since
C relies on pointer arithmetic to refer to the elements of the array.
The "fred" and "greg" variables are pointers to the beginning of the
corresponding memory area, and referring fred[i] goes to the start of
the array at fred, and then goes i elements forward to end up with the
wanted element.
--
Jussi Lehtola
Fedora Project Contributor
jussilehtola@fedoraproject.org
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

On Fri, 20 Aug 2010, Jussi Lehtola wrote:

> On Fri, 20 Aug 2010 13:22:33 -0500 (CDT)
> Michael Hennebry <hennebry@web.cs.ndsu.nodak.edu> wrote:
>> It makes sense that if a process insists on physically
>> contiguous memory and can't get it, the process would die,
>> but the above code does not tell the compiler what is to be achieved.
>>
>> In the following, would fred or greg necessarily
>> refer to physically contiguous memory?
>>
>> #include <stdlib.h>
>> extern void hank(char *);
>>
>> int main(*args[], int argsNum)
>> {
>> char fred[69000];
>> char *greg=malloc(96000);
>> hank(fred);
>> hank(greg);
>> return 0;
>> }
>
> If I remember my Kerningham-Ritchie correctly, the answer is yes, since
> C relies on pointer arithmetic to refer to the elements of the array.
> The "fred" and "greg" variables are pointers to the beginning of the
> corresponding memory area, and referring fred[i] goes to the start of
> the array at fred, and then goes i elements forward to end up with the
> wanted element.

That is contiguous in terms of virtual memory.
Adjacent virtual addresses do not have to have adjacent physical addresses.

--
Michael hennebry@web.cs.ndsu.NoDak.edu
"Pessimist: The glass is half empty.
Optimist: The glass is half full.
Engineer: The glass is twice as big as it needs to be."
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines