Difference between revisions of "Memory Limits"
From Linux-VServer
(+cat) |
|||
| (19 intermediate revisions by 15 users not shown) | |||
| Line 1: | Line 1: | ||
| + | == Overview == | ||
A vserver kernel keeps track many resources used by each guest (context). Some of these relate to memory usage by the guest. You can place limits on these resources to prevent guests from using all the host memory and making the host unusable. | A vserver kernel keeps track many resources used by each guest (context). Some of these relate to memory usage by the guest. You can place limits on these resources to prevent guests from using all the host memory and making the host unusable. | ||
| − | Two resources are particularly important in this regard: | + | Two resources are particularly important in this regard: |
| − | + | * The '''Resident Set Size''' (<code>rss</code>) is the amount of pages currently present in RAM. | |
| + | * The '''Address Space''' (<code>as</code>) is the total amount of memory (pages) mapped in all the processes in the context. | ||
| − | + | Both are measured in '''pages''', which are 4 kB each on Intel machines (i386). So a value of 200000 means a limit of 800,000 kB, a little less than 800 MB. | |
| + | |||
| + | ''To easily find out the page size on your host try this line and ignore the warnings'' | ||
| + | <pre>echo 'int main () { printf ("%dKiB\n", getpagesize ()/1024); return 0; }' | gcc -xc - -o getpagesize && ./getpagesize</pre> | ||
| + | Each resource has a '''soft''' and a '''hard limit'''. | ||
| + | |||
| + | * If a guest exceeds the <code>rss</code> hard limit, the kernel will invoke the Out-of-Memory (OOM) killer to kill some process in the guest. | ||
| + | * The <code>rss</code> soft limit is shown inside the guest as the maximum available memory. If a guest exceeds the <code>rss</code> soft limit, it will get an extra "bonus" for the OOM killer (proportional to the oversize). | ||
| + | * If a guest exceeds the <code>as</code> hard limit, memory allocation attempts will return an error, but no process is killed. | ||
| + | * The <code>as</code> soft limit is not in utilized until now. In the future it may be used to penalizing guests over that limit or it could be used to force swapping on them and such ... | ||
Bertl explained the difference between '''rss''' and '''as''' with the following example. If two processes share 100 MB of memory, then only 100 MB worth of virtual memory pages can be used at most, so the RSS use of the guest increases by 100 MB. However, two processes are using it, so the AS use increases by 200 MB. | Bertl explained the difference between '''rss''' and '''as''' with the following example. If two processes share 100 MB of memory, then only 100 MB worth of virtual memory pages can be used at most, so the RSS use of the guest increases by 100 MB. However, two processes are using it, so the AS use increases by 200 MB. | ||
| Line 11: | Line 22: | ||
This makes me think that limiting AS is less useful than limiting RSS, since it doesn't directly reflect real, limited resources (RAM and swap) on the host, that deprive other virtual machines of those resources. Bertl says that AS limits can be used to give guests a "gentle" warning that they are running out of memory, but I don't know how much more gentle it is, or how to set it accurately. | This makes me think that limiting AS is less useful than limiting RSS, since it doesn't directly reflect real, limited resources (RAM and swap) on the host, that deprive other virtual machines of those resources. Bertl says that AS limits can be used to give guests a "gentle" warning that they are running out of memory, but I don't know how much more gentle it is, or how to set it accurately. | ||
| − | For example, 100 processes each mapping a 100 | + | For example, 100 processes each mapping a 100 MB file would consume a total of 10 GB of address space (AS), but no more than 100 MB of resources on the host. But if you set the AS limit to 10 GB, then it will not stop one process from allocating 4 GB of RAM, which could kill the host or result in that process being killed by the OOM killer. |
| + | |||
| + | this is posted by siddharth | ||
| + | |||
| + | == Setting memory limits == | ||
| + | '''Beginning with vs2.3.0.36.29 you should use cgroups to set memory limits. More information: [[util-vserver:Cgroups#using_cgroup_to_enforce_memory_limits|using cgroup to enforce memory limits]]''' | ||
You can set the hard limit on a particular context, effective immediately, with this command: | You can set the hard limit on a particular context, effective immediately, with this command: | ||
| Line 19: | Line 35: | ||
</pre> | </pre> | ||
| − | For example, if you want to change the '''rss''' hard limit for the vserver with <xid> | + | <code><xid></code> is the context ID of the guest, which you can determine with the <code>/usr/sbin/vserver-stat</code> command. |
| + | |||
| + | For example, if you want to change the '''rss''' hard limit for the vserver with <code><xid></code> 49000, and limit it to 10,000 pages (40 MB), you could use this command: | ||
<pre> | <pre> | ||
| − | /usr/sbin/vlimit -c 49000 --rss | + | /usr/sbin/vlimit -c 49000 --rss 10000 |
</pre> | </pre> | ||
| − | You can change the soft limit instead by adding the -S parameter. | + | You can change the soft limit instead by adding the <code>-S</code> parameter. |
| − | Changes made with the vlimit command are effective only until the vserver is stopped. To make permanent changes, write the value to /etc/vservers/<name>/rlimits/<resource> | + | Changes made with the vlimit command are effective only until the vserver is stopped. To make permanent changes, write the value to this file: |
| + | |||
| + | <pre> | ||
| + | /etc/vservers/<name>/rlimits/<resource>.hard | ||
| + | </pre> | ||
| + | |||
| + | To set a soft limit, use the same file name with the <code>.soft</code> extension. The <code>rlimits</code> directory is not created by default, so you may need to create it yourself. | ||
| + | |||
| + | If you omit the suffix after the <code>/etc/vservers/<name>/rlimits/rss</code> file, the value will be set for both, the hard and soft limit. | ||
| + | |||
| + | Changes to these files take effect only when the vserver is started. To make immediate and permanent changes to a running vserver, you need to run <code>vlimit</code> '''and''' update the rlimits file. | ||
The safest setting, to prevent any guest from interfering with any other, is to set the total of all RSS hard limits (across all running guests) to be less than the total virtual memory (RAM and swap) on the host. It should be sufficiently less to leave room for processes running on the host, and some disk cache, perhaps 100 MB. | The safest setting, to prevent any guest from interfering with any other, is to set the total of all RSS hard limits (across all running guests) to be less than the total virtual memory (RAM and swap) on the host. It should be sufficiently less to leave room for processes running on the host, and some disk cache, perhaps 100 MB. | ||
However, this is very conservative, since it assumes the worst case where all guests are using the maximum amount of memory at one time. In practice, you can usually get away with contended resources, i.e. allowing guests to use more than this value. | However, this is very conservative, since it assumes the worst case where all guests are using the maximum amount of memory at one time. In practice, you can usually get away with contended resources, i.e. allowing guests to use more than this value. | ||
| + | |||
| + | == Displaying current memory limits == | ||
| + | To display the currently active RSS limits for a vserver execute the following command: | ||
| + | <pre> | ||
| + | vlimit -c <xid> -a -d | grep RSS | ||
| + | </pre> | ||
| + | |||
| + | The above command will display a similar result, whereas the third value (5000) is the soft limit and the last reflects the current hard limit (10000). | ||
| + | <pre> | ||
| + | RSS N/A 5000 10000 | ||
| + | </pre> | ||
| + | |||
| + | == Display memory limits within a vserver == | ||
| + | Normally the <code>top</code> and <code>free</code> command will display the RAM and Swap usage of the host while invoked within a vserver. To change this behavior, add the <code>VIRT_MEM</code> [[Capabilities_and_Flags#Context_flags_.28cflags.29|context flag]] to your vserver configuration: | ||
| + | <pre> | ||
| + | echo "VIRT_MEM" >> /etc/vservers/<name>/flags | ||
| + | </pre> | ||
| + | |||
| + | After a successful restart of the related vserver, the total available RAM will equal to the value of <code>rss.soft</code> while the difference of <code>rss.hard</code> - <code>rss.soft</code> will be displayed as swap space. | ||
| + | |||
| + | As an example, if you set the <code>rss.hard</code> limit to 10'000 pages and the <code>rss.soft</code> limit to 7'000 pages: | ||
| + | <pre> | ||
| + | vlimit -c <xid> --rss 10000 | ||
| + | vlimit -c <xid> -S --rss 7000 | ||
| + | vlimit -c <xid> -a -d | grep RSS | ||
| + | </pre> | ||
| + | <pre> | ||
| + | RSS N/A 7000 10000 | ||
| + | </pre> | ||
| + | |||
| + | <code>free</code> will report 28'000 KB (7'000 pages) of total memory and 12'000 KB (10'000 - 7'000 pages) of Swap space (assuming that one page is 4 KB): | ||
| + | <pre> | ||
| + | free -k | ||
| + | </pre> | ||
| + | <pre> | ||
| + | total used free shared buffers cached | ||
| + | Mem: 28000 4396 23604 0 0 0 | ||
| + | -/+ buffers/cache: 4396 23604 | ||
| + | Swap: 12000 0 12000 | ||
| + | </pre> | ||
| + | |||
| + | '''Note:''' According to Herbert, the kernel won't use any ''real'' swap as soon as the <code>rss.soft</code> limit has been reached. Swapping will be done on the host level, not per vserver (see [http://list.linux-vserver.org/archive?mss:653:200801:iiagcodpghkhehndkgjg "free" command inside vserver]). | ||
| + | |||
| + | [[Category:Theory]] | ||
Latest revision as of 20:25, 21 October 2011
Contents |
[edit] Overview
A vserver kernel keeps track many resources used by each guest (context). Some of these relate to memory usage by the guest. You can place limits on these resources to prevent guests from using all the host memory and making the host unusable.
Two resources are particularly important in this regard:
- The Resident Set Size (
rss) is the amount of pages currently present in RAM. - The Address Space (
as) is the total amount of memory (pages) mapped in all the processes in the context.
Both are measured in pages, which are 4 kB each on Intel machines (i386). So a value of 200000 means a limit of 800,000 kB, a little less than 800 MB.
To easily find out the page size on your host try this line and ignore the warnings
echo 'int main () { printf ("%dKiB\n", getpagesize ()/1024); return 0; }' | gcc -xc - -o getpagesize && ./getpagesize
Each resource has a soft and a hard limit.
- If a guest exceeds the
rsshard limit, the kernel will invoke the Out-of-Memory (OOM) killer to kill some process in the guest. - The
rsssoft limit is shown inside the guest as the maximum available memory. If a guest exceeds thersssoft limit, it will get an extra "bonus" for the OOM killer (proportional to the oversize). - If a guest exceeds the
ashard limit, memory allocation attempts will return an error, but no process is killed. - The
assoft limit is not in utilized until now. In the future it may be used to penalizing guests over that limit or it could be used to force swapping on them and such ...
Bertl explained the difference between rss and as with the following example. If two processes share 100 MB of memory, then only 100 MB worth of virtual memory pages can be used at most, so the RSS use of the guest increases by 100 MB. However, two processes are using it, so the AS use increases by 200 MB.
This makes me think that limiting AS is less useful than limiting RSS, since it doesn't directly reflect real, limited resources (RAM and swap) on the host, that deprive other virtual machines of those resources. Bertl says that AS limits can be used to give guests a "gentle" warning that they are running out of memory, but I don't know how much more gentle it is, or how to set it accurately.
For example, 100 processes each mapping a 100 MB file would consume a total of 10 GB of address space (AS), but no more than 100 MB of resources on the host. But if you set the AS limit to 10 GB, then it will not stop one process from allocating 4 GB of RAM, which could kill the host or result in that process being killed by the OOM killer.
this is posted by siddharth
[edit] Setting memory limits
Beginning with vs2.3.0.36.29 you should use cgroups to set memory limits. More information: using cgroup to enforce memory limits
You can set the hard limit on a particular context, effective immediately, with this command:
/usr/sbin/vlimit -c <xid> --<resource> <value>
<xid> is the context ID of the guest, which you can determine with the /usr/sbin/vserver-stat command.
For example, if you want to change the rss hard limit for the vserver with <xid> 49000, and limit it to 10,000 pages (40 MB), you could use this command:
/usr/sbin/vlimit -c 49000 --rss 10000
You can change the soft limit instead by adding the -S parameter.
Changes made with the vlimit command are effective only until the vserver is stopped. To make permanent changes, write the value to this file:
/etc/vservers/<name>/rlimits/<resource>.hard
To set a soft limit, use the same file name with the .soft extension. The rlimits directory is not created by default, so you may need to create it yourself.
If you omit the suffix after the /etc/vservers/<name>/rlimits/rss file, the value will be set for both, the hard and soft limit.
Changes to these files take effect only when the vserver is started. To make immediate and permanent changes to a running vserver, you need to run vlimit and update the rlimits file.
The safest setting, to prevent any guest from interfering with any other, is to set the total of all RSS hard limits (across all running guests) to be less than the total virtual memory (RAM and swap) on the host. It should be sufficiently less to leave room for processes running on the host, and some disk cache, perhaps 100 MB.
However, this is very conservative, since it assumes the worst case where all guests are using the maximum amount of memory at one time. In practice, you can usually get away with contended resources, i.e. allowing guests to use more than this value.
[edit] Displaying current memory limits
To display the currently active RSS limits for a vserver execute the following command:
vlimit -c <xid> -a -d | grep RSS
The above command will display a similar result, whereas the third value (5000) is the soft limit and the last reflects the current hard limit (10000).
RSS N/A 5000 10000
[edit] Display memory limits within a vserver
Normally the top and free command will display the RAM and Swap usage of the host while invoked within a vserver. To change this behavior, add the VIRT_MEM context flag to your vserver configuration:
echo "VIRT_MEM" >> /etc/vservers/<name>/flags
After a successful restart of the related vserver, the total available RAM will equal to the value of rss.soft while the difference of rss.hard - rss.soft will be displayed as swap space.
As an example, if you set the rss.hard limit to 10'000 pages and the rss.soft limit to 7'000 pages:
vlimit -c <xid> --rss 10000 vlimit -c <xid> -S --rss 7000 vlimit -c <xid> -a -d | grep RSS
RSS N/A 7000 10000
free will report 28'000 KB (7'000 pages) of total memory and 12'000 KB (10'000 - 7'000 pages) of Swap space (assuming that one page is 4 KB):
free -k
total used free shared buffers cached
Mem: 28000 4396 23604 0 0 0
-/+ buffers/cache: 4396 23604
Swap: 12000 0 12000
Note: According to Herbert, the kernel won't use any real swap as soon as the rss.soft limit has been reached. Swapping will be done on the host level, not per vserver (see "free" command inside vserver).
