Transcription
Memory Management 101: Introductionto Memory Management in LinuxChristopher Lameter, Ph.D. cl@linux.com Jump Trading LLC@qant1
Overview Memory and processesReal/Virtual memory and PagingMachine configurationProcesses use of memoryOvercommitKnobsThere is an advanced MM talk tomorrow called“Flavors of Memory”2
Pages and physical page frame numbers Division of memory into “pages” 1-N bytes become split at pagesize boundaries and becomeM N/page size pages Refer to memory by the PageFrame Number (PFN) and anoffset into the page. Common size is 4k (Intel legacy issues) MMU creates virtual addresses.
Basics of “paging” Process have virtual memory- PFNPage TablesFaults Major Minor Virtual vs physical
Process memory Virtual memory maps tophysical memory View of memory that is distinctfor each process. Pages shared Access control Copy on Write
Swap, Zero pages etc. Swap pageZero pageRead data behaviorWrite data behaviorAnonymous vs file backed pages
Linux Basic memory information/proc/meminfo/sys/devices/system/has lots of more detailedinformation on hardware(processors andmemory)Commands:numactl --hardwarefree, top, dmesgMemTotal:31798552 kBMemFree:25949124 kBMemAvailable: 30823580 kBBuffers:220988 kBCached:4679188 kBSwapCached:0 kBActive:2803000 kBInactive:2336992 kBActive(anon):240776 kBInactive(anon):6432 kBActive(file):2562224 kBInactive(file):2330560 kBUnevictable:0 kBMlocked:0 kBSwapTotal:2097148 kBSwapFree:2097148 kBDirty:48 kBWriteback:0 aim:KernelStack:PageTables:239716 kB195596 kB7396 kB550628 kB443040 kB107588 kB6840 kB11176 kB
Inspecting a process memory use/proc/ pid /status/proc/ pid /*maps(there are other files in /proc/ pid /*with more information about theprocesses)Commands:ps, topName: sshdVmPeak:65772 kBVmSize:65772 kBVmLck:0 kBVmPin:0 kBVmHWM:6008 kBVmRSS:6008 kBRssAnon:1216 kBRssFile:4792 kBRssShmem:0 kBVmData: 1332 kBVmStk:132 kBVmExe:492 kBVmLib:8076 kBVmPTE:168 kBVmSwap:0 kB
User limit (ulimit) max memory sizevirtual memorystack sizeand lots of othercontrols.cl@nuc-kabylake:/proc/6713 ulimit -acore file size(blocks, -c) 0data seg size(kbytes, -d) unlimitedscheduling priority(-e)0file size(blocks, -f)unlimitedpending signals(-i)123132max locked memory(kbytes, -l) 16384max memory size(kbytes, -m) unlimitedopen files(-n)1024pipe size(512 bytes, -p) 8POSIX message queues (bytes, -q) 819200real-time priority(-r)0stack size(kbytes, -s)8192cpu time(seconds, -t) unlimitedmax user processes(-u) 123132virtual memory(kbytes, -v) unlimitedfile locks(-x)unlimited
Overcommit configurationVirtual memory use vs physicalovercommit kbytesovercommit memory0 - overcommit. Guess if mem is available.1 - Overcommit. Never say there is no memory2 - Only allocate according to the ratioovercommit ratiototal swap physical * ratio
Important VM control knobsFound in /proc/sys/vmMore descriptions of these knobs in Kernel sourcecode. linux/Documentation/admin-guideadmin reserve kbytes dirty writeback centisecs min free kbytes numa zonelist orderstat refresh block dump drop caches min slab ratio oom dump tasks swappinesscompact memory extfrag threshold min unmapped ratio oom kill allocating taskuser reserve kbytes compact unevictable allowed hugetlb shm group mmap min addrovercommit kbytes vfs cache pressure dirty background bytes laptop modemmap rnd bits overcommit memory watermark scale factor dirty background ratiolegacy va layout mmap rnd compat bits overcommit ratio zone reclaim mode dirty byteslowmem reserve ratio nr hugepages page-cluster dirty expire centisecs max map countnr hugepages mempolicy panic on oom dirty ratio memory failure early killnr overcommit hugepages percpu pagelist fraction dirtytime expire secondsmemory failure recovery numa stat stat interval
Resources Admin Guide guide/index.html Kernel.org has wikis and documentation(www.kernel.org ) manpages (especially for system callsand coding)
Questions / CommentsYou can reach me at cl@linux.com or@qant on twitter13
14
“Simple” Memory Access UMA (Uniform Memory Access) Any access to memory has the same characteristics(performance and latency) The vast major of systems have only UMA. But there is always a processor cache hierarchy– The CPU is fast, memory is slow– Caches exist to avoid accesses to mainmemory Aliasing Coloring Cache Miss Trashing15
NUMA Memory Memory with different accesscharacteristics Memory Affinities depending onwhere a process was started Control NUMA allocs with memorypolicies System Partitioning using Cpusetsand Containers Manual memory migration Automatic memory migration16
Huge Memory Typical memory is handled in chunks of base page size (Intel 4k,IBM PowerX 64K, ARM 64K)Systems support larger memory chunks of memory called Hugepages (Intel 2M)Must be pre configured on boot in order to guarantee that they are availableRequired often for I/O bottlenecks on Intel.4TB requires 1 billion descriptors with 4K pages.Most of this is needed to compensate forarchitectural problems on Intel. Intel processorshave difficulties using modern SSDs and highspeed devices without this.Large contiguous segments (I/O performance)Fragmentation issuesUses files on a special file system that must be explicitly requested bymmap operations from special files.17
An Introduction to Linux memory management. The basics of paging. Understandingbasic hardware memory management and the difference between virtual, physical andswap memory. How do determine hardware installed and how to figure out howprocesses use that memory. How a process uses physical and virtual memory effectively.How to control overcommit and virtual and/or physical memory limits.Basic knobs in Linux to control memory management. System calls for a process tocontrol its memory usage18
An Introduction to Linux memory management. The basics of paging. Understanding basic hardware memory management and the difference between virtual, physical and swap memory. How do determine hardware installed and how to figure out how processes use that memory. How a process uses physical and virtual memory effectively.
