Memory leak on nodes running collectd due to unbounded memory usage in collectd when it's not configured with any write plugin - Red Hat OpenStack Platform 13

Issue

Memory leak on nodes running collectd due to unbounded memory usage in collectd when it's not configured with any write plugin in older versions of Red Hat OpenStack Platform 16 and 17.1

This issue can affect any node where collectd is running at.

Example for Ceph nodes affected by this

• One ceph storage node ran out of memory:

[root@overcloud-ceph-0 log]# sar -r 1 -f /var/log/sa/sa24
Linux 3.10.0-1062.1.1.el7.x86_64 (999711-ceph02)    02/24/2020  _x86_64_    (48 CPU)

12:00:02 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
12:10:02 AM  10811524 186775740     94.53     43000    980060  97456840     49.32 180205700    574896       204
12:20:01 AM  10738280 186848984     94.57     42776    978700  97174824     49.18 180381348    572824       200
12:30:01 AM  10619816 186967448     94.63     42676    981852  97302932     49.25 180588648    574504       184
12:40:01 AM  10270824 187316440     94.80     42592    980484  97251020     49.22 180835588    572588       204
12:50:01 AM  10003108 187584156     94.94     42576    989896  97289232     49.24 181077328    579528       196
01:00:01 AM   9855424 187731840     95.01     42556    984660  97162104     49.17 181299964    574244       152
01:10:01 AM   9574856 188012408     95.15     43064    986780  97502376     49.35 181569564    575276         8
01:20:02 AM   9380904 188206360     95.25     43052    985276  97511204     49.35 181770604    573344         8
01:30:01 AM   9318656 188268608     95.28     42840    992892  96973140     49.08 181932196    578268       124
01:40:01 AM   8968112 188619152     95.46     42836    990940  97153036     49.17 182135996    576020         0
01:50:01 AM   8790300 188796964     95.55     42820    989436  97287920     49.24 182397460    573816        12
02:00:01 AM   8591808 188995456     95.65     42784    992540  97571700     49.38 182598408    575448         8
02:10:01 AM   8476032 189111232     95.71     43140    995988  97313064     49.25 182804264    577028        32
02:20:01 AM   8030788 189556476     95.94     43040    994332  97511088     49.35 183030060    574900         4
02:30:01 AM   8064256 189523008     95.92     42976    995872  97329468     49.26 183211796    575860         4
02:40:02 AM   7817952 189769312     96.04     42844    995068  97349824     49.27 183430076    574036         4
02:50:01 AM   7475680 190111584     96.22     42828    998168  97558468     49.37 183626612    575320        16
03:00:01 AM   7258436 190328828     96.33     42812    996060  97570740     49.38 183857000    572752         4
03:10:01 AM   7064440 190522824     96.42     43092   1003632  97719820     49.46 184116464    577776         8
03:20:01 AM   6810820 190776444     96.55     43092   1001068  97770376     49.48 184336860    575264        12
03:30:01 AM   6603232 190984032     96.66     43028   1003480  97941212     49.57 184526356    576568         4
03:40:01 AM   6445556 191141708     96.74     42996   1006948  97910928     49.55 184681476    579004         4
03:50:02 AM   6416556 191170708     96.75     42804    877836  97482648     49.34 184744996    593692        16
04:00:01 AM   6232892 191354372     96.85     42708    875880  97499352     49.34 184952620    591500       160
04:10:01 AM   6121388 191465876     96.90     42964    879900  97745304     49.47 185143336    594812         0
04:20:01 AM   6706372 190880892     96.61      4836    191412  97507252     49.35 185089680    131640        28
04:30:01 AM   6377360 191209904     96.77      8312    395140  97246688     49.22 185341104    221596         8
04:40:01 AM   6299512 191287752     96.81      8312    408000  97239388     49.21 185415596    229144       168
04:50:01 AM   8537324 189049940     95.68     10676    424948  95337092     48.25 182968292    241304         0
05:00:02 AM   8132648 189454616     95.88     10672    424168  95569824     48.37 183345512    240408         4
05:10:01 AM   7974252 189613012     95.96     30084    430368  95292196     48.23 183564256    259604       756
05:20:01 AM   7758556 189828708     96.07     30004    423580  95346912     48.26 183820624    253196       760
05:30:01 AM   7343460 190243804     96.28     30004    425704  95543240     48.35 184033864    254440       132
05:40:01 AM   7210348 190376916     96.35     30004    429712  95653512     48.41 184242452    256352        12
05:50:02 AM   7111008 190476256     96.40     29972    435124  95374764     48.27 184455076    261312         0
06:00:01 AM   6753644 190833620     96.58     29904    432796  95439872     48.30 184706252    258688        12
06:10:01 AM   6663980 190923284     96.63     29988    439508  95147496     48.15 184924228    245252         0
06:20:01 AM   6419080 191168184     96.75     29968    434364  95232840     48.20 185159940    240388         4
06:30:01 AM   6199224 191388040     96.86     29832    440428  95200208     48.18 185318324    244328         4
06:40:01 AM   7264780 190322484     96.32         0    206520  93275656     47.21 184546104    171564       124
06:50:01 AM  55748332 141838932     71.79      2384    215424  94860692     48.01 136025016    179660       160

06:50:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
07:00:01 AM  55639736 141947528     71.84      2384    212156  94847212     48.00 136106124    175720       172
07:10:01 AM  55477772 142109492     71.92     29072    214260  94582800     47.87 136219556    200196       188
07:20:01 AM  54034112 143553152     72.65     29008    216656  95396752     48.28 136380448    201144       152
07:30:01 AM  54995248 142592016     72.17     28960    219244  94886944     48.02 136426624    203584        88
07:40:01 AM  55081528 142505736     72.12     28960    381500  94884908     48.02 136453644    361952       200
07:50:01 AM  55104288 142482976     72.11     28784    378812  94884932     48.02 136475436    359076       192
08:00:02 AM  55109436 142477828     72.11     28784    380844  94884628     48.02 136529760    360312       288
08:10:01 AM  54870768 142716496     72.23     28780    380148  94804568     47.98 136639008    335388       184
08:20:01 AM  54999376 142587888     72.16     28776    382060  94641564     47.90 136823544    336168       160
08:30:01 AM  54903184 142684080     72.21     28612    387728  94649848     47.90 136918812    339908       184
08:40:01 AM  54563340 143023924     72.39     28608    383816  94968228     48.06 137049928    336312       168
08:50:01 AM  54071444 143515820     72.63     28608    384352  94864728     48.01 137062280    336460       172
09:00:01 AM  52924192 144663072     73.21     28600    387876  95511624     48.34 137231668    337980       192
09:10:01 AM  54484516 143102748     72.43     29092    395304  94715400     47.94 137330196    345280       188
09:20:01 AM  54446328 143140936     72.44     28960    400212  94715024     47.94 137374336    348696       160
09:30:01 AM  54361116 143226148     72.49     28816    401576  94905324     48.03 137465808    349752       156
09:40:01 AM  54187700 143399564     72.58     28816    406892  94937756     48.05 137636060    353700       176
09:50:01 AM  54111172 143476092     72.61     28792    404408  94937660     48.05 137715004    350708       212
10:00:01 AM  54045612 143541652     72.65     28636    410700  94938416     48.05 137782192    355212       176
10:10:01 AM  53949492 143637772     72.70     28836    408496  94946504     48.05 137873456    352552       148
10:20:01 AM  53860072 143727192     72.74     28580    408904  94970204     48.06 137960976    352784       164
10:30:01 AM  53796140 143791124     72.77     28548    411840  94970520     48.07 138028056    354500       180
10:40:01 AM  53724480 143862784     72.81     28536    409440  94977132     48.07 138101608    351576       204
10:50:01 AM  53632208 143955056     72.86     28500    414820  94994768     48.08 138184264    355100       340
11:00:01 AM  53558640 144028624     72.89     28484    412272  95012300     48.09 138257464    352060       192
11:10:01 AM  53450056 144137208     72.95     29000    411924  95081460     48.12 138378372    352092       140
11:20:01 AM  53353684 144233580     73.00     28856    417760  95089924     48.13 138462776    355984       196
11:30:01 AM  53275284 144311980     73.04     28680    411144  95114180     48.14 138555004    349532       168
11:40:01 AM  53180856 144406408     73.08     28644    416928  95132444     48.15 138748064    241796       192
11:50:02 AM  53104720 144482544     73.12     28644    414872  95148888     48.16 138830952    240016       188
12:00:01 PM  53038364 144548900     73.16     28436    422188  95157284     48.16 138887716    244864       260
12:10:01 PM  52940960 144646304     73.21     28968    422612  95174276     48.17 138990432    245104       188
12:20:01 PM  52839704 144747560     73.26     28952    416772  95214816     48.19 139094124    239336       224
12:30:01 PM  52741608 144845656     73.31     28780    422560  95240468     48.20 139189332    243516       184
12:40:01 PM  52652076 144935188     73.35     28684    422440  95263852     48.21 139273696    242704       156
12:50:01 PM  52580036 145007228     73.39     28560    423224  95276864     48.22 139347252    242972       172
01:00:01 PM  52470796 145116468     73.44     28544    429764  95307508     48.24 139450480    246944        52
01:10:01 PM  52386888 145200376     73.49     28916    430736  95324964     48.24 139535056    247872       172
01:20:01 PM  52288428 145298836     73.54     28740    437240  95346152     48.26 139622800    252660       172
Average:     30905961 166681303     84.36     31022    571368  95891363     48.53 160607542    374746       136

• top shows high memory usage for collectd:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  10959 root      20   0   94.8g  93.7g   1084 S   0.0 49.7   1408:39 collectd
  11693 ceph      20   0 5530948   4.2g   6336 S   2.9  2.2 438:26.95 ceph-osd
  11865 ceph      20   0 5223240   3.9g   6320 S   4.3  2.1 310:21.04 ceph-osd
  11439 ceph      20   0 5061992   3.7g   6336 S   5.8  2.0 428:49.36 ceph-osd
  11837 ceph      20   0 5040588   3.7g   6344 S   3.6  2.0 399:11.14 ceph-osd
  11393 ceph      20   0 5145832   3.7g   6356 S   2.9  2.0 485:14.45 ceph-osd

• One of the ceph-osd process was killed:

[root@overcloud-ceph-- log]# ps -elf |grep ceph-os
4 S ceph       11187       1  3  80   0 - 1077076 futex_ Feb20 ?      04:04:53 /usr/bin/ceph-osd -f --cluster ceph --id 31 --setuser ceph --setgroup ceph
4 S ceph       11188       1  3  80   0 - 1043714 futex_ Feb20 ?      03:26:18 /usr/bin/ceph-osd -f --cluster ceph --id 34 --setuser ceph --setgroup ceph
4 S ceph       11189       1  3  80   0 - 1036474 futex_ Feb20 ?      03:24:50 /usr/bin/ceph-osd -f --cluster ceph --id 24 --setuser ceph --setgroup ceph
4 S ceph       11190       1  3  80   0 - 1131482 futex_ Feb20 ?      03:22:22 /usr/bin/ceph-osd -f --cluster ceph --id 30 --setuser ceph --setgroup ceph
4 S ceph       11194       1  2  80   0 - 1036458 futex_ Feb20 ?      02:16:49 /usr/bin/ceph-osd -f --cluster ceph --id 43 --setuser ceph --setgroup ceph
4 S ceph       11198       1  2  80   0 - 1029214 futex_ Feb20 ?      02:32:22 /usr/bin/ceph-osd -f --cluster ceph --id 28 --setuser ceph --setgroup ceph
4 S ceph       11199       1  2  80   0 - 945938 futex_ Feb20 ?       02:48:27 /usr/bin/ceph-osd -f --cluster ceph --id 39 --setuser ceph --setgroup ceph
4 S ceph       11203       1  3  80   0 - 979937 futex_ Feb20 ?       03:35:00 /usr/bin/ceph-osd -f --cluster ceph --id 41 --setuser ceph --setgroup ceph
4 S ceph       11205       1  3  80   0 - 1012111 futex_ Feb20 ?      03:10:16 /usr/bin/ceph-osd -f --cluster ceph --id 32 --setuser ceph --setgroup ceph
4 S ceph       11216       1  3  80   0 - 1082650 futex_ Feb20 ?      03:18:27 /usr/bin/ceph-osd -f --cluster ceph --id 38 --setuser ceph --setgroup ceph
4 S ceph       11222       1  6  80   0 - 1053951 futex_ Feb20 ?      06:22:22 /usr/bin/ceph-osd -f --cluster ceph --id 29 --setuser ceph --setgroup ceph
4 S ceph       11224       1  2  80   0 - 1000603 futex_ Feb20 ?      03:04:12 /usr/bin/ceph-osd -f --cluster ceph --id 26 --setuser ceph --setgroup ceph
4 S ceph       11228       1  2  80   0 - 980181 futex_ Feb20 ?       02:48:44 /usr/bin/ceph-osd -f --cluster ceph --id 44 --setuser ceph --setgroup ceph
4 S ceph       11235       1  2  80   0 - 1019063 futex_ Feb20 ?      03:07:07 /usr/bin/ceph-osd -f --cluster ceph --id 36 --setuser ceph --setgroup ceph
4 S ceph       11241       1  2  80   0 - 1032931 futex_ Feb20 ?      03:07:18 /usr/bin/ceph-osd -f --cluster ceph --id 42 --setuser ceph --setgroup ceph
4 S ceph       11245       1  4  80   0 - 1004392 futex_ Feb20 ?      04:10:48 /usr/bin/ceph-osd -f --cluster ceph --id 33 --setuser ceph --setgroup ceph
4 S ceph       11255       1  2  80   0 - 1023141 futex_ Feb20 ?      03:04:46 /usr/bin/ceph-osd -f --cluster ceph --id 25 --setuser ceph --setgroup ceph
4 S ceph       11258       1  2  80   0 - 1153967 futex_ Feb20 ?      02:38:46 /usr/bin/ceph-osd -f --cluster ceph --id 40 --setuser ceph --setgroup ceph
4 S ceph       11259       1  3  80   0 - 962888 futex_ Feb20 ?       03:37:45 /usr/bin/ceph-osd -f --cluster ceph --id 45 --setuser ceph --setgroup ceph
4 S ceph       11267       1  3  80   0 - 1093080 futex_ Feb20 ?      03:20:00 /usr/bin/ceph-osd -f --cluster ceph --id 27 --setuser ceph --setgroup ceph
4 S ceph       11269       1  4  80   0 - 962968 futex_ Feb20 ?       04:38:46 /usr/bin/ceph-osd -f --cluster ceph --id 47 --setuser ceph --setgroup ceph
4 S ceph       11274       1  3  80   0 - 978950 futex_ Feb20 ?       04:00:13 /usr/bin/ceph-osd -f --cluster ceph --id 37 --setuser ceph --setgroup ceph
4 S ceph       11277       1  3  80   0 - 1019468 futex_ Feb20 ?      03:20:33 /usr/bin/ceph-osd -f --cluster ceph --id 35 --setuser ceph --setgroup ceph
4 S ceph     2642023       1  3  80   0 - 464133 futex_ 06:41 ?       00:14:25 /usr/bin/ceph-osd -f --cluster ceph --id 46 --setuser ceph --setgroup ceph   <--------

with an error similar to this one:

Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: in thread 7f3cf93ebd80 thread_name:ceph-osd
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: ceph version 12.2.12-48.el7cp (26388d73d88602005946d4381cc5796d42904858) luminous (stable)
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 1: (()+0xa65921) [0x558eec5fe921]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 2: (()+0xf630) [0x7f3cf6aa7630]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 3: (std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)+0x4) [0x7f3cf63e5e54]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 4: (pg_log_t::pg_log_t(pg_log_t const&)+0x109) [0x558eec19df89]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 5: (PGLog::IndexedLog::operator=(PGLog::IndexedLog const&)+0x29) [0x558eec1a8c39]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 6: (void PGLog::read_log_and_missing<pg_missing_set<true> >(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, PGLog::IndexedLog&, pg_missing_set<true>&, bool, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, bool, bool*, DoutPrefixProvider const*, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*, bool)+0x793) [0x558eec1a9ef3]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 7: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x52b) [0x558eec14bd3b]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 8: (OSD::load_pgs()+0x9b4) [0x558eec099e74]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 9: (OSD::init()+0x2169) [0x558eec0b89b9]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 10: (main()+0x2d07) [0x558eebfba007]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 11: (__libc_start_main()+0xf5) [0x7f3cf5ab3545]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: 12: (()+0x4c0ed3) [0x558eec059ed3]
Feb 24 06:41:01 overcloud-ceph-0 ceph-osd: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Feb 24 06:41:02 overcloud-ceph-0 systemd: ceph-osd@46.service: main process exited, code=killed, status=11/SEGV
Feb 24 06:41:02 overcloud-ceph-0 systemd: Unit ceph-osd@46.service entered failed state.
Feb 24 06:41:02 overcloud-ceph-0 systemd: ceph-osd@46.service failed.
Feb 24 06:41:13 overcloud-ceph-0 collectd: dumped all
Feb 24 06:41:13 overcloud-ceph-0 kernel: reader#0[2042781]: segfault at 24 ip 00007efcad3e36ea sp 00007efc9a7fadd0 error 6 in libpython2.7.so.1.0[7efcad359000+17e000]
Feb 24 06:41:18 overcloud-ceph-0 collectd: dumped monmap epoch 1
Feb 24 06:41:23 overcloud-ceph-0 collectd: dumped all
Feb 24 06:41:23 overcloud-ceph-0 systemd: ceph-osd@46.service holdoff time over, scheduling restart.
Feb 24 06:41:23 overcloud-ceph-0 systemd: Cannot add dependency job for unit rpcbind.socket, ignoring: Unit not found.
Feb 24 06:41:23 overcloud-ceph-0 systemd: Stopped Ceph object storage daemon osd.46.
Feb 24 06:41:23 overcloud-ceph-0 systemd: Starting Ceph object storage daemon osd.46...
Feb 24 06:41:24 overcloud-ceph-0 systemd: Started Ceph object storage daemon osd.46.

Example for compute nodes affected by this

On compute nodes, this can affected active virtual machines:

var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager Traceback (most recent call last):
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 7426, in update_available_resource_for_node
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     rt.update_available_resource(context, nodename)
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 673, in update_available_resource
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     resources = self.driver.get_available_resource(nodename)
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6457, in get_available_resource
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     data["hypervisor_version"] = self._host.get_version()
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 672, in get_version
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     return self.get_connection().getVersion()
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     result = proxy_call(self._autowrap, f, *args, **kwargs)
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     rv = execute(f, *args, **kwargs)
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     six.reraise(c, e, tb)
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     rv = meth(*args, **kwargs)
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3981, in getVersion
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager     if ret == -1: raise libvirtError ('virConnectGetVersion() failed', conn=self)
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager libvirtError: cannot fork child process: Cannot allocate memory
var/log/containers/nova/nova-compute.log.8:2020-06-18 14:35:34.811 1 ERROR nova.compute.manager

The OOM killer reaps the collectd process which can be seen with a big RSS:

Jun 18 14:35:41 compute-0 kernel: Out of memory: Kill process 529856 (collectd) score 26 or sacrifice child
Jun 18 14:35:41 compute-0 kernel: Killed process 529856 (collectd), UID 0, total-vm:15272768kB, anon-rss:14423004kB, file-rss:808kB, shmem-rss:0kB
Jun 18 14:35:41 compute-0 kernel: docker-current invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
Jun 18 14:35:41 compute-0 kernel: docker-current cpuset=/ mems_allowed=0-1

In the above example, collectd had total_vm 3818192, RSS 3605953 (= [14423004+808] / 4) 4K pages ; or in kB: total-vm:15272768kB, anon-rss:14423004kB, file-rss:808kB, shmem-rss:0kB

Meaning that it consumed ca. 14423004÷1024÷1024=13.7548 GB of anon-rss memory before being killed by the OOM killer.
Note: anon-rss is resident set size in memory for this process and its libraries