On a sunny day (Fri, 24 Jan 2020 11:19:03 -0500) it happened Dennis Lee Bieber
wrote in :
>On Fri, 24 Jan 2020 08:00:58 GMT, Jan Panteltje
>declaimed the following:
>
>>
>>
>>kernel claims all memory space as cache, that leaves virtually nothing (here
only 71.6 MB) for programs
>>those programs then start swapping!
>>
> While the OS has claimed 3.4GB as cache, note the line below reports
>3.3GB as available -- meaning the OS can release that much of the cache for
>use by applications. After all, the R-Pi doesn't have 6.7+GB of memory.
>
>>
>>BAD BAD BAD
>>Linux error!!!!
>>
>>Big thread about it here with possible solutions:
>>
https://unix.stackexchange.com/questions/253816/restrict-size-of-buffer-cache-i
n-linux
>>
>
> Note that the first entry on that link complains about how long it
>takes for the cache to be freed, thereby delaying the startup of fresh
>applications. It does not implicate a situation with page swapping starting
>up. The first "answer" refers to /file system buffers/ -- eg; output that
>has not yet been forced to the file system (SD card); and with a journaling
>file system, file system updates take multiple stages (things like:
>allocate fresh blocks and write data; write journal log indicating where
>the new data is; later update file system so log contents are now active --
>read journal/write primary file system, remove relevant journal log
>information).
>
> I really doubt that the cache usage part of the kernel has been
changed
>recently in a way that it does not free pages when new applications are
>started.
>
> In your original process list, you show FOURTEEN rxvt (X-Window
>terminal) processes, along with SSH and ONE xterm. My suspicion is that
>something you are doing is invoking terminals, but never closing them.
>Surely you aren't really running 14+ terminals at once! Each terminal
>likely having a few pages dedicated to the "screen contents" of the
>terminal (the main code is no doubt shared, or you'd have even more memory
>tagged "used").
Yes, a number of scripts (starting with your last remark) are running,
below is one that checks for a flag and then starts a new recording of a
camera:
flag is set from crontab ever few hours
so I get nice bacth of recordings like this
-rw-r--r-- 1 root root 153587406 Jan 24 19:00 camera6-270.wav
-rw-r--r-- 1 root root 943014702 Jan 24 19:00 camera6-270.avi
-rw-r--r-- 1 root root 1081724392 Jan 24 19:00 camera4-275.avi
-rw-r--r-- 1 root root 5560318832 Jan 25 02:00 hcam_3_504.ts
-rw-r--r-- 1 root root 5608550044 Jan 25 02:00 hcam_4_335.ts
-rw-r--r-- 1 root root 5605979896 Jan 25 02:00 hcam_2_434.ts
-rw-r--r-- 1 root root 5607202836 Jan 25 02:00 hcam_1_487.ts
-rw-r--r-- 1 root root 1080950850 Jan 25 07:00 camera4-276.avi
-rw-r--r-- 1 root root 176570366 Jan 25 07:00 camera6-271.wav
-rw-r--r-- 1 root root 1081851124 Jan 25 07:00 camera6-271.avi
-rw-r--r-- 1 root root 1328002836 Jan 25 07:53 bp65.avi
-rw-r--r-- 1 root root 79959598 Jan 25 07:53 camera6-272.avi
-rw-r--r-- 1 root root 13107294 Jan 25 07:53 camera6-272.wav
-rw-r--r-- 1 root root 80483886 Jan 25 07:53 camera4-277.avi
-rw-r--r-- 1 root root 4062141192 Jan 25 07:53 hcam_3_505.ts
-rw-r--r-- 1 root root 4135147984 Jan 25 07:53 hcam_2_435.ts
-rw-r--r-- 1 root root 577800268 Jan 25 07:53 e2500_8.mkv
-rw-r--r-- 1 root root 4138018368 Jan 25 07:53 hcam_1_488.ts
-rw-r--r-- 1 root root 4138335524 Jan 25 07:53 hcam_4_336.ts
Example of such a script:
raspi95: /usr/local/sbin # cat test-update-camera6
#!/bin/bash
while true
do
if test -f /video/camera6-update-flag
then
killall -KILL wget2
killall -KILL mcamip2
serial_number=/bin/cat /video/camera6_serial.txt
#/bin/echo "serial_number=$serial_number"
wget2 --user=panteltje --password=1234567
http://192.168.178.160:84/videostream.asf -O - 2>/dev/zero \
| \
ffmpeg -f asf -i - -acodec copy -y
/mnt/sda2/security/video/camera6-$serial_number.wav \
1>/dev/zero 2>/dev/zero &
mcamip2 -a 192.168.178.160 -p 84 -u panteltje -w 1234567
-x -t -f 2 -q -m 1 -y 2>/dev/zero \
| \
ffmpeg -f yuv4mpegpipe -i - -vcodec mjpeg -an -y
/mnt/sda2/security/video/camera6-$serial_number.avi \
1>/dev/zero 2>/dev/zero &
let old_serial_number=serial_number-6
# This file may still be there if reboot.
let older_serial_number=serial_number-7
#/bin/echo "updating serial number"
let serial_number=serial_number+1
/bin/echo $serial_number > /video/camera6_serial.txt
#/bin/echo "removing old file
/video/camera6-$old_serial_number.*"
rm /mnt/sda2/security/video/camera6-$old_serial_number.avi
rm /mnt/sda2/security/video/camera6-$old_serial_number.wav
#/bin/echo "removing older file
/video/camera6-$older_serial_number.*"
rm /video/camera6-$older_serial_number.avi 2>/dev/zero
rm /video/camera6-$older_serial_number.wav 2>/dev/zero
rm /video/camera6-update-flag
fi
sleep 54
done
There are 9 terminals running, and in each terminal a zsh shell is running
Those terminals are always on (9 virtual desktops, xfm, fvwm so not the usual
interface,
that same configuration runs (apart that there it is native zsh) on all my
computahs.
As to 'does it help etc?'
I am monitoring now the swap file it seems not to be getting bigger with the
modifications I
mentioned in the other posting.
Fri 24 Jan 2020 03:10:42 PM CET:
MiB Mem : 3906.0 total, 2891.6 free, 378.2 used, 636.2 buff/cache
MiB Swap: 100.0 total, 20.4 free, 79.6 used. 3378.8 avail Mem
Sat 25 Jan 2020 06:46:05 AM CET
MiB Mem : 3906.0 total, 2653.7 free, 372.0 used, 880.4 buff/cache
MiB Swap: 100.0 total, 24.7 free, 75.3 used. 3380.4 avail Mem
so that is good!
It will (hopefully) keep running for a long time (weeks etc is on an UPS)
and will check every day what now happens to the swap file.
Normally it runs via ssh, but I confess I have used it with HDMI monitor
to do all sort of other things such as web browsing, watching youtube videos,
recording satellite TV,
etc,
Raspi 4 is a very powerful thing, can indeed for many applications replace a
desktop PC,
have a 3.4 TB USB harddisk connected
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 29771644 14374360 14106048 51% /
devtmpfs 1867796 0 1867796 0% /dev
tmpfs 1999892 0 1999892 0% /dev/shm
tmpfs 1999892 33484 1966408 2% /run
tmpfs 5120 8 5112 1% /run/lock
tmpfs 1999892 0 1999892 0% /sys/fs/cgroup
/dev/mmcblk0p1 258095 53032 205064 21% /boot
tmpfs 399976 0 399976 0% /run/user/1000
/dev/sda2 3844510712 2643749788 1005400260 73% /mnt/sda2
tmpfs 399976 0 399976 0% /run/user/0
We will see where it goes :-)
--- SoupGate-Win32 v1.05
* Origin: Agency HUB, Dunedin - New Zealand | FidoUsenet Gateway (3:770/3)
|