lwn.net

The mmap_sem lock used in the memory-management subsystem has been a known scalability problem for years, but it has proved difficult to remove. During a session in the memory-management track of the 2019 Linux Storage, Filesystem, and Memory-Management Summit, Laurent Dufour and Matthew Wilcox discussed a possible solution: replacing the red-black tree currently used to track virtual memory areas (VMAs) with a new data structure called a "maple tree".

For years, kernel developers have been told to avoid allocating large chunks of physically contiguous memory; as the system runs and memory becomes fragmented, satisfying such allocations becomes increasingly difficult. But, as Zi Yan pointed out in a memory-management track session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit, there are times when contiguous memory is useful. In this session, the memory-management developers discussed ways to make such allocations more likely to succeed.

kmalloc() is one of the kernel's fundamental memory-allocation primitives for relatively small objects. Most of the time, developers don't worry about the alignment of memory returned from kmalloc(), and things generally just work. But, Vlastimil Babka said during a plenary session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit, every now and then kmalloc() will do something surprising. He proposed tightening the guarantees around object alignment in the hope of generating fewer surprises in the future.

At the 2018 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Boaz Harrosh introduced the ZUFS filesystem. At this year's event, he was back to talk about what it would take to merge ZUFS into the mainline. ZUFS, which Harrosh pronounced as both "zoo-eff-ess" and "zoofs", has been running in production for his employer's (NetApp's) customers for some time now, so he wondered if it was something that could go upstream.

Stable kernels 5.0.14, 4.19.41, 4.14.117, and 4.9.174 have been released. As usual there are important fixes and users should upgrade.

Security updates have been issued by Arch Linux (dovecot, kernel, linux-zen, munin, nautilus, perl-email-address, and tcpreplay), Debian (atftp), Fedora (perl-YAML and teeworlds), Mageia (java-1.8.0-openjdk, ldb, libsolv, and putty/filezilla/wxgtk), openSUSE (freeradius-server, libjpeg-turbo, pacemaker, rubygem-actionpack-5_1, wpa_supplicant, and yubico-piv-tool), Red Hat (chromium-browser, container-tools:rhel8, edk2, firefox, flatpak, ghostscript, httpd:2.4, mod_auth_mellon, openwsman, python-jinja2, python27:2.7, python3, python36:3.6, redhat-virtualization-host, systemd, and wget), SUSE (freeradius-server), and Ubuntu (ghostscript and wpa).

One fixture of the memory-management track at the Linux Storage, Filesystem, and Memory-Management Summit is a discussion with subsystem maintainer Andrew Morton on how the development process is going. The 2019 version indicated that the memory-management developers are mostly happy with how the process is working, but there are still things that they would like to see changed. While some of the issues are old and intractable, others may be amenable to short-term improvement.

The problems associated with the kernel's internal get_user_pages() function have been a topic of discussion at the Linux Storage, Filesystem, and Memory-Management Summit for a few years. At the 2019 event, Jan Kara began a plenary session by saying that it would be "like last year's session". It turned out rather differently, though, perhaps due to the plenary setting; this discussion (along with the related session that followed) turned out to be one of the most heated at the entire conference.

Red Hat has announced the release of Red Hat Enterprise Linux 8. "Modern IT is hybrid IT. But turning a sprawling ecosystem—from traditional datacenters to public cloud services—into a true hybrid environment requires a few things. Scaling as needed. Moving workloads seamlessly. Developing and managing applications that run anywhere. There's an operating system that makes those things possible. And now it gives you predictive analytics and remediation." See the release notes for more information.

Memory control groups exist to track and limit the amount of memory used by sets of processes. Normally, one would not expect that memory used by one group would be charged to another but, as Shakeel Butt described in a memory-management track session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit, that does happen in a number of different situations. It's often a problem, but occasionally it's also a useful feature.

Security updates have been issued by Debian (389-ds-base, firefox-esr, and symfony), Fedora (poppler), SUSE (audit, ovmf, and webkit2gtk3), and Ubuntu (aria2, FFmpeg, gnome-shell, and sudo).

Control groups are a useful mechanism for managing resource usage in the system, but what happens when the control groups themselves become a resource problem? In a plenary session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit, Roman Gushchin described problems he has been facing with deleted control groups that take their time before actually going away. Some of these problems have been fixed, but the issue has not been truly resolved.

Shakeel Butt started his 2019 Linux Storage, Filesystem, and Memory-Management Summit session by noting that memory makes up a big part of the total cost of equipping a data center. As a result, data-center operators try to make the best use of memory they can, generally overcommitting it significantly. In this session, Butt described a scheme in use at Google to try to improve memory utilization; while the need for the described functionality was generally agreed upon, the developers in the room were not entirely happy with the solution presented.

The performance of the fget() function in the kernel was the topic of a discussion led by Dave Watson at the 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM). fget() is used to take a reference to a file (i.e. bump a reference count), based on its file descriptor, and to return the struct file pointer for it; references are dropped with fput(). Some recent profiling at Watson's employer, Facebook, found the function to be taking a sizable portion of the CPU time for some applications, so he wanted to talk about some of the things he has tried to make that situation better.

In a combined filesystem and storage session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit (LSFMM), Dennis Zhou wanted to talk about discard, which is the process of sending commands (e.g. TRIM) to block devices to indicate blocks that are no longer in use. Discard is a "serious black box", he said; it is a third way to interact with a drive, but Linux developers have no real insight into what its actual effects will be. That can lead to performance and other problems.

While persistent memory is normally valued for its persistence, there is also a subcurrent of interest in using it in settings where persistence is not important. In particular, the fact that this memory is relatively inexpensive makes it appealing to use instead of ordinary RAM in budget-conscious settings. At the 2019 Linux Storage, Filesystem, and Memory-Management Summit, two sessions in the memory-management track looked at how the kernel's NUMA mechanism could be pressed into service to manage non-persistent uses of persistent memory.

Sometimes, the kernel's no-regression rule may not have the desired result. Andrea Arcangeli led a session at the 2019 Linux Storage, Filesystem, and Memory-Management Summit to make the point that the recent reversion of a fix after a performance regression was reported has led to worse performance overall — with, as is his wont, a lot of technical information to back up that point. With a wider understanding of what is at stake here, he hopes, the reversion can itself be reverted.

Security updates have been issued by Debian (jquery, librecad, and phpbb3), Fedora (bubblewrap, java-11-openjdk, libvirt, openssh, and pacemaker), Mageia (virtualbox), openSUSE (chromium, ImageMagick, and java-11-openjdk), and SUSE (openssl-1_1).

There is a new Firefox browser release available; its main claim to fame is that it has a fix for the certificate issue that disabled all extensions.

Linus has released the 5.1 kernel, right on schedule. Some of the significant changes in the release include BPF spinlocks, more year-2038 preparation, the TEO CPU-frequency governor, The io_uring fast asynchronous I/O mechanism, initial support for pidfds (file descriptors that refer to a process), the SafeSetID security module, and much more. See the KernelNewbies 5.1 page for lots of details.