Some allocator fixes for kernel

Signed-off-by: Robbie Harwood <rharwood@redhat.com>

0270-Make-debug-file-show-which-file-filters-get-run.patch

@@ -0,0 +1,45 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Peter Jones <pjones@redhat.com>
+Date: Fri, 29 Jul 2022 15:56:00 -0400
+Subject: [PATCH] Make debug=file show which file filters get run.
+
+If one of the file filters breaks things, it's hard to figure out where
+it has happened.
+
+This makes grub log which filter is being run, which makes it easier to
+figure out where you are in the sequence of events.
+
+Signed-off-by: Peter Jones <pjones@redhat.com>
+---
+ grub-core/kern/file.c | 11 +++++++++++
+ 1 file changed, 11 insertions(+)
+
+diff --git a/grub-core/kern/file.c b/grub-core/kern/file.c
+index ed69fc0f0f..3f175630ea 100644
+--- a/grub-core/kern/file.c
++++ b/grub-core/kern/file.c
+@@ -30,6 +30,14 @@ void (*EXPORT_VAR (grub_grubnet_fini)) (void);
+ 
+ grub_file_filter_t grub_file_filters[GRUB_FILE_FILTER_MAX];
+ 
++static char *filter_names[] = {
++    [GRUB_FILE_FILTER_VERIFY] = "GRUB_FILE_FILTER_VERIFY",
++    [GRUB_FILE_FILTER_GZIO] = "GRUB_FILE_FILTER_GZIO",
++    [GRUB_FILE_FILTER_XZIO] = "GRUB_FILE_FILTER_XZIO",
++    [GRUB_FILE_FILTER_LZOPIO] = "GRUB_FILE_FILTER_LZOPIO",
++    [GRUB_FILE_FILTER_MAX] = "GRUB_FILE_FILTER_MAX"
++};
++
+ /* Get the device part of the filename NAME. It is enclosed by parentheses.  */
+ char *
+ grub_file_get_device_name (const char *name)
+@@ -121,6 +129,9 @@ grub_file_open (const char *name, enum grub_file_type type)
+     if (grub_file_filters[filter])
+       {
+ 	last_file = file;
++	if (filter < GRUB_FILE_FILTER_MAX)
++	  grub_dprintf ("file", "Running %s file filter\n",
++			filter_names[filter]);
+ 	file = grub_file_filters[filter] (file, type);
+ 	if (file && file != last_file)
+ 	  {

0271-efi-make-the-default-arena-most-of-ram.patch

@@ -0,0 +1,73 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Peter Jones <pjones@redhat.com>
+Date: Fri, 29 Jul 2022 15:57:57 -0400
+Subject: [PATCH] efi: make the default arena most of ram
+
+Currently when populating the initial memory arena on EFI systems, we
+count the available regions below GRUB_EFI_MAX_ALLOCATION_ADDRESS from
+the EFI memory map and then allocates one quarter of that for our arena.
+
+Because many systems come up without IOMMUs, we currently set
+GRUB_EFI_MAX_ALLOCATION_ADDRESS to 0x7fffffff, i.e. all addresses
+allocated must be below 2G[0].  Due to firmware and other
+considerations, this makes the most memory we can possibly have in our
+arena 512M.
+
+Because our EFI loader doesn't get kernel and initrd memory from grub's
+allocator, but rather reserves it directly from UEFI and then simply
+marks those as allocated if they're within grub's arena, it was
+historically possible to have initrds that are larger than 512M, because
+we could use any memory region below 4G, without concern for grub's
+choice of arena size.
+
+Unfortunately, when we switched to using the "verifiers" API (and thus
+the file_filter_t API) to do measurement of kernel and initrd, this
+introduced a pattern that allocates the entire file when we call
+grub_file_open(), and buffers it to pass to the filter.  This results in
+needing to have enough space for the initramfs in the grub arena.
+
+This is bad.
+
+Since it's unlikely you're going to do anything *other* than loading a
+kernel and initramfs that takes much of the available free memory from
+UEFI, this patch introduces a workaround by changing the amount we give
+to the arena be three quarters of the available memory, rather than one
+quarter, thus changing our theoretical initrd limit to 1.5G.  In
+practice, it may still be smaller than that depending on allocation
+fragmentation, but generally it will be most of it.
+
+Note that this doesn't fix the underlying flaw, which is that there is
+no safe way to do the validation correctly using the "verifiers" system
+with the current file API without buffering the whole file before
+grub_file_read() is ever called, and thus you can't set an allocation
+policy for the initial buffer of the file at all, so unless we raise the
+allocation limit to >4G, it can't be allocated in the big region.
+
+[0] I'm not sure there was a good reason not to pick 4G, but even if we
+    had, at least one common firmware routes the first 2G of physical
+    RAM to 0x0, and any additional memory starting at 0x100000000.
+
+Related: rhbz#2112134
+
+Signed-off-by: Peter Jones <pjones@redhat.com>
+---
+ grub-core/kern/efi/mm.c | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/grub-core/kern/efi/mm.c b/grub-core/kern/efi/mm.c
+index e460b072e6..150e412ee7 100644
+--- a/grub-core/kern/efi/mm.c
++++ b/grub-core/kern/efi/mm.c
+@@ -737,10 +737,10 @@ grub_efi_mm_init (void)
+   filtered_memory_map_end = filter_memory_map (memory_map, filtered_memory_map,
+ 					       desc_size, memory_map_end);
+ 
+-  /* By default, request a quarter of the available memory.  */
++  /* By default, request three quarters of the available memory.  */
+   total_pages = get_total_pages (filtered_memory_map, desc_size,
+ 				 filtered_memory_map_end);
+-  required_pages = (total_pages >> 2);
++  required_pages = (total_pages >> 1) + (total_pages >> 2);
+   if (required_pages < BYTES_TO_PAGES (MIN_HEAP_SIZE))
+     required_pages = BYTES_TO_PAGES (MIN_HEAP_SIZE);
+   else if (required_pages > BYTES_TO_PAGES (MAX_HEAP_SIZE))

0272-efi-use-enumerated-array-positions-for-our-allocatio.patch

@@ -0,0 +1,81 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Peter Jones <pjones@redhat.com>
+Date: Mon, 1 Aug 2022 14:06:30 -0400
+Subject: [PATCH] efi: use enumerated array positions for our allocation
+ choices
+
+In our kernel allocator on EFI systems, we currently have a growing
+amount of code that references the various allocation policies by
+position in the array, and of course maintenance of this code scales
+very poorly.
+
+This patch changes them to be enumerated, so they're easier to refer to
+farther along in the code without confusion.
+
+Signed-off-by: Peter Jones <pjones@redhat.com>
+---
+ grub-core/loader/i386/efi/linux.c | 31 ++++++++++++++++++++-----------
+ 1 file changed, 20 insertions(+), 11 deletions(-)
+
+diff --git a/grub-core/loader/i386/efi/linux.c b/grub-core/loader/i386/efi/linux.c
+index 91ae274299..8daa070132 100644
+--- a/grub-core/loader/i386/efi/linux.c
++++ b/grub-core/loader/i386/efi/linux.c
+@@ -60,17 +60,26 @@ struct allocation_choice {
+     grub_efi_allocate_type_t alloc_type;
+ };
+ 
+-static struct allocation_choice max_addresses[4] =
++enum {
++    KERNEL_PREF_ADDRESS,
++    KERNEL_4G_LIMIT,
++    KERNEL_NO_LIMIT,
++};
++
++static struct allocation_choice max_addresses[] =
+   {
+     /* the kernel overrides this one with pref_address and
+      * GRUB_EFI_ALLOCATE_ADDRESS */
+-    { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
++    [KERNEL_PREF_ADDRESS] =
++      { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
++    /* If the flag in params is set, this one gets changed to be above 4GB. */
++    [KERNEL_4G_LIMIT] =
++      { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
+     /* this one is always below 4GB, which we still *prefer* even if the flag
+      * is set. */
+-    { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
+-    /* If the flag in params is set, this one gets changed to be above 4GB. */
+-    { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
+-    { 0, 0 }
++    [KERNEL_NO_LIMIT] =
++      { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
++    { NO_MEM, 0, 0 }
+   };
+ static struct allocation_choice saved_addresses[4];
+ 
+@@ -405,7 +414,7 @@ grub_cmd_linux (grub_command_t cmd __attribute__ ((unused)),
+   if (lh->xloadflags & LINUX_XLF_CAN_BE_LOADED_ABOVE_4G)
+     {
+       grub_dprintf ("linux", "Loading kernel above 4GB is supported; enabling.\n");
+-      max_addresses[2].addr = GRUB_EFI_MAX_USABLE_ADDRESS;
++      max_addresses[KERNEL_NO_LIMIT].addr = GRUB_EFI_MAX_USABLE_ADDRESS;
+     }
+   else
+     {
+@@ -478,11 +487,11 @@ grub_cmd_linux (grub_command_t cmd __attribute__ ((unused)),
+   grub_dprintf ("linux", "lh->pref_address: %p\n", (void *)(grub_addr_t)lh->pref_address);
+   if (lh->pref_address < (grub_uint64_t)GRUB_EFI_MAX_ALLOCATION_ADDRESS)
+     {
+-      max_addresses[0].addr = lh->pref_address;
+-      max_addresses[0].alloc_type = GRUB_EFI_ALLOCATE_ADDRESS;
++      max_addresses[KERNEL_PREF_ADDRESS].addr = lh->pref_address;
++      max_addresses[KERNEL_PREF_ADDRESS].alloc_type = GRUB_EFI_ALLOCATE_ADDRESS;
+     }
+-  max_addresses[1].addr = GRUB_EFI_MAX_ALLOCATION_ADDRESS;
+-  max_addresses[2].addr = GRUB_EFI_MAX_ALLOCATION_ADDRESS;
++  max_addresses[KERNEL_4G_LIMIT].addr = GRUB_EFI_MAX_ALLOCATION_ADDRESS;
++  max_addresses[KERNEL_NO_LIMIT].addr = GRUB_EFI_MAX_ALLOCATION_ADDRESS;
+   kernel_size = lh->init_size;
+   kernel_mem = kernel_alloc (kernel_size, GRUB_EFI_RUNTIME_SERVICES_CODE,
+ 			     N_("can't allocate kernel"));

0273-efi-split-allocation-policy-for-kernel-vs-initrd-mem.patch

@@ -0,0 +1,127 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Peter Jones <pjones@redhat.com>
+Date: Mon, 1 Aug 2022 14:24:39 -0400
+Subject: [PATCH] efi: split allocation policy for kernel vs initrd memories.
+
+Currently in our kernel allocator, we use the same set of choices for
+all of our various kernel and initramfs allocations, though they do not
+have exactly the same constraints.
+
+This patch adds the concept of an allocation purpose, which currently
+can be KERNEL_MEM or INITRD_MEM, and updates kernel_alloc() calls
+appropriately, but does not change any current policy decision.  It
+also adds a few debug prints.
+
+Signed-off-by: Peter Jones <pjones@redhat.com>
+---
+ grub-core/loader/i386/efi/linux.c | 35 +++++++++++++++++++++++++++--------
+ 1 file changed, 27 insertions(+), 8 deletions(-)
+
+diff --git a/grub-core/loader/i386/efi/linux.c b/grub-core/loader/i386/efi/linux.c
+index 8daa070132..e6b8998e5e 100644
+--- a/grub-core/loader/i386/efi/linux.c
++++ b/grub-core/loader/i386/efi/linux.c
+@@ -55,7 +55,14 @@ struct grub_linuxefi_context {
+ 
+ #define BYTES_TO_PAGES(bytes)   (((bytes) + 0xfff) >> 12)
+ 
++typedef enum {
++    NO_MEM,
++    KERNEL_MEM,
++    INITRD_MEM,
++} kernel_alloc_purpose_t;
++
+ struct allocation_choice {
++    kernel_alloc_purpose_t purpose;
+     grub_efi_physical_address_t addr;
+     grub_efi_allocate_type_t alloc_type;
+ };
+@@ -64,6 +71,7 @@ enum {
+     KERNEL_PREF_ADDRESS,
+     KERNEL_4G_LIMIT,
+     KERNEL_NO_LIMIT,
++    INITRD_MAX_ADDRESS,
+ };
+ 
+ static struct allocation_choice max_addresses[] =
+@@ -71,14 +79,17 @@ static struct allocation_choice max_addresses[] =
+     /* the kernel overrides this one with pref_address and
+      * GRUB_EFI_ALLOCATE_ADDRESS */
+     [KERNEL_PREF_ADDRESS] =
+-      { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
++      { KERNEL_MEM, GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
+     /* If the flag in params is set, this one gets changed to be above 4GB. */
+     [KERNEL_4G_LIMIT] =
+-      { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
++      { KERNEL_MEM, GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
+     /* this one is always below 4GB, which we still *prefer* even if the flag
+      * is set. */
+     [KERNEL_NO_LIMIT] =
+-      { GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
++      { KERNEL_MEM, GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
++    /* this is for the initrd */
++    [INITRD_MAX_ADDRESS] =
++      { INITRD_MEM, GRUB_EFI_MAX_ALLOCATION_ADDRESS, GRUB_EFI_ALLOCATE_MAX_ADDRESS },
+     { NO_MEM, 0, 0 }
+   };
+ static struct allocation_choice saved_addresses[4];
+@@ -95,7 +106,8 @@ kernel_free(void *addr, grub_efi_uintn_t size)
+ }
+ 
+ static void *
+-kernel_alloc(grub_efi_uintn_t size,
++kernel_alloc(kernel_alloc_purpose_t purpose,
++	     grub_efi_uintn_t size,
+ 	     grub_efi_memory_type_t memtype,
+ 	     const char * const errmsg)
+ {
+@@ -108,6 +120,9 @@ kernel_alloc(grub_efi_uintn_t size,
+       grub_uint64_t max = max_addresses[i].addr;
+       grub_efi_uintn_t pages;
+ 
++      if (purpose != max_addresses[i].purpose)
++	continue;
++
+       /*
+        * When we're *not* loading the kernel, or >4GB allocations aren't
+        * supported, these entries are basically all the same, so don't re-try
+@@ -261,7 +276,8 @@ grub_cmd_initrd (grub_command_t cmd, int argc, char *argv[])
+ 	}
+     }
+ 
+-  initrd_mem = kernel_alloc(size, GRUB_EFI_RUNTIME_SERVICES_DATA,
++  grub_dprintf ("linux", "Trying to allocate initrd mem\n");
++  initrd_mem = kernel_alloc(INITRD_MEM, size, GRUB_EFI_RUNTIME_SERVICES_DATA,
+ 			    N_("can't allocate initrd"));
+   if (initrd_mem == NULL)
+     goto fail;
+@@ -422,7 +438,8 @@ grub_cmd_linux (grub_command_t cmd __attribute__ ((unused)),
+     }
+ #endif
+ 
+-  params = kernel_alloc (sizeof(*params), GRUB_EFI_RUNTIME_SERVICES_DATA,
++  params = kernel_alloc (KERNEL_MEM, sizeof(*params),
++			 GRUB_EFI_RUNTIME_SERVICES_DATA,
+ 			 "cannot allocate kernel parameters");
+   if (!params)
+     goto fail;
+@@ -445,7 +462,7 @@ grub_cmd_linux (grub_command_t cmd __attribute__ ((unused)),
+   grub_dprintf ("linux", "new lh is at %p\n", lh);
+ 
+   grub_dprintf ("linux", "setting up cmdline\n");
+-  cmdline = kernel_alloc (lh->cmdline_size + 1,
++  cmdline = kernel_alloc (KERNEL_MEM, lh->cmdline_size + 1,
+ 			  GRUB_EFI_RUNTIME_SERVICES_DATA,
+ 			  N_("can't allocate cmdline"));
+   if (!cmdline)
+@@ -493,7 +510,9 @@ grub_cmd_linux (grub_command_t cmd __attribute__ ((unused)),
+   max_addresses[KERNEL_4G_LIMIT].addr = GRUB_EFI_MAX_ALLOCATION_ADDRESS;
+   max_addresses[KERNEL_NO_LIMIT].addr = GRUB_EFI_MAX_ALLOCATION_ADDRESS;
+   kernel_size = lh->init_size;
+-  kernel_mem = kernel_alloc (kernel_size, GRUB_EFI_RUNTIME_SERVICES_CODE,
++  grub_dprintf ("linux", "Trying to allocate kernel mem\n");
++  kernel_mem = kernel_alloc (KERNEL_MEM, kernel_size,
++			     GRUB_EFI_RUNTIME_SERVICES_CODE,
+ 			     N_("can't allocate kernel"));
+   restore_addresses();
+   if (!kernel_mem)

grub.patches

@@ -267,3 +267,7 @@ Patch0266: 0266-nx-set-the-nx-compatible-flag-in-EFI-grub-images.patch
 Patch0267: 0267-grub-probe-document-the-behavior-of-multiple-v.patch
 Patch0268: 0268-grub_fs_probe-dprint-errors-from-filesystems.patch
 Patch0269: 0269-fs-fat-don-t-error-when-mtime-is-0.patch
+Patch0270: 0270-Make-debug-file-show-which-file-filters-get-run.patch
+Patch0271: 0271-efi-make-the-default-arena-most-of-ram.patch
+Patch0272: 0272-efi-use-enumerated-array-positions-for-our-allocatio.patch
+Patch0273: 0273-efi-split-allocation-policy-for-kernel-vs-initrd-mem.patch

grub2.spec

@@ -17,7 +17,7 @@
 Name:		grub2
 Epoch:		1
 Version:	2.06
-Release:	43%{?dist}
+Release:	44%{?dist}
 Summary:	Bootloader with support for Linux, Multiboot and more
 License:	GPLv3+
 URL:		http://www.gnu.org/software/grub/
@@ -530,6 +530,9 @@ mv ${EFI_HOME}/grub.cfg.stb ${EFI_HOME}/grub.cfg
 %endif
 
 %changelog
+* Mon Aug 01 2022 Robbie Harwood <rharwood@redhat.com> - 2.06-44
+- Some allocator fixes for kernel
+
 * Tue Jul 19 2022 Robbie Harwood <rharwood@redhat.com> - 2.06-43
 - Handle FAT mtime of 0
 - Resolves: #2096192