fbmem: don't allow too huge resolutions

commit 8c28051cdc upstream.

syzbot is reporting page fault at vga16fb_fillrect() [1], for
vga16fb_check_var() is failing to detect multiplication overflow.

  if (vxres * vyres > maxmem) {
    vyres = maxmem / vxres;
    if (vyres < yres)
      return -ENOMEM;
  }

Since no module would accept too huge resolutions where multiplication
overflow happens, let's reject in the common path.

Link: https://syzkaller.appspot.com/bug?extid=04168c8063cfdde1db5e [1]
Reported-by: syzbot <syzbot+04168c8063cfdde1db5e@syzkaller.appspotmail.com>
Debugged-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: stable@vger.kernel.org
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/185175d6-227a-7b55-433d-b070929b262c@i-love.sakura.ne.jp
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This commit is contained in:
Tetsuo Handa
2021-09-08 19:27:49 +09:00
committed by Greg Kroah-Hartman
parent 5797d00212
commit 6e5f7ead44

View File

@@ -32,6 +32,7 @@
#include <linux/device.h>
#include <linux/efi.h>
#include <linux/fb.h>
#include <linux/overflow.h>
#include <asm/fb.h>
@@ -981,6 +982,7 @@ fb_set_var(struct fb_info *info, struct fb_var_screeninfo *var)
if ((var->activate & FB_ACTIVATE_FORCE) ||
memcmp(&info->var, var, sizeof(struct fb_var_screeninfo))) {
u32 activate = var->activate;
u32 unused;
/* When using FOURCC mode, make sure the red, green, blue and
* transp fields are set to 0.
@@ -1005,6 +1007,11 @@ fb_set_var(struct fb_info *info, struct fb_var_screeninfo *var)
if (var->xres < 8 || var->yres < 8)
return -EINVAL;
/* Too huge resolution causes multiplication overflow. */
if (check_mul_overflow(var->xres, var->yres, &unused) ||
check_mul_overflow(var->xres_virtual, var->yres_virtual, &unused))
return -EINVAL;
ret = info->fbops->fb_check_var(var, info);
if (ret)