media: v4l2-common: fix overflow in v4l_bound_align_image()
authorMaciej Matraszek <m.matraszek@samsung.com>
Mon, 15 Sep 2014 08:14:48 +0000 (05:14 -0300)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Fri, 14 Nov 2014 16:47:56 +0000 (08:47 -0800)
commited070066305551c2bf71b0737fcc5fe3e136aed6
tree390ff094f61bc3d654f160252a6f5370b1e1bcd0
parent3b163cb4c566c773d049648ec8da752d37f5c200
media: v4l2-common: fix overflow in v4l_bound_align_image()

commit 3bacc10cd4a85bc70bc0b6c001d3bf995c7fe04c upstream.

Fix clamp_align() used in v4l_bound_align_image() to prevent overflow
when passed large value like UINT32_MAX.

 In the current implementation:
    clamp_align(UINT32_MAX, 8, 8192, 3)

returns 8, because in line:

    x = (x + (1 << (align - 1))) & mask;

x overflows to (-1 + 4) & 0x7 = 3, while expected value is 8192.

v4l_bound_align_image() is heavily used in VIDIOC_S_FMT and
VIDIOC_SUBDEV_S_FMT ioctls handlers, and documentation of the latter
explicitly states that:

"The modified format should be as close as possible to the original
request."
  -- http://linuxtv.org/downloads/v4l-dvb-apis/vidioc-subdev-g-fmt.html

Thus one would expect, that passing UINT32_MAX as format width and
height will result in setting maximum possible resolution for the
device. Particularly, when the driver doesn't support
VIDIOC_ENUM_FRAMESIZES ioctl, which is common in the codebase.

Fixes changeset: b0d3159be9a3

Signed-off-by: Maciej Matraszek <m.matraszek@samsung.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
drivers/media/v4l2-core/v4l2-common.c