vn_getf/vn_releasef should not accept negative file descriptors

C type coercion rules require that negative numbers be converted into
positive numbers via wraparound such that a negative -1 becomes a
positive 1. This causes vn_getf to return a file handle when it should
return NULL whenever a positive file descriptor existed with the same
value. We should check for a negative file descriptor and return NULL
instead.

This was caught by ClusterHQ's unit testing.

Reference:
http://stackoverflow.com/questions/50605/signed-to-unsigned-conversion-in-c-is-it-always-safe

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Andriy Gapon <avg@FreeBSD.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #450
This commit is contained in:
Richard Yao 2015-04-16 10:29:41 -04:00 committed by Brian Behlendorf
parent cd69f020e4
commit 313b1ea622

View File

@ -648,6 +648,9 @@ vn_getf(int fd)
vnode_t *vp;
int rc = 0;
if (fd < 0)
return (NULL);
/* Already open just take an extra reference */
spin_lock(&vn_file_lock);
@ -733,6 +736,9 @@ vn_releasef(int fd)
{
file_t *fp;
if (fd < 0)
return;
spin_lock(&vn_file_lock);
fp = file_find(fd);
if (fp) {