vfs: work around the race between vget() and vnlru

Specifically, do not let vtryrecycle() to recycle a used vnode. It is
possible for a vnode to be vref-ed or vuse-ed lockless after it is held
by vhold_recycle_free(). Then, since vtryrecycle() does not recheck the
hold count, we might end up freeing vused vnode.

Since vget_finish() increments v_usecount after obtaining the vnode
lock, we would observe the hold reference anyway when the parallel
vget() is blocked waiting on the vnode lock.

PR:	281749
Reported and tested by:	Steve Peurifoy <ssw01@mathistry.net>, Vladimir Grebenshchikov <vova@zote.me>
Reviewed by:	olce
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D57305

sys/kern/vfs_subr.c

@@ -1936,9 +1936,14 @@ vtryrecycle(struct vnode *vp, bool isvnlru)
 	 * anyone picked up this vnode from another list.  If not, we will
 	 * mark it with DOOMED via vgonel() so that anyone who does find it
 	 * will skip over it.
+	 *
+	 * We cannot check only for v_usecount > 0 there, since
+	 * v_usecount increment is lockless.  Instead check for
+	 * v_holdcnt > 1, with the side effect that a parallel vhold()
+	 * also aborts freeing this vnode.
 	 */
 	VI_LOCK(vp);
-	if (vp->v_usecount) {
+	if (vp->v_holdcnt > 1) {
 		VOP_UNLOCK(vp);
 		vdropl_recycle(vp);
 		vn_finished_write(vnmp);