Move is_restricted_var() to libsa/environment.c so it can be leveraged
by boot_setenv called from subr_boot with not truted input.
Also, allow for local tuning via ENV_IS_RESTRICTED_ALLOWED_LIST and
ENV_IS_RESTRICTED_LIST
Sponsored by: Hewlett Packard Enterprise Development LP.
Reviewed by: kevans, imp
Differential Revision: https://reviews.freebsd.org/D56287
Notable upstream pull request merges:
#17000 -multiple cmd/zfs: support creating parents with canmount=no
#17926b481a8bbb Make zpool status dedup table support raw bytes -p output
#18238 -multiple Report space metrics per allocation class
#182405dad9459d Add --no-preserve-encryption flag
#182571eace5906 libzfs: use mount_setattr for selective remount including
legacy mounts
#182591e2c94a04 More consistent use of TREE_* macros in AVL comparators
#18262c58b8b7dc Fix check for .cfi_negate_ra_state on aarch64
#18264f109c7bb9 Add the --file-layout (-f) option to zdb(8)
#18268304de7f19 libzfs: handle EDOM error in zpool_create
#18275753f1e1e2 zstream: add a drop_record subcommand
#182760f90a797d Fix vdev_rebuild_range() tx commit
#182771e1d64d66 Fix log vdev removal issues
#1827806b0abfe6 Fix the send --exclude option to work with encryption
#18281e73ada771 libzpool: lift zfs_file ops out to separate source file
#18284d97945776 zstream: consolidate shared code
#18285 -multiple Separate SIMD compile checks for userspace and kernel
#18288c5905b2cb Implement lzc_send_progress
#18289 -multiple L2ARC: Add depth cap and write budget fairness for
persistent markers
#18290d45c8d648 FreeBSD: Improve dmesg kernel message prefix
#18294b403040c4 draid: fix data corruption after disk clear
#18296 -multiple libzfs: mnttab cleanup
#18300d35951b18 zpool clear: remove undocumented rewind flags
#18301ae7fcd5f9 fix libzfs diff mem leak in an error path
#1830365165df12 zilstat: add man page
#183147f65e04ab libzfs: scrub: only include start and end nv pairs if
needed for scrub
#18315 -multiple Sync ZVOLs block cloning conditions with file systems
#18330f80338fac zarcsummary: add man page
#18337f259a47c7 zpool-iostat.8: clarify first report shows per-second
averages
Obtained from: OpenZFS
OpenZFS commit: 3ee08abd2f
Notable upstream pull request merges:
#173584975430cf Add vdev property to disable vdev scheduler
#18031c77f17b75 Add snapshots_changed_nsecs dataset property
#18080dbb3f247e cmd/zfs: clone: accept `-u` to not mount newly created
datasets
#18089 -multiple Zstd: Update bundled library to version 1.5.7
#180912301755df Fix zfs_open() to skip zil_async_to_sync() for the
snapshot
#18093 -multiple L2ARC: Rework write throttling with DWPD rate limiting
and parallel writes
#180952dbd6af5e Rename several printf attributes declarations to
__printf__
#180968605bdfdd FreeBSD: unbreak compilation on i386
#18105794f1587d When receiving a stream with the large block flag,
activate feature
#18115765929cb4 DDT: Add locking for table ZAP destruction
#1811809e4e01e9 Fix history logging for `zpool create -t`
#181192f1f25217 icp: emit .note.GNU-stack section for all ELF targets
#181313fffe4e70 Fix --enable-invariants on FreeBSD
#18133d2f5cb3a5 Move range_tree, btree, highbit64 to common code
#1813654b141fab FreeBSD: Remove references to DEBUG_VFS_LOCKS
#18138cdf89f413 Flush RRD only when TXGs contain data
#18139a157ef62a Make sure we can still write data to txg
#18140cd895f0e5 remove thread unsafe debug code causing FreeBSD double
free panic
#181444f180e095 Fix activating large_microzap on receive
#1814635b2d3970 Lock db_mtx around arc_release() in couple places
#18154b36472052 nvpair: chase FreeBSD xdrproc_t definition
#1816021bbe7cb6 Improve caching for dbuf prefetches
#18177 -multiple Multihost Improvements
#181792646bd558 Allow rewrite skip cloned and snapshotted blocks
#18180aa29455dd Restrict cloning with different properties
#18184040ba7a7c libzfs: improve error message for zpool create with
ENXIO
#181881412bdc6c zfs_vnops_os.c: Move a vput() to after
zfs_setattr_dir()
#18198cc184fe98 Fix `send:raw` permission for send `-w -I`
#18208ba970eb20 Cleanup allocation class selection
#182120f9564e85 Simplify dnode_level_is_l2cacheable()
#18214370570890 Remove parent ZIO from dbuf_prefetch()
#18218bfb276e55 freebsd: Fix TIMESPEC_OVERFLOW for PowerPC
#18222d06a1d9ac Fix available space accounting for special/dedup
#18225d48967728 ICP: AES-GCM VAES-AVX2: fix typos and document
source files
#18226c8a72a27e ICP: AES-GCM assembly: remove unused Gmul functions
#18230 -multiple Fix zdb --key crash for unencrypted datasets, and
teach tests to understand this better
#18233 -multiple icp: add SHA-512 implementation using Intel SHA512
extension
#18245991fc56fa Introduce dedupused/dedupsaved pool properties
#182516a717f31e Improve misleading error messages for
ZPOOL_STATUS_CORRUPT_POOL
#182547744f0496 SIMD: libspl: test the correct CPUID bit for AVX512VL
#182556495dafd5 range_tree: use zfs_panic_recover() for
partial-overlap remov
#182563408332d7 zhack: Fix importing large allocation profiles on
small pools
#18258f8457fbdc Fix deadlock on dmu_tx_assign() from vdev_rebuild()
#18263f8e5af53e Fix redundant declaration of dsl_pool_t
Obtained from: OpenZFS
OpenZFS commit: f8e5af53e9
re-wrap the function calls in a couple of ifs in get_zfs_root so they
are more readable. They really didn't need to be wrapped like this in
the first place.
Sponsored by: Netflix
ZFSSRC is abiguous on its surface and too clos to ZFSTOP, so rename it
to SAZFSSRC.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D53901
Use ZFSTOP instead of OZFS. They are the saame thing.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D53900
There is a pool feature, dynamic_gang_header, that is enabled by default
in new pools. When this feature is active, gang headers may be larger
than 512 bytes. The loader needs to be taught to cope with that.
Try using the vdev ashift to pick the gang block header size. If the
checksum fails, fall back to the old gang block header size.
This is based on a patch by Paul Dagnelie, with testing, bug-fixing and
some simplifications from me.
PR: 289690
Co-authored by: Paul Dagnelie <paul.dagnelie@klarasystems.com>
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D53578
The reimplementation is a bit cleaner than the original code,
although it is also slightly slower. This shouldn't matter too
much as we will have asm code for the major platforms.
The code is unrolled when built in the kernel or user space, but
not when in libsa, as to reduce the code size.
Differential Revision: https://reviews.freebsd.org/D45670
Reviewed by: jrtc27, imp
Approved by: markj (mentor)
MFC after: 1 month
Both files are used by kernel and userspace.
Move them to sys/crypto where they belong.
No functional changes intended.
In preparation of D45670.
Reviewed by: markj
Approved by: markj (mentor)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D52909
What we really want here is to know if pointers can refer to 64-bit
addresses, regardless of whether they also hold other information (such
as capabilities in CHERI). __SIZEOF_SIZE_T__ is probably the closest
indication to that piece of information, so let's use it. __ILP32__
wasn't wrong in practice though, as we don't support 32-bit CHERI
hardware (and likely never will).
Consistently with this change, test whether we can actually address the
64-bit SMBIOS's structure table by converting the end address to
'size_t' and checking whether its value is preserved.
Suggested by: jhb (for the __ILP32__ => __SIZEOF_SIZE_T__ part)
Reviewed by: jhb, imp
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49318
A directory with a zero-valued d_reclen causes boot to hang, This
patch checks for directory entries with value zero and bails out
rather than spin forever. The hope is that the user has other options.
Sadly this was reported on Jun 16 2015 and is just now attended to
as part of a sweep of old unclosed phabricator reports.
Reported-by: Daniel O'Connor darius-dons.net.au
Differential Revision: https://reviews.freebsd.org/D2844
MFC-after: 1 week
Sponsored-by: Netflix
Our only user of this at the moment is teken.{fg,bg}_color. These are
special because teken is a library common to both the kernel and the
loader, and we want to avoid having special vars to control the loader
vs. the kernel. Ideally, if a user wants a different set of console
colors, then they set the appropriate teken variable and it
Just Works(TM) everywhere. We can't just avoid setting the env vars,
because we specifically want to install a hook to adjust how loader is
drawn.
This allows us to avoid breaking a kernel config(5) that has some
default teken colors set with our defaults. That's a valid
configuration, even if it might seem weird that they don't want to set
colors in both loader and the kernel -- they may not anticipate spending
any time in loader, and thus prefer to just let it do its default
behavior.
NOKENV is expected to be unset if the value is overwritten, rather than
acting as a persistent marker that we do not want the value to persist
under any circumstance. We can always add another flag bit later for
persistence if we find a use for that, but most variables are fine to
carry over. This is mostly needed for environment variables that we
really just want to set a hook for.
Future work could expand this to break it out to the scripted
interfaces. We have discussed some options like a new built-in command,
or adding a flag to the existing `set` command, but haven't really come
up with a concrete plan to avoid confusion.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D50888
When doing file verification, tftp needs to be able to handle multiple
open files concurrently.
We also need tftp_stat() to provide useful values for st_dev and st_ino.
Allow an architecture to define NETPROTO_DEFAULT.
The default is NET_NFS for backwards compatability.
In net_parse_rootpath() fix parsing of
<scheme>://<ip>[:<port]/<path>
and ensure we return INADDR_NONE unless we successfully
parsed an addr, so we don't end up clobbering rootip obtained
from bootp().
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D51187
Attempting to mount or even open / with some tftp servers
causes a several minute delay in boot.
Since opening a directory via TFTP does not make sense, we
avoid it. We don't know if using TFTP until after net_open()
has been called.
Add an is_tftp() accessor to avoid everyone having to include
all the net* headers.
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D51447
To avoid a layering violation in open() allow fs_ops to
indicate that devopen() should be skipped.
This is only true for pkgfs.
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D51684
This function is used only once and also pulls three variables from the
nvlist that the only caller have already pulled out.
Reviewed by: mav, imp
Differential Revision: https://reviews.freebsd.org/D52031
Fix two problems with 6dd0803ffd. First problem is that situation when
newer label was read before stale one, was handled differently to reverse
order case. Second problem is that vdev_free() would free the fully
initialized leaf vdev that carried stale label. In a case when vdev
carries a stale label, but is still referenced by a different label with
new a configuration, we don't want to free it, but rather insert it into
the new configuration.
o Provide a helper function nvlist_find_vdev_guid() that checks presence
of certain GUID in a label.
o In top level vdev store the GUID of vdev used to instantiate top vdev.
o Cover all possible cases in the block in vdev_probe() where we encounter
a known configuration. Make the diagnostic print more informative and
looking same regardless of probe order. Make this whole block easier to
read reducing one level of indentation for a price of a single comparison
at runtime.
Reviewed by: mav, imp
Differential Revision: https://reviews.freebsd.org/D51913
Fixes: 6dd0803ffd
Before this change in vdev_insert() we would avoid inserting a duplicate
vdev to the list of children, however this duplicate being unlinked from
the parent is still stored on the global list with initialized v_guid.
Such leaked duplicate can later be returned by vdev_find(). After
6dd0803ffd such leaked vdev may be freed or pointing to a freed parent,
which leads to a loader crash. Note that the leak problem was there
before 6dd0803ffd.
First, in vdev_insert() free conflicting vdev and return the existing one.
Update callers accordingly. There is only one caller that actually may
encounter this condition.
Second, eliminate global list of vdevs and make vdev_find() to work
recursively on the tree that a caller must provide. Of course, a chance
of GUID collision between members of different pools is extremely low. The
main motivation here is just to increase code robustness and fully isolate
the data structures of different pools being tasted by the loader, and
make easier debugging of bugs like the one being fixed.
Reviewed by: mav, imp
Differential Revision: https://reviews.freebsd.org/D51912
Fixes: 6dd0803ffd
An empty list case is properly covered by the cycle.
Don't pass pointer to vdev being looked up, pass just id.
Reviewed by: mav, imp
Differential Revision: https://reviews.freebsd.org/D51911
Zero txg means this vdev has been correctly removed from the pool. For
reference see module/zfs/vdev_removal.c in OpenZFS. It calls
vdev_label_init(vd, 0, VDEV_LABEL_REMOVE), and second argument there is
the txg to write.
Reviewed by: tsoome, mav, imp
Differential Revision: https://reviews.freebsd.org/D51910
In vdev_init_from_label() we obtain the pool guid but don't use it.
Reviewed by: tsoome, mav, imp
Differential Revision: https://reviews.freebsd.org/D51909
Well, it does not really work and we are getting retransmits.
To replicate, set nfs.read_size large enough.
What needs to happen is, we read ethernet packet, if it has
IPv4 payload and that payload is fragment, we create reassembly
queue (sorted by growing fragment offset) and on last
fragment, we can build complete packet. Once done properly,
the network load can utilize larger read sizes.
While there, move ARP (and other) processing out of readipv4().
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D51690
Notable upstream pull request merges:
#16853894edd084 Add TXG timestamp database
#17004 -multiple Implement dynamic gang header sizes
#17405dea0fc969 ZVOL: Return early, if volmode is ZFS_VOLMODE_NONE
on FreeBSD side
#17455cf146460c Default to zfs_bclone_wait_dirty=1
#17481523d9d600 Validate mountpoint on path-based unmount using statx
#1748292da9e0e9 ZVOL: Implement zvol_alloc() function on FreeBSD side
#17489dee62e074 spa: ZIO_TASKQ_ISSUE: Use symbolic priority
#17496bf846dcb7 Release topology restrictions on special/dedup
#174974e92aee23 Relax special_small_blocks restrictions
#17503ea38787f2 Revert "Fix incorrect expected error in ztest"
#17505be1e991a1 Allow and prefer special vdevs as ZIL
#17506ee0cb4cb8 ztest: Fix false positive of ENOSPC handling
#1750792d3b4ee2 zio: rename `io_reexecute` as `io_post`;
use it for the direct IO checksum error flag
#175086af8db61b metaslab: don't pass whole zio to throttle reserve APIs
#175184c2a7f85d FreeBSD: Add support for _PC_HAS_HIDDENSYSTEM
#17521d7ab07dfb ZIL: Force writing of open LWB on suspend
#17524b6e8db509 zpool/zfs: Add '-a|--all' option to scrub, trim,
initialize
#17531c1e51c55f Correct weight recalculation of space-based metaslabs
#17533d323fbf49 FreeBSD: zfs_putpages: don't undirty pages until
after write completes
#17536b21e04e8d Fix zdb pool/ with -k
#17537 -multiple Userspace tunables
#175402957eabbe Add support for FreeBSD's Solaris style extended
attribute interface
#175474bd7a2eaa zdb: fix checksum calculation for decompressed blocks
#17551 -multiple cleanup: remove var init/update
#175610f8a1105e Skip dbuf_evict_one() from dbuf_evict_notify()
for reclaim thread
#1756310a78e264 Faster checksum benchmark on system boot
#1756400ce064d8 spa: update blkptr diagram to include vdev padding
on encrypted blocks
#17565 -multiple Physical rewrite
#17566fc885f308 Don't use wrong weight when passivating group
#17572f70c85086 BRT: Fix ZAP entry endianness
#175750b6fd024a ZVOL: Unify zvol minors operations and improve error
handling
#17581cb5e7e097 range_tree: Provide more debug details upon unexpected
add/remove
#17587 -multiple Fix two issues with dynamic gang headers
Obtained from: OpenZFS
OpenZFS commit: 8302b6e32b
loader ip implementation is using pointer to structure ip to receive
the packet and is using this pointer to cast on other data types
(namely structure arphdr). Problem does arise when those data structures
are declared with different alignment rules and when/if the compiler
does check those rules. To work around and silence warnings, use
void * generic pointer instead.
Error seen with gcc 14 (-Werror=address-of-packed-member).
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D51662
stand/ is mostly debugged with printfs, in an ad hoc and sometimes
fragile manner. For example BOOTP_DEBUG in bootp.c cannot be defined
unless NETIF_DEBUG is defined in dev_net.c or build fails for lack of the
symbol debug.
The DEBUG_PRINTF implementation in stand.h addresses that and allows
for more control over debug output. It is compatible with the
usage in libsecureboot.
Simply define _DEBUG_LEVEL to the desired level of debug
or in the case of libsecureboot _DEBUG_LEVEL_VAR to the variable that
will hold that value - default is _debug which is static so each
translation unit can be controlled independently.
The 1st arg to DEBUG_PRINTF is a level which must be greater than or
equal to _DEBUG_LEVEL_VAR if the printf is to be called.
See libsecureboot for more examples.
Reviewed by: imp
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D51269
Before this change, the first probed member of a pool would initialize
vdev tree for the pool. Now, imagine a situation when a machine has a
disk that has been removed from the pool, but the ZFS label was not
erased. That's a typical scenario - disk goes offline, it is replaced
with a spare, no data changes written to the gone disk. Then, disk
appears back at boot time and it is the first one to be probed by the
loader. It has the same pool GUID as all other members and naive loader
would not see a conflict. Then the disk will be used as source of truth
to read the bootenv.
To fix that, provide vdev_free() that allows to rollback the already
prebuilt vdev tree, so that a new one can be built from scratch. Upon
encountering a newer configuration for already known top level part of a
pool, call vdev_free() and let vdev_probe() to build a new one.
The change has been tested with loader_lua and userboot.so, but it should
have same effect on the legacy boot1 loader.
Reviewed by: tsoome, mav, imp
Differential Revision: https://reviews.freebsd.org/D51219
This is just the value from the current label and it is not used in the
function. Later zfs_spa_init() will call vdev_init_from_nvlist() and
it will obtain the value.
Switch to using sys/stdarg.h for va_list type and va_* builtins.
Make an attempt to insert the include in a sensible place. Where
style(9) was followed this is easy, where it was ignored, aim for the
first block of sys/*.h headers and don't get too fussy or try to fix
other style bugs.
Reviewed by: imp
Exp-run by: antoine (PR 286274)
Pull Request: https://github.com/freebsd/freebsd-src/pull/1595
When such an address is known, do not search elsewhere, contrary to what
we are doing on non-EFI boot (see SMBIOS_START and SMBIOS_LENGTH).
Reviewed by: markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49291
In particular, this allows the caller to know whether the detection was
successful.
Reviewed by: imp, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49290
This is in preparation for modifying the EFI loader to favor a v3 table
if present.
As an impact, caller smbios_match() has been changed so that it only
calls smbios_probe() with NULL (non-EFI discovery) once.
While here, expand the original XXXRP comment in smbios_match().
Reviewed by: imp, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49289
On amd64, boot blocks and the non-EFI loader are 32-bit compiled as
clients of BTX, so cannot access addresses beyond 4GB. However, the
64-bit entry point may refer to a structure table below 4GB, which we
want to use if the BIOS does not provide a 32-bit entry point. The
situation is similar for powerpc64.
Consequently, always compile-in support for the 64-bit entry point, but
ensure that it is not selected on 32-bit-compiled boot loaders if the
structure table it points to grows beyond 4GB (as it is then not
accessible).
PR: 284460
Reviewed by: markj
MFC after: 2 weeks
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49288
When both the 32-bit and 64-bit entry points are present, the SMBIOS
specification says that the 64-bit entry point always has at least all
the structures the 32-bit entry point refers. In other words, the
32-bit entry point is provided for compatibility, so we assume the
64-bit one has more chances to be filled with adequate values.
Doing this also increases consistency with the kernel's smbios(4)
driver.
Reviewed by: imp, markj
MFC after: 2 weeks
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49287
According to the specification, such an entry point may have different
data in bytes at offsets 0x0c to 0x17 (included). In such a case,
interpreting them as the Structure Table Maximum Size and Address fields
could have catastrophic consequences.
Reviewed by: imp, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49286
This structure exists since SMBIOS v2.2 and indicates that there are no
structures to be parsed beyond this point. For backwards compatibility,
the standard recommends that system software ensures that this structure
covers the rest of the SMBIOS structure table area as reported by the
Structure Table Address, and the Structure Table Maximum Size (64-bit
entry point) or the Structure Table Length (32-bit entry point), which
makes existing parsers continue to work correctly as they usually ignore
unknown structure types. However, this is not a requirement, so be
bullet proof and immediately stop parsing in this case.
Reviewed by: imp, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49285
The code parsing the BCD revision is only meaningful on v2, so move it
away into the appropriate 'if' branch to ease reading (and to avoid
a useless test).
Expand comments. In particular, make it clear that setting
'smbios.count' to '-1' removes the limit of the number of structures to
parse.
No functional change.
Reviewed by: imp, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49284
Via the kenv 'smbios.entry_point_type'.
Reviewed by: imp, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49283
Rename HAS_SMBV3 to SMBIOS_64BIT_EP, as it will soon be externally
defined in some cases, and 'isv3' to 'is_64bit_ep'. These are more
accurate names for what they actually control/indicate.
No functional change.
Reviewed by: imp, markj
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D49281
Simon J. Gerraty