2010-05-21 Brian Behlendorf * : Tag spl-0.4.9 * : Build system improvements: - Added support for the 'make -s' silent build option. - Allow spl_config.h to be included by dependent packages. - Minor spec file updates. * AUTHORS, COPYING, DISCLAIMER, and INSTALL: Updated. * *.c, *.h, *.sh, AUTHORS, COPYING, DISCLAIMER, and INSTALL: Added standard header to source files which includes the copyright, license, and author information. Additionally, updated the listed top level files to the latest versions. * module/spl/spl-debug.c: Debugging defaults adjusted. The default log dump path is now /tmp/spl-log and panic on SBUG has been disabled. * include/sys/processor.h, include/sys/systm.h, include/sys/types.h: Added 3 missing typedefs processorid_t, pc_t, and index_t. * include/sys/console.h: Added support for console_* functions(). * module/spl/spl-time.c: Updated gethrtime() to use the function do_posix_clock_monotonic_gettime() as described in the comment. * module/spl/spl-condvar.c: Added cv_wait_interruptible() function as an extension to the existing condition variable ABI to enable sleeping threads a way to cleanly handle signals in the kernel. * module/spl/spl-debug.c: Dump log from current process as required. * module/spl/spl-taskq.c: Assume TQ_SLEEP in taskq_dispatch() if the caller does not specify, this is the Solaris default. * module/spl/spl-vnode.c: Extend the vn_rdwr() implementation to correctly handle the FAPPEND option. * module/spl/spl-generic.c, module/spl/spl-vnode.c: Update vn_set_pwd() to allow a kernal address as filename. This turns out to be critical because spl_setup()->The vn_set_pwd("/") which is called at module load time was failing with -EFAULT. * include/sys/rwlock.h: Disable rw_tryupgrade() for kernels built without CONFIG_RWSEM_GENERIC_SPINLOCK defined. The existing implementation is not safe in this case and needs to be replaced with a correct native per-arch implementation. * module/spl/spl-kmem.c: Reduce maximum kmem based slab size. Allowing MAX_ORDER-1 sized memory allocations while the system is low on memory has been observed to result in deadlocks. Callers should not be using the slab in this way but we should still handle the case more cleanly. * module/spl/spl-kmem.c: Remove kmem_set_warning() interface and replace it with KM_NODEBUG flag which can be specific per allocation. * module/splat/splat-kmem.c: Tweaked what the idea of a large object is for the kmem:slab_large regression tests because failures were observed for very large kmem objects in newer kernels. * include/sys/rwlock.h, module/splat/splat-rwlock.c: The rwlock implementation was simplified because failures were being observed when using the non-generic implementation. This change leverages the rwsem_is_locked() function instead of directly consulting the rwsem internals which differ per-arch and per-kernel-version. 2010-03-11 Brian Behlendorf * : Tag spl-0.4.8 * : Build system improvements: - Remove Module.markers and Module.symver{s} in clean target. - Improved kernel source detection when none specified. - Fix RPM definitions for the unknown distro/installation. * config/spl-build.m4: Linux 2.6.32 compat, SPL_AC_5ARGS_PROC_HANDLER macro added to handle updated proc_handler () API. * module/spl/spl-err.c: Fix panic() string, which was being used as a format string, instead of an already-formatted string. * /module/spl/spl-taskq.c: Optimize lowest outstanding taskqid calculation in taskq_lowest_id(). The motivation for this change was that I was observing as much as 10% of the total CPU time go to waiting on the tq->tq_lock when the pending list was long. * module/spl/spl-kmem.c: Strip __GFP_ZERO from kmalloc it is not available for older kernels. * module/spl/spl-proc.c: Add skc_flags and full header to /proc/spl/kmem/slab * module/spl/spl-generic.c: Correctly handle division on 32-bit RHEL5 systems by returning dividend. * include/sys/rwlock.h: When using x86 specific rwsem correctly intepret rwsem->count. Additionally check for changed guard macro in 2.6.28+ for rwsem implementation * module/splat/splat-vnode.c, module/splat/splat-kmem.c, module/splat/splat-taskq.c: Several new tests were added to the test suite and various bug fixes. * include/asm/atomic_compat.h: Atomic64 compatibility for 32-bit systems without kernel support. * .gitignore: Added .gitignore files. 2009-11-20 Brian Behlendorf * : Tag spl-0.4.7 * config/spl-build.m4: Linux 2.6.31 compat, SPL_AC_2ARGS_SET_FS_PWD macro updated to explicitly include linux/fs_struct.h which was dropped from linux/sched.h. * include/linux/mm_compat.h: Linux 2.6.31 compat, use the macros min_wmark_pages, low_wmark_pages, and high_wmark_pages which were introduced. For older kernels a compability macro is used. * include/sys/kmem.h: Linux 2.6.31 compat, the __GFP_NOFAIL flag should no longer be used and it may disappear from the kernel at any time. To handle this I have simply added *_nofail wrappers in the kmem implementation which perform the retry for non-atomic allocations. * module/spl/spl-kmem.c: Linux 2.6.31 compat, kmem cache alignment fixes and cleanup. Always perform allocations for the kmem cache using __get_free_pages() or __vmalloc() to ensure the returned memory is page aligned. Also introduce helper functions which use P2ROUNDUP_TYPE instead of P2ROUNDUP force all types to be explicit. * include/sys/mutex.h: Add mutex_enter_nested() as wrapper for mutex_lock_nested() to permit integration with the lock checker. * include/sys/mutex.h: Always use the generic mutex_destroy(). * Makefile.am: Ensure *.order and *.markers build products are removed by distclean rule. 2009-11-02 Brian Behlendorf * : Tag spl-0.4.6 * config/spl-build.m4: Cleanup --enable-debug-* configure options. --enable-debug= generic debug support. --enable-debug-kmem= basic kmem and slab accounting. --enable-debug-kmem-tracking= detailed kmem tracking. --enable-debug-kstat removed option support always enabled. --enable-debug-callb removed option which never did anything. * module/spl/spl-atomic.c: Reimplemented atomic functions by layering them on top of the Linux atomic functions. This improves performance by removing the need for a global spinlock. * module/spl/spl-debug.c: Rebase cmn_err on vcmn_err. Handle the case where the message contains a leading ! which means only send it to syslog. And suppress warnings for messages which do not end in a '\n'. * module/splat/splat-ctl.c: Update to use kobject_set_name() for increased portability. * module/spl/spl-generic.c: Set cwd to '/' for the task during module load for SPL dependent modules. Under Solaris this is the expected behavior, on a Linux system your cwd remains set to whatever the tasks cwd was when loading a module. * module/spl/spl-mutex.c: Reimplemented mutexs for improved performance and cleaner integration with the Linux kernels built in lock analyzer. As of 2.6.29 based kernels adaptive mutexs are supported, for these newer kernels task_curr() does not need to be exported. Finally, DEBUG_MUTEX support was dropped in favor of more detailed kernel profiling which is now available. * module/spl/spl-rwlock.c: Reimplemented rwlocks for improved performance and cleaner integration with the Linux kernels built in lock analyzer. Additionally, the rwlock regression tests were improved to help validate correctness. * spl-modules.spec.in: Various spec file tweaks for the supported distros: RHEL5, RHEL6, SLES10, SLES11, Chaos4, Fedora 11. 2009-08-04 Brian Behlendorf * : Tag spl-0.4.5 * FC11 and SLES11 support: This includes all compatibility changes to support 2.6.29 based kernels and the required build system improvements. * module/spl/spl-module.c: Register a basic compat ioctl handler for 32-bit user vs 64-bit kernel compatibility. This is the default build environment for all 64-bit SLES systems. * config/spl-build.m4: Check arch/default path when detecting kernel objects under SLES. We still preferentially use arch/arch if available but if that fails it is acceptable to use default. * module/spl/spl-vnode.: Use HAVE_PATH_IN_NAMEIDATA compat macros for maximum kernel compatibility. * config/spl-build.m4: Remove LINUXINCLUDE from autoconf wrapper. This breaks the 2.6.28+ kernels build system, all kernel build systems at least post 2.6.16 will set this properly so we should not. * include/sys/vmsystm.h, module/spl/spl-kmem.c: Perferentially use the global_page_state() API when your kernel supports the required enums. * include/sys/debug.h: Add ASSERTV macro to simplify removing variables (the V in ASSERTV) which are only used in ASSERT(). * include/sys/debug.h: Disable stack overflow checking by default when build with --enable-debug, modern kernels now provide this. * patches/fc11-spl-export-symbols.patch: Recommended missing symbols patch for FC11. This is not required for correct functionality but it is recommended for individuals who are already patching the kernel. * Additional Solaris API emulation: * include/sys/taskq.h: Add basic taskq support for the flag TASKQ_THREADS_CPU_PCT which is used to scale the number of threads based on the number of online CPUs. * include/sys/file.hi, module/spl/spl-generic.c: Add support for fake ioctls which originate from the kernel instead of user space. * include/sys/acl.h: Add ACE_ALL_PERMS ACL definition. * include/sys/cred.h, module/splat/splat-cred.c: Add basic credential support and splat regression tests. * Bug fixes: * include/sys/isa_defs.h: Add more endianess paranoia. * module/spl/spl-time.c: Integer overflow after ~164 days. * include/sys/debug.h: NULL dereference by tcd_for_each(). * module/splat/splat-kmem.c: Allow kmem or vmem based slabs for slab_lock and slab_overcommit tests. This may still be an issue on 32-bit systems due to the small virtual address space. * module/spl/spl-module.c: Positive Solaris ioctl return codes are need to be negated for use by libc to ensure errno is set correctly. 2009-07-02 Brian Behlendorf * : Tag spl-0.4.4 * : Generic distro friendly build system / packaging improvements for rpm based distros including CHAOS, RHEL, Fedora, and SLES. spl-.src.rpm - Fully rebuildable source rpm for utils. spl-modules-.src.rpm - Fully rebuildable source rpm for kernel modules. spl-..rpm - Binary rpm for utils. The utils in this package are compatible with all spl-module rpms of the same version. spl-modules--.arch.rpm - Binary rpm containing the kernel modules for a specific kernel build. The package name contains the kernel version and you should have one of these packages installed to match every kernel on your system. spl-modules-devel--.arch.rpm - Binary rpm containing development header and module symbols needed for building additional kernel modules which are dependent on the spl module. * : Added SLES10 support. This includes all compatibility changes to support 2.6.16 based kernels and the required build system improvements. * : Build System changes for SLES10: - Exclude -obj when detecting installed kernel source. - Detect -obj directory for out of tree kernel builds. - Allow kernel build system to set CC to ensure -m64 is set properly. This is an issue on 64-bit SLES systems which by default always build 32-bit binaries (unlike RHEL/Fedora which default to 64-bit) * : Configure Checks added for SLES10: - div64_64() renamed to div64_u64() as of 2.6.26. - global_page_state() fuction was introduced in 2.6.18 kernels. The earlier 2.6.16 based SLES10 must not try and use it, thankfully get_zone_counts() is still available. - monotonic_clock() is unavailable __gethrtime() must perform the HZ division as an 'unsigned long long' because the SPL only implements __udivdi3(), and not __divdi3() for 'long long' division on 32-bit arches. - mutex_lock_nested() was introduced as part of the mutex validator in 2.6.18, when it is unavailable it is safe to fallback to using a plain mutex_lock(). - SLES specific API change to vfs_unlink() and vfs_rename() which added a 'struct vfsmount *' argument. This was for something called the linux-security-module, but it appears that it was never adopted upstream. - spl_device_create() correctly mapped to class_device_create() for 2.6.13 to 2.6.17 based kernels, this is the preferred API. - Prior to 2.6.17 there were no *_pgdat helper functions in mm/mmzone.c. Instead for_each_zone() operated directly on pgdat_list which may or may not have been exported depending on how your kernel was compiled. A configure check was added to determine if you have the helpers or not, and if the needed symbols are exported. If they are not exported then they are dynamically aquired at runtime by kallsyms_lookup_name(). * : Packaging changes for SLES10 - Properly honor --prefix in build system and rpm spec file. - Add '--define require_kdir' to spec file to support building rpms against kernel sources installed in non-default locations. - Add '--define require_kobj' to spec file to support building rpms against kernel object installed in non-default locations. - Stop suppressing errors in autogen.sh script. - Improved logic to detect missing kernel objects when they are not located with the source. This is the common case for SLES as well as in-tree chaos kernel builds and is done to simply support for multiple arches. - Moved spl-devel build products to /usr/src/spl-, a spl symlink is created to reference the last installed version. - Allow checking for exported symbols in both Module.symvers and Module.symvers. My stock SLES kernel ships an objects directory with Module.symvers, yet produces a Module.symvers in the local build directory. * : Added powerpc64 support. - Enable builds for powerpc64 ISA type. - Add DIV_ROUND_UP and roundup macros if unavailable. - Cast 64-bit values for %lld format string to (long long) to quiet compile warning. * module/splat/splat-ctl.c: Proper ioctl() 32/64-bit binary compatibility. We need to ensure the ioctl data itself is always packed the same for 32/64-bit binaries. Additionally, the correct thing to do is encode this size in bytes as part of the command using _IOC_SIZE(). * modules/spl/spl-kmem.c: Fixed a long standing bug in the debug tracing. The tcd_for_each() macro expected a NULL to terminate the trace_data[i] array but this was only ever true due to luck. All trace_data[] iterators are now properly capped by TCD_TYPE_MAX. * modules/spl/spl-kmem.c: To simplify debugging all symbols aquired dynamically using spl_kallsyms_lookup_name() are initially poisoned with SYMBOL_POISON. 2009-03-20 Brian Behlendorf * : Tag spl-0.4.3 * configure.ac, *Makefile.am: Build system update. This includes resolving various build issues and adding support for the remaining common build targets. Available targets now include: - make all # Build everything - make install # Install everything - make clean # Clean up build products - make distclean # Clean up everything - make dist # Create package tarball - make srpm # Create package source RPM - make rpm # Create package binary RPMs - make tags # Create ctags and etags for everything Extra care was taken to ensure that the source RPMs are fully rebuildable against Fedora/RHEL/Chaos kernels. To build binary RPMs from the source RPM for your system simply run: rpmbuild --rebuild spl-x.y.z-1.src.rpm This will produce two binary RPMs with correct 'requires' dependencies for your kernel. One will contain all zpl modules and support utilities, the other is a devel package for compiling additional kernel modules which are dependent on the spl. spl-x.y.z-1_.x86_64.rpm spl-devel-x.y.2-1_.x86_64.rpm * : FC10 (linux-2.6.27.19) and i686 compatibility update. The list of support platforms has been extended to include FC10 systems and x86 architectures. It should be noted that kernels older the 2.6.27.19 should work but have not have not been tested. * configure.ac: Fix build issue preventing spl_config.h from being cleanly included by dependent packages. * module/spl/spl-taskq.c: Fix taskq_wait() not waiting bug. * module/spl/spl-xdr.c: Add XDR implementation provided by Ricardo Correia from Sun. * module/spl/spl-kmem.c: Linux VM integration cleanup. * module/spl/spl-kmem.c: Slab cache improvements and fixes. * modules/splat/*: Include additional SPLAT regression tests. * : Various bug fixes are more clearly detailed in the git commit logs. For a detailed summary of changes post version 0.4.2 check the git commit logs. git log -35 2009-02-05 Brian Behlendorf * : Tag spl-0.4.2 * module/spl/spl-kmem.c include/sys/kmem.h: Slab cache improvements: - Implement kmem cache alignment. - Implement slab ageing. - Optimized slab packing algorithm. - Fixed deadlock due to calling call kv_free() under the skc_lock. - Added additional SPLAT test cases - Performance optimizations * module/spl/spl-kmem.c include/sys/kmem.h: Linux VM integration. The Solaris global VM symbols minfree, desfree, lotsfree, needfree, swapfs_minfree, swapfs_reserve, availrmem, freemem, and physmem are now available and loosely integrating with the Linux VM. Some tuning will undoubtably be needed and these tunables are available in /proc/sys/kernel/spl/vm/* for this very reason. * config/spl-build.m4: New configure checks needed when building against 2.6.27+ kernels. More work is needed here. * : Minor cleanup see the 'git log' for full details. 2009-01-21 Brian Behlendorf * : Tag spl-0.4.1 * : Implement ksid_*, ddi_strto*, and system taskq functionality. In addition, several other small Solaris compatibility changes were made, see the 'git log' for full details. 2008-11-26 Brian Behlendorf * : Tag spl-0.4.0 (Development now done with Git) * : Imported SPL SVN repo in to Git Repo for core development. 2008-11-26 Brian Behlendorf * : Tag spl-0.3.5 * : Include META file support. 2008-11-05 Jim Garlick * : Add autogen.sh products. * configure.ac : Use AC_CONFIG_AUX_DIR to put autograph products in ./auotconf. * autogen.sh : Use --copy to avoid symlinks, remove error redirection, run aclocal before libtoolize. 2008-11-13 Brian Behlendorf * include/sys/sunddi.h, modules/spl/spl-module.c : Removed default udev support from sunddi implementation because it uses GPL-only symbols. This support is optionally available for SPL consumers if they define HAVE_GPL_ONLY_SYMBOLS and license their module as GPL using the MODULE_LICENSE("GPL") macro. 2008-11-05 Brian Behlendorf * : Tag spl-0.3.4 * : Coverity clean. * : Patches from Ricardo M. Correia applied with minor revisions: spl-00-rm-gpl-symbol-notifier_chain.patch spl-01-rm-gpl-symbol-set_cpus_allowed.patch spl-02-rm-gpl-symbols-device.patch spl-03-rm-gpl-symbol-ktime_get_ts.patch spl-04-fix-taskq-spinlock-lockup.patch spl-05-div64.patch spl-06-atomic64.patch spl-07-kmem-cleanup.patch spl-08-km-sleep-nofail.patch spl-09-fix-kmem-track-oops.patch spl-10-fix-assert-verify-ndebug.patch 2008-06-30 Brian Behlendorf * : Tag spl-0.3.3 * : modules/sys/kmem-slab.c : Refined SPL slab to include per-cpu caches, removed internal hash, other general performance improvements. Much work remain but it's pretty good for an initial implementation. 2008-06-13 Brian Behlendorf * : modules/sys/kmem-slab.c : Re-implemented the slab to no longer be based on the linux slab but to be it's own complete implementation. The new slab behaves much more like the Solaris slab than the Linux slab. 2008-06-04 Brian Behlendorf * : Tag spl-0.3.2 * : Extensive improvements to the build system to detect kernel API changes so we can flexibly build with a wider range of kernel versions. The code has now been testing with the 2.6.18-32chaos and 2.6.25.3-18.fc9 kernels, however we should also be compatible with other kernels in the range of 2.6.18-2.6.25. 2008-05-25 Brian Behlendorf * configure.ac, autoconf/* : Initial pass at resolving API changes introduced by kernels newer than 2.6.18. 2008-05-21 Brian Behlendorf * : Tag spl-0.3.1 * : License headers including URCL added for release. 2008-05-21 Brian Behlendorf * : Tag spl-0.3.0 * configure.ac: Improved autotools support. 2008-04-26 Brian Behlendorf * include/sys/mutex.h : Implemented a close approximation of adaptive mutexes. These changes however required me to export a new symbol from the kernel proper 'task_curr()' which means we are now dependant on a patched kernel. 2008-04-24 Brian Behlendorf * : Tag spl-0.2.1 * modules/spl/spl-proc.c : Add /proc/sys/spl/version. 2008-04-24 Herb Wartens * include/sys/kmem.h : Make sure that when calling __vmem_alloc that we do not have __GFP_ZERO set. Once the memory is allocated then zero out the memory if __GFP_ZERO is passed to __vmem_alloc. 2008-04-16 Herb Wartens * modules/spl/spl-kmem.c : Make sure to disable interrupts when necessary to avoid deadlocks. We were seeing the deadlock when calling kmem_cache_generic_constructor() and then an interrupt forced us to end up calling kmem_cache_generic_destructor() which caused our deadlock. 2008-02-26 Brian Behlendorf : Initial commit of the solaris porting layer (spl). Included in addition to the source is an initial autoconf / configure style build system.