Commit Graph

4973 Commits

Author SHA1 Message Date
behlendo
cfe5749941 Minor tweak to ensure kstat values are printed correctly on x86_64 at least
Additionally fix a minor typo in the .ul ULONG case.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@131 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-06 23:11:34 +00:00
behlendo
c58f753ddb Prep for 0.3.2 tag
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@129 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 23:28:29 +00:00
behlendo
41cf38df92 Add missing () to quiet warnings in NDEBUG case
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@128 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 22:52:13 +00:00
behlendo
3ce1bc96f9 Fix some bad grammer
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@127 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 21:25:57 +00:00
behlendo
475cdc788e Just use CONFIG_SLUB to detect SLUB use
Add ASSERTF to the NDEBUG build
Fix minor issue with various debug build flags



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@126 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 21:09:25 +00:00
behlendo
a02118a89d Whoops, fix a minor proc issue which slipped through with
the recent changes.  Ensure the top level spl is removed.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@125 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 06:09:16 +00:00
behlendo
c30df9c863 Fixes:
1) Ensure mutex_init() never fails in the case of ENOMEM by retrying
   forever.  I don't think I've ever seen this happen but it was clear
   after code inspection that if it did we would immediately crash.

2) Enable full debugging in check.sh for sanity tests.  Might as well
   get as much debug as we can in the case of a failure.

3) Reworked list of kmem caches tracked by SPL in to a hash with the
   key based on the address of the kmem_cache_t.  This should speed
   up the constructor/destructor/shrinker lookup needed now for newer
   kernel which removed the destructor support.

4) Updated kmem_cache_create to handle the case where CONFIG_SLUB
   is defined.  The slub would occasionally merge slab caches which
   resulted in non-unique keys for our hash lookup in 3).  To fix this
   we detect if the slub is enabled and then set the needed flag
   to prevent this merging from ever occuring.

5) New kernels removed the proc_dir_entry pointer from items
   registered by sysctl.  This means we can no long be sneaky and
   manually insert things in to the sysctl tree simply by walking
   the proc tree.  So I'm forced to create a seperate tree for
   all the things I can't easily support via sysctl interface.
   I don't like it but it will do for now.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@124 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 06:00:46 +00:00
behlendo
691d2bd733 Update utsname to use proper compatible interface to avoid API issues.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@123 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-03 21:20:18 +00:00
behlendo
684f787474 Fix missing return resulting in a double unlock of &files->file_lock
and a hang on subsequent sys_close.  I'm not quite sure why the Fedora
kernel caught this bug the Chaos kernel did not, but I'm glad!

Convert remaining BUG_ON's to ASSERTs



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@122 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-03 20:58:55 +00:00
behlendo
fe81cb1c43 Add the minimal set of kernel patches need to for the SPL. Hopefully
even these will not be needed over the next few weeks.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@121 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-02 19:45:04 +00:00
behlendo
f93f7c8dbe This should have been part of the previous autoconf commit.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@120 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-02 18:41:30 +00:00
behlendo
57d862349b Breaking the world for a little bit. If anyone is going to continue
working on this branch for the next few days I suggested you work
off of the 0.3.1 tag.  The following changes are fairly extensive
and are designed to make the SPL compatible with all kernels in
the range of 2.6.18-2.6.25.  There were 13 relevant API changes
between these releases and I have added the needed autoconf tests
to check for them.  However, this has not all been tested extensively.
I'll sort of the breakage on Fedora Core 9 and RHEL5 this week.

SPL_AC_TYPE_UINTPTR_T
SPL_AC_TYPE_KMEM_CACHE_T
SPL_AC_KMEM_CACHE_DESTROY_INT
SPL_AC_ATOMIC_PANIC_NOTIFIER
SPL_AC_3ARGS_INIT_WORK
SPL_AC_2ARGS_REGISTER_SYSCTL
SPL_AC_KMEM_CACHE_T
SPL_AC_KMEM_CACHE_CREATE_DTOR
SPL_AC_3ARG_KMEM_CACHE_CREATE_CTOR
SPL_AC_SET_SHRINKER
SPL_AC_PATH_IN_NAMEIDATA
SPL_AC_TASK_CURR
SPL_AC_CTL_UNNUMBERED



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@119 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-02 17:28:49 +00:00
behlendo
65a045dace Make a tag just for release to CEA, this has the URCL
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@115 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 05:01:15 +00:00
behlendo
715f625146 Go through and add a header with the proper UCRL number.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
behlendo
b2585b36d3 Prep for for 0.3.0 tag, this is the tag which was used for all
the performance results to date.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@112 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-21 21:11:47 +00:00
behlendo
cc7449ccd6 - Properly fix the debug support for all the ASSERT's, VERIFIES, etc can be
compiled out when doing performance runs.
- Bite the bullet and fully autoconfize the debug options in the configure
  time parameters.  By default all the debug support is disable in the core
  SPL build, but available to modules which enable it when building against
  the SPL.  To enable particular SPL debug support use the follow configure
  options:

  --enable-debug		Internal ASSERTs
  --enable-debug-kmem		Detailed memory accounting
  --enable-debug-mutex		Detailed mutex tracking
  --enable-debug_kstat          Kstat info exported to /proc
  --enable-debug-callb		Additional callb debug



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@111 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-19 02:49:12 +00:00
behlendo
6ab69573ff SPL additions to increase support for updated ZFS build
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@110 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-15 23:39:19 +00:00
behlendo
56f9245330 Disable adaptive mutexs by default (always sleep), and while
I'm at it add a module option for easy tuning.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@109 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-15 17:32:41 +00:00
behlendo
4efd41189a Rework condition variable implementation to be consistent with
other primitive implementations.  Additionally ensure that GFP_ATOMIC
is use for allocations when in interrupt context.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@108 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-15 17:10:30 +00:00
behlendo
a97df54e83 Enhanse the thread interface to do something quasi inteligent
with the function name passed to be used as a thread name.  Leaving
the trailing _thread is just redundant so just strip it this
make the thread names far more readable.

Use a strncpy in spl-mutex  just to be safe.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@107 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-12 18:54:08 +00:00
behlendo
8464443f8d Add a comment so I remember to fix this.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@106 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-12 16:53:41 +00:00
behlendo
c6dc93d6a8 By default disable extra KMEM and MUTEX debugging to aid performance.
They can easily be re-enabled when new stability issues are uncovered.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@105 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-09 22:53:20 +00:00
behlendo
5c2bb9b2c3 Stability hack. Under Solaris when KM_SLEEP is set kmem_cache_alloc()
may not fail.  To get this behavior I'd added a retry to the shim layer
even though it is abusive to the VM, at least it should prevent the crash.
Additionally I added a proc counter so I can easily check how often this
is happening.  It should be fairly rare, but likely will get worse and
worse the longer the machine has been up.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@104 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-09 21:21:33 +00:00
behlendo
04a479f706 Add an almost feature complete implemenation of kstat. I chose
not to support a few flags (we assert if they are used), and I
did not add the libkstat interface and instead exported everything
to proc for easy access.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@103 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-08 23:21:47 +00:00
behlendo
d4c540de38 Same deal as ZFS, we're quite stable now so tag it.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@101 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 20:12:44 +00:00
behlendo
427a782d7d Decrease of kmem warnign threshold back to 2 pages, no worse than a stack.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@100 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 19:33:01 +00:00
behlendo
13cdca65ec Add vmem memory accounting
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@99 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 18:54:32 +00:00
behlendo
404992e31a - Relocate 'stats_per' in to proper /proc/sys/spl/mutex/ directory
- Shift to spinlock for mutex list addition and removal



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@98 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 17:58:22 +00:00
behlendo
4f86a887d8 Remaining issues fixed after reenabled mutex debugging.
- Ensure the mutex_stats_sem and mutex_stats_list are initialized
- Only spin if you have to in mutex_init



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@97 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:19:27 +00:00
behlendo
e8b31e8482 - Updated rwlock's to reside in a .c file instead of a static inline
- Updated rwlock's so they can be safely initialized in ctors.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@96 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:00:49 +00:00
behlendo
d6a26c6a32 Lots of fixes here:
- Detailed kmem memory allocation tracking.  We can now get on
  spl module unload a list of all memory allocations which were
  not free'd and where the original alloc was.  E.g.

SPL: 15554:632:(spl-kmem.c:442:kmem_fini()) kmem leaked 90/319332 bytes
SPL: 15554:648:(spl-kmem.c:451:kmem_fini()) address          size  data             func:line
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff8100734b68b8 32    0100000001005a5a __spl_mutex_init:70
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff8100734b6148 13    &tl->tl_lock     __spl_mutex_init:74
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff81007ac43730 32    0100000001005a5a __spl_mutex_init:70
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff81007ac437d8 13    &tl->tl_lock     __spl_mutex_init:74

- Shift to using rwsems in kmem implmentation, to simply locking and
  improve concurency.

- Shift to using rwsems in mutex implementation, additionally ensure we
  never sleep in the init function if non-zero preempt_count or 
  interrupts are disabled as can happen in a slab cache ctor/dtor.

- Other minor formating fixes and such.

TODO:

- Finish the vmem memory allocation tracking

- Vet all other SPL primatives for potential sleeping during *_init.  I
suspect the rwlock implemenation does this and should be fixes just
like the mutex implemenation.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@95 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 20:38:28 +00:00
behlendo
9ab1ac14ad Commit adaptive mutexes. This seems to have introduced some new
crashes but it's not clear to me yet if these are a problem with
the mutex implementation or ZFSs usage of it.

Minor taskq fixes to add new tasks to the end of the pending list.

Minor enhansements to the debug infrastructure.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@94 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-05 20:18:49 +00:00
behlendo
bcd68186d8 New an improved taskq implementation for the SPL. It allows a
configurable number of threads like the Solaris version and almost
all of the options are supported.  Unfortunately, it appears to have
made absolutely no difference to our performance numbers.  I need
to keep looking for where we are bottle necking.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@93 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-25 22:10:47 +00:00
behlendo
839d8b438e Update kmem.h to properly use new debug subsystem.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@92 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 20:21:07 +00:00
behlendo
3561541c24 Prep for 0.2.1 tag
Minor fixes to headers to use debug macros
Added /proc/sys/spl/version



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@90 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:41:23 +00:00
wartens2
1bac409fa3 Forgot to update the ChangeLog.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@89 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:24:02 +00:00
wartens2
8100fe56f1 Make sure that when calling __vmem_alloc that we
do not have __GFP_ZERO set.  Once the memory is allocated
then zero out the memory if __GFP_ZERO is passed to
__vmem_alloc.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@88 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:07:56 +00:00
behlendo
6e605b6e58 Minor improvement to taskq handling. This is a small step towards
dynamic taskqs which still need to be fully implemented.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@87 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-23 21:19:47 +00:00
behlendo
18c9eadf97 Be careful to never use any of the debug infrastructure either
before the debug subsystem is fully set up, or after the debug
subsystem has been torn down.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@86 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-22 22:22:02 +00:00
behlendo
7e4e211333 Give it a real version for a tag
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@84 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-22 20:10:55 +00:00
behlendo
b831734a43 Stack usage is my enemy. Trade cpu cycles in the debug code to
ensure I never add anything to the stack I don't absolutely need.
All this debug code could be removed from a production build
anyway so I'm not so worried about the performance impact.  We
may also consider revisting the mutex and condvar implementation
to ensure no additional stack is used there.

Initial indications are I have reduced the worst case stack
usage to 9080 bytes.  Still to large for the default 8k stacks
so I have been forced to run with 16k stacks until I can
reduce the worst offenders.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@83 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-22 16:55:26 +00:00
behlendo
7fea96c04f More fixes to ensure we get good debug logs even if we're in the
process of destroying the stacks.  Threshhold set fairly aggressively
top 80% of stack usage.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@82 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 22:44:11 +00:00
behlendo
e5bbd245e3 Added 4 missing subsystem flags
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@81 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 18:43:02 +00:00
behlendo
a8ac0b8966 Whoops, missed an instance where we could recursively stack check... bad.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@80 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 18:16:04 +00:00
behlendo
892d51061e Handful of minor stack checking fixes
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@79 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 18:08:33 +00:00
behlendo
937879f11d Update SPL to use new debug infrastructure. This means:
- Replacing all BUG_ON()'s with proper ASSERT()'s
- Using ENTRY,EXIT,GOTO, and RETURN macro to instument call paths



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@78 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 17:29:47 +00:00
behlendo
2fae1b3d0a Frist minor batch of fixes. Catch a dropped ;, and use SBUG instead of BUG.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@77 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-19 00:02:11 +00:00
behlendo
ce86265693 Whoops need this!
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@76 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-18 23:42:45 +00:00
behlendo
57d1b18858 First commit of lustre style internal debug support. These
changes bring over everything lustre had for debugging with
two exceptions.  I dropped by the debug daemon and upcalls
just because it made things a little easier.  They can be
readded easily enough if we feel they are needed.

Everything compiles and seems to work on first inspection
but I suspect there are a handful of issues still lingering
which I'll be sorting out right away.  I just wanted to get
all these changes commited and safe.  I'm getting a little
paranoid about losing them.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@75 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-18 23:39:58 +00:00
wartens2
55152ebbb4 * modules/spl/spl-kmem.c : Make sure to disable interrupts
when necessary to avoid deadlocks.  We were seeing the deadlock
        when calling kmem_cache_generic_constructor() and then an interrupt
        forced us to end up calling kmem_cache_generic_destructor()
        which caused our deadlock.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@74 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-16 16:37:51 +00:00