[Prev][Next][Index][Thread]

OSKit 20000901 snapshot; Labor Day Snapshot

Since it was "Labor Day" weekend, and we were busy, we are making another
OSKit snapshot.  Besides the usual round of and minor improvements and bug
fixes-- including fixed 16-bit support-- we have made a number of important
performance improvements, and have added several new features such as PXE boot
support, POSIX realtime message queues and semaphores, a proportional share
"stride" scheduler, and a new software interrupt mechanism.  We first discuss
the performance improvements, and then features, applications, and fixes.

PERFORMANCE:
* Polled packet reception: Addition of receive side polling capability to
  the oskit_netdev_t and oskit_etherdev_t interfaces. For network device
  drivers that support receive side polling, you can now place the card in
  polling mode (via the open call) instead of interrupt mode, and poll for
  packets via the new "rxpoll" method in the aforementioned interfaces.  We
  have modified the Tulip and EEPRO100 linux ethernet drivers to support
  receive polling. We have also modified the OSKit version of the Click
  Modular Router (version 1.0.6) to use the new polling interface. With
  Click, we have seen an improvment of about 4 times in the number of
  packets that can be forwarded before receive livelock is encountered With
  all of the following optimizations in place, a Click router running on a
  200 MHZ PPro can forward up to about 110,000 packets per second without
  livelock.

* New oskit_skbufio_t interface: Realizing that the oskit_bufio_t interface
  is a little too generic, we added the oskit_skbufio_t interface, whose
  single implementor is the linux network device driver. This new interface
  allows a program to operate directly on the internals of the data
  structure that underlies the oskit_skbufio_t, which of course is a linux
  struct sk_buff. The result is less copying, fewer COM method calls, and
  an overall improvement in packet forwarding and packet sending rates of
  about 10 times.  We have seen similar results in the OSKit version of the
  Click Modular Router (version 1.0.6), which was modified to use the
  skbufio interface instead of the generic bufio interface.

* Improved linux network device driver memory allocation: Two functions,
  oskit_skbmem_{alloc,free}, were added (in linux/dev/skbuff_mem.c) to
  allow a kernel to have direct control over memory used for skbuffs.  By
  overriding these functions, a kernel can provide all memory used by
  skbufios and other skbuff-based bufios that originate from the linuxdev
  (device driver) library.  The default implementation of
  oskit_skbmem_{alloc,free} manages a cache of skbuff memory to reduce the
  frequency of memory allocation calls.  Skbuff memory is also allocated in
  fixed-size chunks (corresponding to 0, 200 and 1600 byte skbuff lengths)
  to reduce memory fragmentation.

* Improved network bufio allocation: An added method in oskit_netio_t
  allows the implementor of a netio interface to provide a specialized
  oskit_bufio_t allocator. This specialized allocator can be used to
  provide an oskit_bufio_t implementation that is specific to the
  corresponding oskit_netio_t implementation. For example, the linux netio
  implementation now allows you to allocate a bufio that is already in the
  correct format (that is, looks like an SKB), so that packets can be more
  directly sent out on the wire.

FEATURES:
* Improved software interrupt support: We have unified the software
  interrupt mechanism so that any component that needs a software interrupt
  to run at the end of a hardware interrupt has a consistent way to do
  that. Previously, individual components rolled their own software
  interrupt mechanism, with the result that they often ran before the
  hardware interrupt was really over, and potentially saw an inconsistent
  hardware state. We have fixed this by providing a single implementation
  that any component can use. Each of the Linux, FreeBSD, and pthreads
  modules have been modified. See oskit/dev/softirq.h for a description of
  the new interface.

* POSIX Realtime message queues and semaphores: New implementations of the
  POSIX Realtime message queue (mq) and semaphore interfaces. These
  interfaces are available to multithreaded oskit kernels, and are briefly
  described in the pthreads chapter of the oskit documentation. Complete
  documentation can found in the POSIX 1003.1b specification. Two small
  demonstration programs: examples/x86/threads/{semtest.c,mqtest.c} are
  provided.
	
* Stride Scheduler: A new Stride (proportional share) scheduler has been
  added to the pthreads scheduler suite. This scheduler implements the
  classic Stride algorithm as described in the paper referenced at the top
  of threads/sched_stride/sched_stride.c. There is a simple demonstration
  program: examples/x86/threads/stride_test.c.  Note that you must edit
  threads/MakeFlags to ensure that the stride scheduler module is built
  into the pthreads library.

* Data compression library (ZLIB): The zlib subdirectory contains an
  environment for building an OSKit version of zlib 1.1.3
  (ftp://ftp.freesoftware.com/pub/infozip/zlib/). See the README file in
  that directory for further instruction.  This zlib library is used in the
  netdisk kernel below.

APPLICATIONS:
* PXE boot support: A new boot program that is PXE Version 2.0 compliant.
  Built on the renovated 16bit code, the PXE boot program fits in the
  required 32K, and functions as a first level boot program downloaded via
  TFTP by the PXE boot ROM. Once loaded, it can load either a multiboot
  compliant image via TFTP, or boot the active disk partition. There is a
  lengthy description in boot/pxe/README. We also provide a little bootinfo
  server that you can use to tell the pxeboot kernel what to do next.

* A prototype network disk loader: The boot/disk directory contains
  makefiles to build a variant of netboot capable of copying a compressed
  disk image across NFS to a raw disk (or disk partition).  This primitive
  ``netdisk'' kernel can be used to load a bare machine with a pre-prepared
  disk image. Think of it as a remote "dd" program.  It is included for
  reference only; we expect to have a better version soon.

IMPORTANT FIXES:
* Fixes to 16 bit code: Numerous fixes and improvements to the 16 bit code
  for new versions of GCC and binutils. It is once again possible to build
  a working linux boot image that is less than 64K is size.


As before, go to http://www.cs.utah.edu/flux/oskit/
and everything hangs off there; currently mostly in ftp.

Thanks to all those who have sent in bug reports and/or fixes for past
snapshots.  We appreciate your continued involvement in making the OSKit
more useful for everyone.

Good Luck from the Flux Group and the
University of Utah School of Computing!