2 Background:

2.1 USB Structure

The universal serial bus was created as a replacement for the venerable RS-232 serial and parallel communication standards that have featured on almost every personal computer ever made. USB was intended to remove the need for costly proprietary interfaces to ISA or PCI busses or the use of costly SCSI interfaces for peripherals such as scanners and removable hard drives. The primary deficiencies in the serial and parallel interfaces were seen as being:

· Limited Bandwidth – UART technology and types of cable specified by RS-232 limits the potential bandwidth of a serial connection to little over 100 kbits/s

· Lack of Chaining – No matter how many serial ports a machine had it might never have enough and so the ability to connect more than one device to a single port was seen as necessary.

· Lack of power distribution – With the exception of mice in most cases a serial or parallel connection cannot power the peripheral. A greater degree of power distribution was desired.

USB as defined by the 1.1 Specification[2] made the following characteristics widely available:

· Up to 127 devices connected together on one bus utilizing up to five layers of hubs.

· Total bandwidth capability of 12MBps.

· Devices may each draw a maximum of 2.5W of power[3] from the bus.

2.2 Topology and Configuration

USB defines various entities that have specific meaning in a USB context: Bus, Device, Configuration, Interface, and Endpoint. With the exception of a Bus each of these entities has a descriptor that holds various characteristics. The descriptors are available to the host system as soon as a device attaches to the bus even if there is no device driver for the device. They are part of the topological and configuration information.

A USB Device has the following characteristics:

· A Device has a Class, Subclass, and Protocol. These correspond to USB defined types of device[4]. For example Classes are defined for mass-storage, audio, modems, scanners, etc. These classes have different sub-classes and can conform to various USB defined protocols. It is possible for a device to be vendor-specific in which case it’s subclass and protocol values are not relevant. It is also possible for Devices to maintain different interfaces of different classes in this case the Device is not of a specific class but some of its interfaces have their own class.

· A Device also carries values for it’s manufacturer serial number and version numbers as part of the device descriptor.

· A Device has one or more possible configurations, one of which is active at any one time.

Each Configuration has the following characteristics:

· A Configuration is either bus-powered or self-powered, and either capable of remote-wake up or not.

· A Configuration has a maximum power usage specified in its descriptor

· A Configuration has one or more Interfaces.

An Interface has the following characteristics:

· An Interface has Class, Subclass, and Protocol. These are similar to the characteristics of Devices but relate only to a particular interface.

· An Interface has one or more endpoints.

An Endpoint has the following characteristics:

· An Endpoint is either an Input or an Output

· An Endpoint is of a particular type relating to the type of data transfer that is possible:

o Control

o Bulk

o Isochronous

o Interrupt

· An Endpoint has a Maximum Packet Length that it can send or receive.

· For Interrupt Endpoints only there is an Interval value that signals the time between interrupts.

2.3 Transfer Types:

The four different types of transfer are:

· Control – This is a data transfer that attempts to change the configuration of the device. This can include a bulk-style data transfer.

· Bulk – This is the lossless one-time bulk movement of data to or from the device.

· Isochronous – This relates to a steady stream of real-time data where the data carries implicit timing in the rate it arrives. Such transfers require a set allocation of bandwidth. This type of transfer is normally reserved for real-world streamed data such as an audio channel or video stream from a camera.

· Interrupt – This transfer type can transfer a set size of data at regular intervals, timed in milliseconds.

2.4 Host Controllers

The USB specification is general enough to allow different implementations of the Host Controller hardware. There are two common implementations in wide usage, these are known as UHCI[6] and OHCI[5]. UHCI was created by Intel and is considered a more simplistic hardware implementation that requires slightly more software management. OHCI was created by Compaq and is commonly found in all Apple USB implementations. The two different host controllers both completely support the USB standard and will work with all USB devices; the only thing that must change is the Host Controller Driver on the Host Computer. [4]

As part of the USB 2.0 improvements (as discussed below) a new host controller has emerged (the only one to support 2.0). This is called EHCI. Having an EHCI host controller will imply the ability to cope with USB 2.0 devices.

2.5 The Future of USB

At about the same time as USB appeared another standard was announced called IEE 1394, sometimes branded as FireWire[5] an i.link[6]. IEE 1394 does not have all the chaining capabilities of USB but it does have vastly greater bandwidth (400MBPS) than USB. IEE 1394 also has the ability to carry out peer-to-peer connections instead of one acting as a host and the other as a much simpler device.

The emerging consumer digital multimedia market has adopted IEE 1394 widely[7]. It is the most common interface on new digital camcorders and is now present on high-end games platforms. This has demonstrated the demand for higher-than-existing-USB bandwidth when connecting some peripherals, especially in the consumer electronics sector. This has spurred the extension of the existing 1.1 USB specification. In December 2000 the USB 2.0 Specification[3] was released. The principal advantage of USB 2.0 is a new higher bandwidth capability of 480MBPS.

Currently the adoption of USB 2.0 and IEE 1394 is of particular note. There is uncertainty about the adoption of the two standards as many hardware producers wish to standardise on one system. It is currently uncertain as to whether USB 2.0 will be able to usurp IEE 1394’s dominant position in the consumer electronics market but it is unlikely that IEE 1394 will replace USB as a standardised method for connection of peripherals to Personal Computers. An IEE 1394 mouse is a little far fetched. It is unclear whether IEE 1394’s better peer-to-peer networking capability will make it more popular for the interconnection of broadband digital consumer devices.[29]

2.6 The Linux Kernel

Due to its open source distribution the Linux Kernel is extremely configurable and it is very common for experienced users to recompile their own kernel with slightly altered configuration options. The principal area of configuration is what parts are included in the kernel as statically linked parts of a macrokernel and what as dynamically loadable modules, and what is not included at all. The Linux USB sub-system can be implemented in whole or in part as dynamically loadable modules. It is common to keep the Linux-USB core statically linked while dynamically loading the different device drivers. Other features that can be configured in this way is which Host Controller Driver is used by the system and the level of debug information sent to the kernel system log.

The latest stable version of the Linux kernel (at the time of writing) is 2.4.5. Version 2.4.0 was released in January 2001; this was a major update from the previous stable kernel 2.2.17. Minor updates (changes to the third number) happen quite often and do not usually signify major change. Kernel versions with odd middle numbers (e.g. 2.3.x) are unstable kernels that are used for development and experimentation.

2.7 The Linux Virtual Filesystem

The Virtual File System (VFS) is a core part of the Linux Operating System and key to much of its functionality. Linux can mount new filesystems within one unified file-structure. Not all files that appear in the unified filesystem are sitting on the same disk; in fact not all files are actually files on any disk. The Virtual File System allows storage devices that use very different formats to be used together in one filesystem. The VFS has a set interface and filesystems are written to support it. This results in the details of a particular filesystem being completely transparent from the user. The user cannot tell (from how it is accessed) if a file is held on a DOS or EXT2 partition[8] of an IDE or SCSI hard drive or even a Zip drive on a parallel port interface, it is completely transparent.

This flexibility and transparency combined with a very strictly defined interface that has not changed much over time has lead to the use of the Virtual File System for different purposes. Files in the VFS can represent devices and services. Direct interfaces to devices are often found in the /dev file-space. Services such as CPU usage reporting are reported through files in the /proc filesystem.

In Linux a filesystem has an entity called a superblock. The superblock is used to register and mount the filesystem. The superblock contains all details of how a filesystem is accessed and points to functions that define inode structures for particular files. An inode is the kernel reference to a file in a filesystem. The principal function of the Virtual Filesystem is to cache inodes and manage their access. User-programs never actually get to find out the inode of a file. The user program is given a file-descriptor, which acts as a pointer to a reference of the actual inode of the file. This indirection is a feature of the Virtual File System cache.

An inode structure contains details about the file such as modification time, ownership (user and group), as well as a pointer to a file_operations structure. The file_operations structure contains pointers to functions that implement the possible actions on a file such as open, read, write, release, ioctl, etc.

2.8 Linux and USB

Linux adopted USB later than its proprietary rivals. USB support has only really become widespread since the release of the 2.4 kernel[9], which was first released in January 2001. Major Distributions have adopted this kernel in subsequent months and now USB support is a standard feature of most new Linux installations.

Support for USB devices in Linux is now growing dramatically, Major device classes are well supported such as the Human Interface Device (HID) driver (this supports mice, keyboards etc.) and the mass-storage driver (this is used by floppy drives, Zip drives, CDRs, even some MP3 players and Digital Cameras). Proprietary interfaces have also begun to be supported; high profile examples include Alcatel’s ADSL USB modem and Phillips popular digital cameras[25].

The Linux USB Subsystem has been developed by a group of developers spread over the world communicating via the public mailing lists. The basic design is shown in Figure 2. The Linux USB subsystem is a part of the kernel. It is composed of three layers: The Device Drivers, the USB Core, and the Host Controller Driver. Due to the nature of the distributed and disjointed development of open source systems the interfaces between the layers have become tightly defined. This means that there is a tightly defined interface between drivers and the USB core, and the USB core and the Host Controller driver.

Linux USB device drivers are normally kernel based drivers that interface to user applications via a number of methods such as the /dev device filesystem and the SCSI filesystem where USB storage devices are represented as if they are SCSI storage devices. It is possible however, to have user-space based device drivers. These user space drivers interface to the USB core via the Linux USB filesystem.

Figure 2: The existing Linux USB Subsystem

Diagram showing the layout of the Linux USB sub-system.

2.9 Linux USB Host Controller Drivers

Linux has a stable driver for the OHCI host controller, it is commonly known as usb-ohci, however Linux has had several problems with its implementation of the most common UHCI host controller. Two different implementations now exist, known by the filenames of their object code: uhci and usb-uhci. Both drivers are under active development by different people but use significantly different internal methods. The history is complex but in brief the uhci driver is older and for a long time had problems with isochronous transfers; while the newer usb-uhci driver was developed to use similar internal structure and methods as the OHCI driver. There is active discussion about how best to move to a simpler situation for UHCI users. This, along with early development of the EHCI driver has lead to a strong discussion of the possibility and desirability of merging common functions in the different host controller drivers in what is referred to as a Host Controller layer. The exact future of this is currently uncertain.

In theory it should not matter to users or device driver writers which host controller driver a system is using, as the interface to the USB core is common between all the Host Controller drivers.

2.10 Linux USB Data Transfers

The Linux USB core handles all four types of data transfer (control, bulk, interrupt, and isochronous) via one structure. This is called a URB (Universal Request Block). A URB contains all important information about a data transfer including a pointer to the actual data buffer. URBs are queued asynchronously and can trigger completion routines via a callback function specified in the URB (this is also extremely important to how monitoring can be achieved). The Host Controller determines the exact scheduling of the transfers. The URB structure contains parameters that are required to be set before a URB is submitted and results that will be filled in by the core and returned when the URB has completed.

2.11 The Linux USB filesystem

The Linux USB filesystem has its own filesystem. This can be mounted in the /proc filesystem at /proc/bus/usb. The /proc filesystem is a memory-only filesystem that allows the kernel mode to send and receive data as if it were the contents of a file being read from or written to respectively. When a file in the /proc filesystem is read or written the filesystem securely calls kernel mode functions that reply to the request with dynamically generated contents of the requested ‘file’.

The Linux USB filesystem is intended as both a method for relaying from the kernel to user mode applications the configuration details of the USB peripherals and as a method for user-mode device drivers – allowing the sending and receiving of URBs from user-mode processes.

The configuration information is found in two files in the top level of the usb filesystem (/proc/bus/usb). These files are called ‘devices’ and ‘drivers’. The ‘drivers’ file consists of a list of the currently loaded drivers. The ‘devices’ file contains the details of the bus topology and the contents of all the device descriptors of attached devices. This includes all characteristics of devices, configurations, interfaces, and endpoints. A detailed explanation of the rather complicated formatting of this file is in [15].

The device interface part of the Linux USB filesystem is provided in bus directories. There is one directory per bus, the name being the three-digit bus number allocated by the USB core. For single bus systems this will almost always be 001. Inside these directories there is a file for every device, the name being the three-digit device number allocated on connection. So the device interface for device 1 on bus 1 is at /proc/bus/usb/001/001. These device interface files can be used to submit data transfers and carry out other actions on the devices.

2.12 Linux USB development in the future

There has been much discussion among the Linux USB developers about the future direction of development effort. One of the prime efforts is USB 2.0 support. As well as this there is active discussion of standardisation within Linux of the way peripheral interfaces such as PCI, USB, IEE 1394, IRDA, and possibly Bluetooth represent devices and their configuration. This has already led to a standardisation of the arrangements for Hot-plugging devices. The Hotplug system is more general than just the USB subsystem and is used by other parts of the operating system.[22]

The important aspects of this are really what is not about to change rather than what is. There is little prospect of the URB system and the Host Controller interface changing even with the introduction of USB 2.0. The standardisation between peripheral interfaces in Linux should not affect the internals of the USB core or device driver design to any great extent (to break existing drivers on a large scale would be very bad and is extremely unlikely).

2.13 USB on other Operating Systems

USB has been extremely widely adopted. In 1998 there were 138 million USB enabled PCs in the world and that is projected to increase to over 500 million in 2001 with 100 per cent of new PCs supporting USB[30]. This level of support from the hardware providers is matched by almost all widely used operating systems. Different versions of Windows[27] and Mac-OS[28] both have extensive USB support. Because these different operating systems are closed source and tend to have more single-user orientated security models the position of the USB drivers is different. There are often things called ‘device layers’ which make all devices look similar to drivers whatever interface they have. This means that monitoring the goings on of a USB is very different in other operating systems.

2.14 Existing USB Monitoring Solutions

There is an application called USB Snoopy[19] available for Microsoft Windows 98 that allows the analysis of data sent between drivers and devices. This is often very useful to software engineers who are attempting to write ‘port’ a driver to a new operating system such as Linux. USB Snoopy is still at a delicate stage of development and can cause problems with other Windows applications. A Linux USB device driver called usb-robot[20] has been developed that can use the output of USB Snoopy to replay USB transactions on devices where there is no existing Linux device driver, this is used to reverse engineer drivers.

Figure 3: The main window of USBview showing a well populated bus.

On Linux there is no equivalent to the USB Snoopy Windows application in Linux; that is partially the aim of this project. There is an application available for Linux called USBview[21] that displays the topology and configuration of the USB attached to the host. Figure 3 shows a screenshot from USBview. USBview only shows information available in either the devices or drivers files in the Linux USB filesystem. This data is presented in a split frame with a tree format showing each device as a node on the left and the details of a selected device in the right-hand.

The functions that USBview provides are often inadequate for device developers and driver writers. These users need not only to be able to tell what configuration and topology devices and busses have, they need to know exactly what is being communicated and when between the device and the driver. This has resulted in a significant market for expensive hardware monitoring systems that physically monitor the wires in a USB bus. Providers of these include Intel, Cypress, and CATC. These devices are normally extremely expensive and require an additional monitoring computer to analyse the data that they produce.

4 System Design

The system design is clearly split into two parts: the kernel patch and the monitoring application. The kernel patch is a list of additions (and deletions) to the kernel source. This can be applied to a kernel source, the kernel then configured as to the system’s requirements in the usual way and compiled.

The monitoring should be flexible. Monitoring all the data that is passed to or from an endpoint may be excessive. So three levels of monitoring are defined – No Monitoring, URB Headers only, and Full Data Monitoring.

The Monitoring Application is a user-space application that can set the monitoring level of the kernel element and read its output and display this information in a useful way.

4.1 Functional Requirements of the kernel patch

The patch to the kernel needs to enable three things to happen:

1. URBs passing to and from the host computer must be selectively intercepted. This will involve:

a. Trapping all URBs after they have completed but before the driver is notified of completion.

b. Analysing the trapped URB and determining what (if any) details to monitor of it, by referring to a maintained list of monitored endpoints.

c. Additionally the monitored URB should be time-stamped at the time of monitoring.

2. The relevant information obtained from a URB must be stored in a buffer. Execution must then return to the normal kernel operations.

3. The information in the buffer must be transferred to user space when stimulated by the monitoring application.

4. The kernel patch should be able to interpret a correctly formatted command sent from the monitoring application and set the monitoring level appropriately

Attributes that should be present in the kernel patch

1. The patch should have minimal performance impact on USB transactions that are not being monitored. This should mean less than a few hundred extra processor cycles for a non-monitored URB to complete.

2. The patch should not in any way affect the operation of any other kernel service or user space application. All existing kernel services should be completely usable with the patch in place.

3. Changes to existing functions in the kernel source should be kept to a minimum so as to reduce the likelihood of developmental changes to the kernel tree breaking the patch.

4.2 Design of the Kernel Patch

4.2.1 Intercepting URBs

The URB structure is defined in usb.h (in the kernel include files) and is included by all the core source files (see Appendix A for URB definition). The URB contains a pointer to a completion function that is key to the monitoring. The completion function is a device driver function that is referenced when the device driver submits the URB to the USB core. This function is then used as a callback as soon as the URB has completed. This callback is in fact called from within the hardware interrupt handler in the Host Controller Driver that is caused by the completion signal from the Host Controller[10].

Initially the idea was to replace this completion call with a new function that performed the monitoring required and then called the original device driver completion function. This was simplified slightly to adding in a new function that is called just before the completion call, with the Host Controller driver’s call to the completion function remaining in the code. So when modified the Host Controller driver will call a ‘pre-completion function’ that will perform the monitoring then the Host Controller driver will call the original device driver defined completion call.

The advantage of this method is that it means less change to the URB structure and causes less intermediate processing to all URBs, thus slightly reducing the performance impact of the kernel patch. The principal disadvantage is that this requires a modification to several places in each of the host controllers. This is an annoyance from a design point of view as it extends slightly the interface between the USB core and the Host Controller Driver. This decision is finely balanced between marginal performance impact versus source code ugliness and reduced future flexibility (in that any new or radically changed host controller driver will also require minor modifications). This is a deviation from the original plan where it was aimed to only modify the USB core while leaving the Host Controller Drivers unaffected by the kernel patch.

4.2.2 Interpreting the Contents of a URB

It is all very well calling the pre-completion function for every URB but not all URBs will need to be monitored. For example it may be that a system includes a USB keyboard, which will cause a URB to occur every time a key is pressed.

The URB structure stores the information about the device, endpoint, direction, and type of the URB in a structure called a pipe. This is in fact an 18-bit structure encoded in an unsigned integer (see Appendix B for details). Assuming that the pre-completion function has access to a list of monitored device-endpoint combinations (see section 4.2.6 on page 34) it can use values that represent masks of the pipe structure to check if the monitored URB is on the list of monitored pipes. So the pre-completion function will take a URB as a parameter and compare its pipe with each of the pipe masks it has in a list of monitored pipes.

Once the pre-completion function has determined that the URB is one that should be monitored it can copy all the appropriate details into a new URB structure (including the data buffer depending on monitoring level). The pre-completion function must also timestamp this new URB. There is no timing information available within the Linux USB subsystem so this is an addition to the URB structure in the kernel patch.

4.2.3 Storing the Monitored Information

Once a URB has been copied the copy must be stored somewhere. It cannot immediately be sent to user space as it is being copied in the middle on an interrupt handler so time is of the essence. The monitored URBs must be stored in a buffer. All the monitored URBs could be held together in one big list. The disadvantage of this is that user-space applications would have to obtain the whole list when they may only want to see if one device has any URBs in its buffer. Separating the monitored URBs at the device level allows multiple different monitoring applications to operate concurrently and monitor different devices. The reason for this is explained below in 4.2.4 Passing the Monitored Information to User Space.

The list of monitored URBs could be separated at the endpoint level, however this would add a small amount of extra processing on every monitored URB (calculating the endpoint, and then finding the correct endpoint structure) as well as increasing the complexity of transporting URBs to user-space. There are no obvious advantages in separating the monitored URBs at this level, as it is very unlikely that two concurrent monitoring applications would want to monitor different endpoints on the same device. The most logical answer then is to separate the URBs at the device level maintaining a separate buffer for URBs on each device.

As well as the pipe information the URB carries a pointer to a usb_device structure. One usb_device structure is maintained for each USB device that is currently on the Bus. This structure contains details about the device and references to structures describing the configurations. This structure provides the perfect location to store the monitored URBs. The functions that pass the URBs to user-space can easily access the correct usb_device structure; this is explained in section 4.3.1.

4.2.4 Passing the Monitored Information to User Space

Moving data from the kernel mode to the user mode and vice versa in a relatively secure operating system such as Linux is surprisingly difficult. The most classical method that was looked at for some time is the idea of new system calls that could be called from user mode to both set the monitoring level and request the dumping of a buffer of monitored URBs down to user mode. This method is largely workable but has various side effects that make it undesirable from an operating system design point of view. Firstly this method has some significant security issues in allowing the dumping of kernel space memory on request; there is ample room for the introduction of security holes. Also the new system calls would have to be statically compiled into the Linux kernel. This would change the nature of the USB sub-system, as at the moment it is possible to compile it all as dynamically loadable modules. Another problem would be the enumeration of new system calls. It is entirely probable that new system calls will be introduced to Linux in the future and these would then introduce incompatibilities with this kernel patch.

A simpler method to move data from the kernel space to the user space is to use the /proc filesystem. Since each ‘file read’ on a file in the /proc filesystem is serviced by a definable kernel function this function can be set to dynamically generate the correctly formatted URBs. This system can therefore dump a monitored URB buffer from the kernel mode to a user application.

The creation of an individual /proc file in kernel code is quite easy, however it is much simpler in terms of creating the correct files at the same time to use the Linux USB sub-system’s own filesystem that is mounted within the /proc structure. This filesystem contains a directory for every maintained bus with each bus directory containing a file for every device on that bus, with the three-digit device number (allocated by the USB core on connection) as the filename.

So as not to break any currently working system all these files and their behaviour should not be changed. Under the modified kernel a new file could be added for every device. As well as a “***” file (where *** is the device number) a second “***M” file would also be created. These monitoring files would be set up in the same way as the existing device files but with different ‘read’ and ‘write’ calls. When a URB for a monitored endpoint completes it’s relevant details are copied to a monitoring buffer. When this file is next read then the buffer is outputted to the file. This action empties the buffer, thereby limiting this system to one monitoring application in operation at any one time per USB device. This is a limitation of this design and there is no simple method around it.

4.2.5 Formatting the Contents of the Monitoring Files

The content of the monitoring file is the information that has been monitored by the kernel and is being passed to the user-space application. This information can contain two main parts: the header information and the actual data buffer. Only on endpoints set to ‘Full Data Monitoring’ is the data buffer included. The URB header contains values from the URB structure that could be useful to monitor as well as a time stamp generated at the time that particular URB was monitored. The values from the URB structure that are passed over are: device number, endpoint number, pipe, status, size, transfer flags, actual length, and error count. The meaning of these values is shown in Figure 6.

Device number	The number (as given by the USB core at connection) of the device this URB was sent to or from
Endpoint number	The number of the endpoint this URB was sent to or from
Pipe	A structure containing device and endpoint numbers as well as transfer type and direction.
Status	The status of the URB when monitored. This indicates whether it completed or failed in some way.[11]
Size	The intended size of the data transfer
Transfer Flags	Flags that can affect transfer scheduling
Actual Length	The actual size of the associated data buffer.
Error Count	The number of physical transmission errors that occurred during the transfer of this URB

Figure 6: Table showing Items in a URB header

The information must be structured in such a way that the monitoring application can parse it easily and unambiguously. It is also useful for debugging and advanced use of the system that the content is easily readable by a user.

A decision was taken to use plain ASCII text to convey information about a URB. This fits with the two attributes above and also continues the style of most /proc files. Linus Torvalds (the original author of Linux) has stated that binary data should only be present in the /proc filesystem when absolutely necessary [13].

For the URB header information plain ASCII text is fine, but the data buffer of the URB must be sent in full binary format. The header information contains details of the lengths of the data buffer. The structure of a monitored URB being reported in a monitoring file is shown in Figure 7.

[URB completed on Device=%03d, Endpoint=%02d, Pipe=%08X, status= % 02d, size= %08d, transfer_flags = %08X, actual_length = %08d, error count = %05d completion_time = %02d:%02d:%02d:%06d dp=%01d data=]\n

Figure 7: URB encoding in a Monitoring File (Number values are shown in C format where %2d is a 2 digit decimal, and %08X is an 8 character long hexadecimal number)

In order for the Monitoring Application to easily parse the content of the monitoring file it can be important that the length of the ASCII header is fixed. This can be achieved by placing leading zeros at the beginning of possibly multi-digit numbers. In C notation this is represented by a 0 before the number of digits in a number modifier, i.e. %02d instead of %2d for a 2 digit number. Note that the status number in Figure 7 uses the C notation %[SPACE]02d and opposed to %02d. This means that if the status is negative (which is possible) a minus sign will be inserted and if positive a space inserted so that the length of the header will not change. Keeping the header length constant is important for some parsing systems as some systems deal very differently with ASCII and binary information.

4.2.6 Setting the Monitoring Level of Devices

Given that the ***M monitoring files are being created anyway to be read from, a logical extension of this idea is to use the writing case to set the monitoring level. A simple command structure can easily be defined where a few bytes represent a monitoring level setting command. Because there are monitoring files for every device the command need only state the particular endpoint and the desired monitoring level. The formatting of the command need not be very complex at all. One option would be simply to have a two-byte command where the first byte is the endpoint number and the second byte is the monitoring level (in fact given the maximum number of endpoints this could all be encoded in one byte), however this is too inflexible for future development. It was decided that a four-byte command structure where the first and third byte is fixed would provide greater redundancy for future features. The first and third byte can be verified by the kernel as a check against accidental input from another source.

Byte No.	1	2	3	4
Value	‘E’	Endpoint Number	‘L’	Monitoring Level

Figure 8: Table containing the byte coding of commands to set the monitoring level. All bytes are plain-text encoded ASCII.

Once the endpoint value and monitoring level (and the device and bus numbers derived implicitly from the path of the file) are known by the ‘write’ function in the filesystem (remember a ‘write’ means information going in to the kernel) it must store them somewhere. The two considerations on deciding where to store this small amount of data (the endpoint to be monitored and the level at which to monitor) are that the location must a) be easily known by the filesystem call that must write to it and b) be easily known by the pre-completion function during the monitoring of URBs. Of these two factors the second is much more important as the pre-completion function is inherently the most performance-sensitive part of the system.

An option would be to have a globally visible list of monitored device-endpoint combinations, however this is seen as bad in terms of kernel design due to it’s lack of scalability and reliance on one unified global structure. In fact, as it turns out, both functions can have very easy access to the usb_device structure of the relevant device. This is possible for the filesystem function because of a field in the file structure, which is used in the kernel to represent files. There is a space called ‘private_data’ where the files system can hide a pointer to a structure of its choice for later reference. For the data outputting functions a pointer to the appropriate usb_device structure was already being placed there so that the reading function (a user ‘read’ means a kernel output) had access to the monitored URBs. This can be re-used in the writing functions to allow the setting of a monitoring level variable in the usb_device structure.

At first it seems odd for the pre-completion function to have easy access to the relevant usb_device structure as it is dealing with URBs appearing on all devices not just one at a time, but in the URB structure that it gets passed as a parameter there is a pointer to the relevant usb_device structure so the pre-completion function will always be able to reference straight to the correct usb_device structure.

Maintaining a list of monitored endpoints on a per device basis has significant implications on the performance of all URB transfers during any monitoring compared with keeping a unified list of all monitored device-endpoint combinations. The speed with which the pre-completion function can determine whether the URB it is analysing is to be monitored or not is proportional to the number of comparisons it must make with device-endpoint combinations. Using this method of storing the monitored endpoints of a device in the usb_device structure reduces the number of comparisons to the number of monitored endpoints in that device rather than on the whole system. This m eans that performance is reduced only for a device on which there is monitoring being performed, others are largely unaffected.

4.3 Kernel Patch Implementation Issues

4.3.1 Adapting the USB filesystem

Within the Linux USB subsystem source code the implementation of the Linux usb filesystem to which additions are made is split between two main source files known as inode.c and devio.c. The devio.c file contains all the filesystem functions that provide the functionality for the device interface files in the bus directories. The inode.c file contains the functions to define these files and build them up in a filesystem and register that filesystem. So for example when a new device is attached a function in inode.c is called that defines a new inode structure, adds references to the functions in devio.c, and adds this inode to the superblock. Then when a directory in the filesystem is read a function in inode.c will return the list of inodes in that directory. If the user space wants the filesystem to open a file it will use the function pointers in the relevant inode structure to call the device file opening functions defined in devio.c. The same method will cause the reading functions in devio.c to be called if the file is read.

So in order to add the monitoring files to this system a number of things must be done. Changes must be made to inode.c to create inodes for the additional monitoring files. These inodes must be added to the superblock in the same way as the device interface files but must have pointers to different file operation functions replacing the existing functions in devio.c.

These additional file operations functions must handle the opening, closing, reading and writing of the files. In order to have access to the appropriate data there is a field in the inode structure that allows a pointer to the relevant usb_device structure. This is what allows access to the URB data that has been monitored. It is important to remember here that file reads translates to the kernel ‘writing’ information to user space and vice versa in reading.

The added opening function for the monitoring files moves the monitored URB buffer to a new buffer, so that monitoring can continue to the old buffer while the user reads this temporary one. This temporary buffer is destroyed by the closing function. This means that when a monitoring file is opened and closed the monitored URBs up until the open are lost from kernel space. This is the attribute of the system that prevents multiple user programs monitoring the same device at one time.

4.3.2 Implementing Timing in the kernel

Implementing timing routines proved more difficult than expected. This was primarily due to incompatibilities between the different versions of timing structures in Linux. Kernel functions do not have access to the standard C library at all but have most of the functions normally provided by the C library provided by very similar Linux calls. However there are sometimes conflicts between these definitions.

There were several options of what structure the timing data could be stored in. The classical C time_t type could be used and set by the time() function. The disadvantage of this is that the granularity is only in seconds, which is not small enough for many of the possible applications that this timing information might be wanted for.

Another option is the timeval structure that includes a time_t-like value and a value for the number of microseconds within that second. This is perfectly adequate for the USB monitoring purposes. There is another option that is a result of the POSIX 4 specification and that is called the timespec structure, which is doe to supersede the timeval structure. This is similar to timeval but stores nanoseconds instead of microseconds. Unfortunately timespec support is incomplete in Linux and although some of the support functions are available it is not considered ready to be used.

So the logical decision to use timeval structures was made and implementation attempted. Obtaining and storing the timeval structure proved no problem. However, it proved impossible to use the normal functions available in C to decompose the time_t part of the timeval structure due to the fact that in the Linux kernel include files there are two possible time.h files that can be included. One file that provides the timeval structures and the time-obtaining functions and one that provide the original time_t structures and the functions to decompose this time_t functions. These two files cannot be included together due to subtle incompatibilities. This is only the case for kernel functions.

After attempting many possible solutions the only workable possibility was to implicitly code the decomposition of the time_t type rather than rely on the normally available functions. This is not too complex as date information is not really relevant so can be disregarded. The time_t type is merely a value giving the number of seconds since January 1^st 1970.

4.3.3 Problems with Interrupt Handlers

For a period of time during the development and testing of the kernel patch the development system exhibited occasional but very fatal kernel crashes (the whole system suddenly froze). This was very intermittent but usually during Usb activity and almost certainly connected to the kernel patch. Searching for a bug that can cause this kind of error is not easy and after some attempts there appeared no easy way of determining the cause. At a later stage of development when full data monitoring was finally implemented in full the system exhibited exactly the same crash symptoms but this time in an entirely repeatable way. The system crashed when attempting to allocate some memory.

After some research the cause was discovered. When in the kernel, memory is allocated using kmalloc() instead of the normal malloc() function. kmalloc() takes an additional parameter which is a flag specifying the priority. GFP_KERNEL had always been used as the second value, which specified the correct priority for normal kernel service. However, the URB intercepting pre_completion function is run from within an interrupt handler and as such is not allowed to sleep under any circumstances. In order to stop kmalloc from sleeping under any circumstances it must be called with GFP_ATOMIC. This means that kmalloc will occasionally fail to allocate memory but will not sleep and therefore will not cause a systemic crash. Changing all the kmallocs in the interrupt handler fixed the intermittent crashes. Only very occasionally does a kmalloc fail in which case it is reported to the system log.

4.3.4 Changes to different Host Controller Drivers

The only alterations to the host controller drivers that is required is the addition of calls to the pre-completion function just prior to all calls to URB completion functions. The different Host Controller Drivers have different internal structure so the number and location of these calls varies widely. The ohci host controller driver centralises its completion call in one function but the usb-uhci driver makes USB completion calls from 11 different locations in the source code. This means that there is literally a one line addition required to the latest ohci host controller driver, whereas it about 15 for the usb-uhci driver. The smaller the number of changes the more likely the patch is to work on future versions of these drivers.

4.3.5 Where to put the code

For a lot of the changes to the kernel code there is no option on where to put the code as it is made up of small modifications to existing functions, however when introducing new functions into the kernel there is a choice between adding to an existing file or adding in a new source file to package up all the functions to do with the patch. This second method would make it easier to make the patch a fully-fledged kernel configuration option, however this would mean changing various Makefiles as well as creating the new files.

Primarily due to the required changes to Makefiles and configuration files it was chosen not to create a new file for the new monitoring files. Instead the new functions were placed at the bottom of the devio.c file as almost all are related to the extensions to the Linux USB filesystem functionality, which is implemented in the devio.c file.

4.3.6 Compatibility on different kernel versions

The kernel modifications were developed using a stock Linux kernel version 2.4.1[12]. The vast majority of changes that are introduced in successive versions of the kernel are fixes to old device drivers, new drivers, or fixes to the Host Controller Drivers. This means that as most of the modifications for this application are to the USB core and in particular the usbdevfs filesystem there should be little conflict with later versions of the Linux kernel. The modifications made to the kernel for this system are mostly additions and specifically adding in new functions, this should reduce further the likelihood of conflict.

When releasing a working example of this system the patch was applied to a 2.4.4 kernel and found to fail in two areas. These two areas were very quickly fixed and were a result of design changes within one of the Host Controller Drivers (usb-ohci). This highlights the disadvantages of the method chosen for intercepting URBs (as discussed in section 4.2.1 on page 30). Whenever the code in a host controller is changed around the calling of the URB completion callback the kernel patch may no longer apply. The Linux Patch utility can cope with some degree of small change but if the actual design changes then this requires manual intervention.

5 System Design – The Monitoring Application

Monitoring applications can have a multitude of functionality that helps the user deal with the data that is monitored. This is ancillary to the core purpose of this application. The main aim of this monitoring application is to fully demonstrate the functionality of the kernel patch and provide a useful and usable interface to it.

A simple command-line interface is too simplistic and inadequate at displaying effectively the monitored data. A graphical user interface provides a much greater ability to show the complexity and flexible nature of the monitoring abilities of the kernel patch. The production of advanced applications using complex GUIs is much simpler now than it has been in the past and there are many libraries that make the implementation of ‘standard’ looking GUIs a minimal extra effort on top of the main application functionality.

5.1 Functional Requirements of a Monitoring Application

1. The Monitoring Application should be able to detect from the existing Linux USB subsystem the topology and configuration of all the attached busses.

2. The Monitoring Application should be capable of setting the monitoring level in the modified kernel of different endpoints independently.

3. The Monitoring Application should be able to read the new monitoring files in the /proc filesystem and parse them in a way such that it can store data about monitored URBs.

4. The Monitoring Application should be able to display the details of URBs on different endpoints on different devices.

5.2 Attributes desirable in the Monitoring Application

1. Running the Application should not heavily impact General System Performance.

2. The Application should be relatively simple to use and data should be easily obtained by anyone experienced in the concepts of USBs.

5.3 Design of the Monitoring Application

5.3.1 Structure of the USB Model

The Monitoring Application must maintain a model of the USB connected to the Host system that the Application is running on. This model must be flexible enough to hold any possible state that a USB can be in. It must also be able to hold all available configuration information about all devices attached to the bus including details about the configurations, interfaces, and endpoints.

Figure 9: Simple Object Model representing a USB configuration

Using Object Orientated Methodology this model contains various classes of object such as Bus, Device, Configuration, Endpoint, and Interface. These classes are interconnected by a form of ownership. For Example a Configuration has one or more Interfaces and each interface has one or more endpoints associated with it. This leads easily to a simple Object Model of the whole bus. In Figure 9 a simple object model is shown for a bus, obviously more than one of these can be held for a system. The details of the implemented USB model objects are contained in Appendix D.

This Object Model provides the perfect location to store the monitored data that is received from the kernel via the monitoring files in the /proc filesystem. This is best kept in the Endpoint objects as this is the level at which monitoring can be switched on or off. Note that this is different from the kernel patch where URBs are buffered in device-level buffers as opposed to endpoint-level buffers. This is not a problem but simply highlights different priorities in different parts of the system. The monitored data can be kept in a new object called URB. This will contain all the attributes that can be monitored about a URB as well as a time stamp that is attached by the kernel patch at the time of monitoring. Optionally the URB object can also contain the data that the URB was transferring.

5.3.2 Obtaining the Topology and Configuration Information

All the topological and configuration information is obtained from the ‘devices’ file in the Linux usb filesystem, as referred to in section 2.11. Its structure is exhaustively defined in [15]. The file gives seven different types of line each describing different sets of characteristics that devices, endpoints, configurations, and interfaces can have. This contains all the characteristics from the device descriptors available to the Linux USB subsystem. This file can be parsed to give the structure and contents of the object model. As the object is parsed the object model can be built up and the fields filled in.

5.3.3 Polling for the Monitored Data

In order to keep receiving monitored URBs as they are monitored in the kernel the monitoring application must poll the /proc monitoring files at regular intervals. This is best achieved in a multi-threaded architecture where a separate thread is started for each monitored device. These threads can simply read the contents of the monitoring file (thus emptying the kernel buffer), construct a URB object for every URB in the monitoring file and place that URB in the list in the relevant endpoint structure.

This design does call for a multithreaded architecture accessing a unified set of data structures. This would imply that there are thread safety contention issues, however because the ‘reader’ threads are only updating a particular set of structures that no other reader thread can be updating and the display threads will only ever read values from these structures for display there is no danger of thread contention problems.

5.3.4 Displaying Configuration and Monitored Data

In deciding how to best present the data it is important to remember the different information that people may be interested in finding out. One of the primary concerns is making it simple to navigate around a bus and select devices of interest. It can be assumed that target users would want to focus on the details of one device at a time although retaining the ability to monitor multiple devices simultaneously.

It is clear that USBview (see page 22) has an efficient way of navigating around a USB bus. A tree is a logical mechanism to navigate around a bus and select a device of interest and USBview shows this. In USBview another pane contains all information about a selected device, however here USBview’s solution of having one pane for all information is inadequate for this system. The potential amount of information that will be stored in the USB model about a device could be huge and far too big to display all in one pane. The solution is to break this information down into different areas and make the second pane selectable as to which information is shown.

The different types of data that a user of the monitoring application will be able to access includes:

· Basic configuration information from the device descriptors (as is given in USBview).

· The Monitoring Level settings of the different endpoints in each device.

· The monitored URBs for each of the endpoints showing the characteristics of each of the URBs and the ability to show their data (if present).

This division lends itself to a second pane having three possible states each serving the three possible uses described above. The three states should be easily switchable between but need not be displayed simultaneously.

Given the way USB devices work, using different endpoints for different parts of one task, it is important to be able to see easily the characteristics of monitored URBs on different endpoints. To this end the URB characteristics can be placed in tables with scrollbars that allow the viewing of a small number of URBs for one endpoint at the same time as a small number of URBs on other endpoints.

On monitored URBs with data attached the data need not be shown all the time. Often there will be a lot of it and most of the time may not be essential to the detail of monitoring. USB monitored data can represent many different things. And as such it would be useful to be able to display the data using different

5.4 Monitoring Application Implementation Issues

5.4.1 Java and Swing vs. C and Gtk

The implementation language is significant primarily because of the library available. The Gtk library is a C based library that is widely used in Linux GUIs. The Swing Library is an integral part of the Java language that allows extremely flexible and intelligent presentation in a standardised cross-platform style.

The decision as to which language to implement the monitoring application in is largely irrelevant. Both language/library combinations are more than adequate for the proposed functionality. In the end the decision was made to use Java and Swing primarily to use the stronger object-orientated features of Java. This proved very useful in implementing the USB object model in the Swing GUI.

5.4.2 The USB Object Model

As described on page 42 the model of the USB can easily be based on a collection of five different classes of object: Bus, Device, Configuration, Interface, and Endpoint. The exact details of these objects are shown in Appendix D.

Many of the objects need to keep variable sized lists of objects. For example a device needs to keep a list of child devices attached to it and a list of possible configuration objects. In these cases variable sized lists have been implemented as Vectors.

5.4.3 The Threaded Monitoring File Reader

Interfacing with the kernel via the monitoring files in the /proc filesystem the Monitoring application must update its usb model as and when it receives new monitoring information. In order to achieve this the Monitoring Application must regularly poll all the relevant monitoring files in the /proc filesystem. Java provides an easy method to set up a thread that will repeat an action repeatedly after a specified time interval. This is via the Timer and TimerTask Classes. The Timer class sets up a timer that will run a method in a TimerTask Object at a set frequency. In setting the frequency of polling it is desirable to have it often enough so that the monitoring appears almost instantaneous while not so fast as to impose too high a workload on the kernel functions that dump the buffers down to the monitoring application.

The actions inside this TimerTask method are simply to:

1. Open the relevant monitoring file.

2. Read all the contents into a byte array.

3. Parse the contents of the byte array and construct URB objects to represent the contents.

4. Place those URB objects in the relevant Endpoint object.

5. Close the file.

The parsing element of this process is slightly complicated by the way java treats text. Since its inception Java has treated all characters as being 16-bit Unicode entities. It has different objects for dealing with character streams (which are converted into Unicode) and byte streams (which are left as unsigned bytes). A problem arises with the use in this system of one monitoring file to deliver a stream containing plain ASCII text descriptions of the URB headers and binary data representing the data content of the URB in the same file. Here the fixed length nature of the URB header in the monitoring file becomes important. Different methods can be used for the header part and the data part. Once the header part has been read, the presence and size of the data part can be determined. Then the data part can be read using an alternative data method that reads the data as bytes instead of Unicode characters.

5.4.4 Displaying the Device Tree

One of the most useful features of the Java/Swing combination is the ability to create standardised user features like tables and trees from different object types as long as they implement an interface by including a few specific methods. This means that the USB model objects can be used as the models for the display structures.

In order to achieve a tree of devices that can be navigated to select devices of interest the Device class can be made to implement the TreeModel interface. This requires the addition of eight methods that allow the implementation of the tree such as get_child(), get_child_count(), is_root(), etc…

Using this method the collection of Device objects in the USB model implements the tree model. A navigable tree can easily then be created using the JTree class. The navigable tree is visible in Figure 10.

5.4.5 Splitting Up the Information

The information displayed about a device selected on the tree has to be selected from three options. The best solution for this is to use a feature of Swing called Tabbed Panes. This allows the display to be quickly changed between ‘screens’.

5.4.6 Displaying the URB Tables

The same method that was used to implement the trees can be used to implement tables. A class can be made to be an extension of the AbstractTableModel class. This involves adding six new methods such as getRowCount(), getColumnName(), getValue(), and setValue(). This turns the class into a table model and a simple table can easily be displayed using the JTable class.

There are two locations where tables are useful. In the Configurations pane different interfaces need to show all the details of their endpoints and the set monitoring level.

The first case can be achieved by making changes to the Interface class. Making a table from the interface class gives a table of endpoints with their characteristics. The monitoring level of the endpoint needs to be editable. There are only three possible values. To avoid erroneous input an editor called a ComboBox can be attached to this column that allows the selection from a drop-down menu. Figure 10 shows the configuration pane showing the endpoints of a USB storage device (in fact a floppy drive).

Figure 10: The Configuration pane showing the endpoints of a USB storage device

The second case is where the details of URBs monitored on a URB need to be displayed. In this case each table model can be represented by an Endpoint class. Similar extensions to those made to the Interface class can be made to the Endpoint class to allow the display of the URBs’ details.

Figure 11: The Endpoints pane showing URBs monitored on a USB storage device

Figure 12: URB data window showing the buffer in hex view

The data element of the URB need not be shown in the table. It is much better if this is shown in an independent window as then it can be compared side to side with other URB data buffers. To achieve this another ComboBox can be used to provide a drop-down menu on URB rows that have an associated data buffer. The data can then be displayed in a new window. The data that is passed in URBs can have very different meanings and so different types of encoding can be used to view the data. A tabbed pane is used to allow access to Hexadecimal, Binary, and Textual views of the data.

6 System Evaluation

6.1 Suitability of Core Design

On purely functional and performance grounds a system that unobtrusively intercepts the USB traffic on the cable outside the host subsystem and passed the monitored information to a separate monitoring computer provides the most unobtrusive and accurate monitoring system. This is why large companies succeed in selling such systems at very high cost. However this system provides a solution to many problems as effectively as an expensive hardware solution.

Types of problem that can be detected and analysed using this system that it was not possible to do before without hardware monitoring:

1. Analyse the data sent to and from a closed source driver.

2. Debug the data sent to and from an open source driver without changing the code.

3. Clearly show the relative activity of different devices on a bus.

4. Determine the time taken between a driver receiving a URB and sending a reply URB.

5. Determine whether errors in the data transmissions are occurring (as drivers may not report them).

Areas that this system cannot observe and is not able to aid:

1. Analysing bugs inside the Host Controller hardware.

2. Analysing bugs inside the Host Controller Driver software.

3. Determining the accuracy of power distribution across the bus.

4. Obtaining the timings of sub-parts of data-transfers such as the time difference between a control request and the acknowledgment.

5. Debug erroneous transfers from devices that are not of the correct format.

Prior to this project Linux users were unable to carry out any real monitoring of USBs without specialist hardware. USBview (as described on page 22) only provides configuration information and only really displays the information that is already available in the devices file in the Linux USB filesystem. This project does provide a new capability to USB on Linux but it does not replace the need for hardware monitoring systems completely. Experts who are designing complex devices and fine-tuning first time device drivers will doubtless want to be certain about the exact signals that are being transmitted. However there is a group of part-time developers who are writing device drivers for niche devices that have been implemented on other operating systems. For this class of device driver developer this system is a very cost-effective solution to debugging the operation of devices and device drivers.

6.2 Effectiveness of Implementation

Writing code for the kernel is often portrayed as a kind of black art where the traditional rules of software do not wholly apply, a bit like the quantum mechanics of programming. This is less true of Linux due to the number of people involved in its development and the amount of documentation available, however there are still some major complexities that must be taken into account.

The kernel patch has proved largely stable. Although not thoroughly or rigorously tested for memory leaks and other possible low-visibility defects, the patch has proved stable being run as the primary kernel on the development system throughout the progression of the project. This has shown both that the (patched) kernel is stable for prolonged periods of time (over a week between re-boots) in that it shows no outward sign of deterioration in either normal use or USB performance.

After the kmalloc() priority changes (see section 4.3.3) there were no more unexplained kernel failures.

There are some areas where the monitoring system is less stable than might be desired. It only has a partial solution to the issue of hotplugging, and can give unwanted error messages when devices are disconnected.

The way the monitoring application handles alternate interfaces is not ideal. The concept of alternate interfaces was largely forgotten about during the design as none of the devices that were experimented on had them. The problem is that each alternate device’s endpoints are given their own endpoint object and assumed to be active (which is not always true). This does not prevent monitoring but is certainly not ideal as the display becomes very cluttered by endpoints that are not relevant to the user.

6.3 Impact on the Linux USB Subsystem of the kernel patch

One of the key attributes desired in the system was that is did not impact on normal USB operation. Verification of this was done in two parts:

1. Impact on Functionality - The Impact on performing normal USB operations and looking for any loss of functionality in USB using operations.

2. The Timing Impact of Monitoring - Obtaining timing information for the added processing done to non-monitored URBs.

6.3.1 Impact of Functionality

Testing functionality was achieved by carrying out a variety of normal tasks that used a range of transfer types and amounts of bandwidth. Items that were tested included:

· A USB mouse (high frequency of interrupt transfers). This device was as a standard mouse input to X Windows environment.

· A USB web cam streaming video image (primarily using isochronous transfer). A video stream was set up and displayed on screen using the Video4Linux API.

· A USB floppy drive using the usb-storage driver (uses bulk, control, and occasional interrupt transfers). File transfers were carried out on a variety of file sizes.

The results were that no difference in behaviour was observed under any of the specified operations.

6.3.2 The impact of Monitoring on processing time

The principal addition in the kernel patch that affects all URB transfers regardless of what the monitoring level is the pre-completion function that is run on every URB prior to the driver being informed of completion. The amount of processing that is carried out in this function is a measure of the cost of monitoring.

Analysis of the code in this function can give factors that affect the amount of processing that will occur. The source code to the pre-completion functions is in Appendix C. The factors that can be deduced from the structure of the code are:

· For devices on which none of the endpoints is monitored in any way there is very little processing that will occur. This is because no comparisons are necessary to check whether the URB needs to be monitored.

· The amount of processing for all transfers on a device with monitored endpoints will increase with the number of endpoints that are monitored. This is because even for transfers on endpoints that are not monitored the function will have to check the endpoint value against more ‘monitored endpoint’ values to determine whether or not this endpoint is on the list to be monitored.

· Transfers on monitored endpoints will require more processing than transfers on endpoints that are not monitored. This is because allocating the memory and copying the details of the URB takes processing time.

· Transfers on endpoints with ‘full data monitoring’ will require more processing than transfers on endpoints set to ‘URB headers only’. This is because of the extra memory that must be allocated and the extra copying.

To obtain the timing information, a modification can be made to the pre-compilation function to get the length of time that the system takes to carry out the processing in that function. In a normal user-space C program this could easily be done using the clock() call and clock_t variables, however the problem that was discovered when implementing timing in the monitoring system (see section 4.3.2 on page 37) resurfaces. The clock() function gives the number of clock cycles that have occurred. This value looks back to zero every few minutes on a fast 32-bit machine but is useful for very accurate timing. Unfortunately the timing functions that are available within the Linux kernel do not include the clock() function. It seems odd that user space can access a service from the kernel that code in the kernel cannot easily access itself, but it was not possible to find a way to do this. There is no evidence in the kernel itself or in any kernel documentation of a similar function.

An alternative to the clock() is to use the timing functions used to give a monitoring time accurate to 1 microsecond. On a fast PC several hundred machine instructions will occur every microsecond but this is just enough to determine the delay caused by the processing in the pre-completion function. The absolute time (in microseconds) can be got at the start of the function and again at the end. The difference can then be sent to the system log. This does not include the additional cost of calling the pre-completion function and returning, this is assumed to be irrelevantly small. The timing results give the approximate amount of time taken to perform the contents of the pre-completion function. It is not completely accurate because the start and finish times are not necessarily on microsecond boundaries so there could be an error of up to 2 microseconds. Taking that into account and after experimenting with this timing mechanism it was decided to average the results over several transfers.

The testing that was timed consisted of file transfers to and from a USB storage device. USB storage devices use the Primary Control pie as well as two bulk pipes (one in and one out), and an incoming Interrupt pipe. File transfers will cause activity on all these pipes. The device was re-mounted between every test in order to clear any file caches. The first run was also repeated to counter the possible effect of memory caches in the system.

Two sets of tests were performed on a relatively high specification machine (650 MHz Pentium III with 128Mb RAM):

Set 1: URB Only Monitoring – All endpoints were initially set to ‘No Monitoring’ with the number of endpoints set to ‘URB only monitoring’ incremented after each timed run. After each timed run the pre-compilation time for 10 transfers on monitored endpoints and 10 on unmonitored endpoints were averaged to a figure.

Set 2: Full Data Monitoring – All endpoints were initially set to ‘No Monitoring’ with the number of endpoints set to ‘Full Data Monitoring’ incremented after each timed run. After each timed run the pre-compilation time for 10 transfers on monitored endpoints and 10 on unmonitored endpoints were averaged to a figure. Here the timing of monitored data points refers only to 512 byte bulk transfers. The usb-storage driver will only shift data in blocks of 512 bytes so this is a good standardisation to eliminate variance depending on the size of the data that is copied.

Set 1: URB Headers only:

Number of Monitored Endpoints	Average time taken on unmonitored endpoints / microseconds	Average time taken on monitored endpoints / microseconds
0	0.5
1	0.6	1.9
2	0.4	1.8
3	0.9	1.4
4		1.5

Figure 13: Table and Graph showing the results of timing analysis of the pre-completion function.

The results of this first set of tests are largely as expected with one slight anomaly. The fact that time taken by monitored transfers is greater than unmonitored transfers is as expected, as is the fact that the time taken by unmonitored endpoints appears to marginally increase with the number of endpoints. The anomaly is that the results indicate that as the number of monitored endpoints is increased the length of time taken inside monitored endpoints appears to decrease. There is no obvious explanation for this. The important conclusions that can be made from these results are that the number of monitored endpoints (up to at least four of a possible sixteen) does not have significant impact on the USB performance of a device and that even monitored transfers are only impacted by about two microseconds per transfer.

Set 2: Full Data Monitoring:

Number of Monitored Endpoints	Average time taken on unmonitored endpoints / microseconds	Average time taken on monitored endpoints / microseconds
0	0.5
1	0.8	5.1
2	0.7	5.3
3	0.8	4.9
4		5.2

The results of the second set of tests shows that Full Data monitoring is much more expensive in processing time that simple URB header only monitoring. Full Data monitoring on a bulk transfer of 512 bytes adds about 5 microseconds to the amount of time taken to process a URB transfer.

The significance of this timing data on potential users can only really be guessed at. USB allows interrupt transfers to happen every millisecond and it is conceivable that when monitoring such an endpoint the difference between a five microsecond penalty as opposed to a two microsecond penalty might be important. Unfortunately it is very difficult (and beyond the scope of this project) to put these figures in context by benchmarking the other elements of the Linux-USB subsystem that are essential to its operation.

6.4 Performance of the Monitoring Application

The monitoring Application’s performance is much less important than the performance of key parts of the kernel patch as the monitoring application runs at a much lower priority in user space and can be interrupted at any time. It is still however relevant as the monitoring application must be run I order for the monitoring that the kernel performs is retrieved and displayed. The performance of the Monitoring Application is heavily dependent on the java virtual machine that is being run on the host system. On a sample system (650MHz Pentium III with 128Mb SDRAM running IBM JVM for Linux 1.3) the Monitoring Application never used more than 10 per cent of available CPU utilisation and mostly much less. This is perfectly acceptable.

6.5 User Adoption

At two points during this project code was released for public consumption and a notice was made on the linux-usb-developers and linux-usb-users mailing lists. The code was released as under the name USBMon. These releases were named versions 0.1 and 0.2 respectively. Version 0.1 only contained a kernel patch that did not support full data monitoring. Simple command line tools were supplied with version 0.1 to control and view the monitoring.

The USBMon system (as well as this document) can be downloaded from http://www.dcs.ed.ac.uk/~dxh/public/USB/. Over thirty-five downloads have been recorded since version 0.2 was released.

6.6 Future Development

Possible future development of this system could take many possible routes depending on the usage of the system and implementation decisions made in the main Linux USB tree. The most obvious next step is to develop the kernel patch into a configuration option in the Kernel. This could even prepare the patch for possible submission into the Linux kernel. Before any submission to the Linux kernel the absolute stability under a much wider range of systems would be necessary. A lot more reviewing of kernel code by experienced kernel developers would also be desirable.

The amount of monitored data values monitored by the kernel could also be increased. The interface to user-space could be made more flexibly defined so as to allow greater selectivity of monitored events by the monitoring application, for example only monitoring URBs that returned with error codes.

A possible addition to the kernel patch is the provision of generalised statistics for usage and activity over the whole USB. This could take the form of a file in the /proc filesystem that gave a few global statistics much in the same way as CPU usage is reported. This could then be used by simple desktop applets to signal activity on the USB bus.

On the monitoring application side there are many possible data analysis mechanisms that could be implemented such as graphs and charts as well as the calculation of generalised statistics on a device basis.

Appendix D - The USB Model Java Classes

Class Bus

java.lang.Object

  +--Bus

public class Bus

extends java.lang.Object

Class used to store information about a Bus.

Field Summary
`int`	`alloc` The amount of bandwidth allocated to this Bus
`int`	`max_bandwidth` The total amount of bandwidth possible on this Bus
`int`	`num_intrpts` The number of interrupt requests
`int`	`num_iso` The number of isochronous requests
`int`	`number` The number of the Bus.
`Device`	`root_hub` The root hub device.

Constructor Summary
`Bus(int bus_no)`

Method Summary
`Device`	`search(int num)` Search for a particular device and return it.
`java.lang.String`	`toString(int indent)` This returns a textual representation of the Device and its children.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Class Device

java.lang.Object

  +--Device

All Implemented Interfaces:

javax.swing.tree.TreeModel

public class Device

extends java.lang.Object

implements javax.swing.tree.TreeModel

Class used to store information about device.

Field Summary
`int`	`bus_num` The number bus that this device is on.
`java.util.Vector`	`cfg_list` The list of configurations.
`java.util.Vector`	`children` The list of devices connected to this one.
`int`	`cls` The Class number.
`java.lang.String`	`cls_string` The Class as a string.
`javax.swing.JComponent`	`configs_pane` This holds the Configurations Pane that is plays the tables of interfaces and Endpoints.
`Endpoint`	`Control_Pipe` The Primary Control Pipe that all USB devices must have.
`javax.swing.JComponent`	`details_pane` This holds the Details Pane that holds the lists of URBs by endpoints.
`int`	`driver` The Driver that is associated with this device (or 'none').
`int`	`level` The level within the bus.
`int`	`max_monitoring_level`
`Mfilereader`	`mfilereader` This is the mfilereader for this device.
`int`	`mxps` The size of packets from endpoint 0.
`int`	`num` The Device Number.
`int`	`num_configs` The number of different configurations.
`int`	`num_endpoints` Number of endpointd ( INTERNAL )
`int`	`num_ports` The number of ports (0 for hub).
`int`	`parent` The device number of the parent
`int`	`parent_port` The port number of the parent that this device is connected to.
`int`	`product` The Manufaturer's device number.
`java.lang.String`	`product_string` The Manufaturer's device name.
`int`	`protocol` The Protocol.
`int`	`revision_maj` The revision number of the hardware.
`int`	`revision_min` The revision number of the hardware.
`Device`	`root_hub` The root_hub device
`java.lang.String`	`serial_number` The Manufaturer's serial number.
`java.lang.String`	`speed` The Speed of the device in Mbps.
`int`	`subclass` The Sub-class number.
`javax.swing.JTabbedPane`	`tabbed_pane` This holds the tabbed pane for this device.
`java.util.Vector`	`URB_list` The list of URBs monitored on this device
`int`	`vendor` The Manufacturer number.
`java.lang.String`	`vendor_string` The Manufacturer as a string.
`int`	`version_maj` The version of USB that this device claims to support (before point).
`int`	`version_min` The version of USB that this device claims to support (after point).

Constructor Summary
`Device()`

Method Summary
`void`	`addTreeModelListener(javax.swing.event.TreeModelListener l)` addTreeModelListener not fully implemented.
`java.lang.String`	`details()` This returns a full textual representation of the Device
`java.lang.Object`	`getChild(java.lang.Object parent, int index)` This returns the child of the particular.
`int`	`getChildCount(java.lang.Object parent)` This method gets the number of children for a parent.
`int`	`getIndexOfChild(java.lang.Object parent, java.lang.Object child)` This method returns the index of the child in parent's children.
`java.lang.Object`	`getRoot()` This returns the Bus object on which this device sits.
`boolean`	`isLeaf(java.lang.Object dev)` This method returns whether the object is a leaf or not.
`void`	`make_configs_pane()` This makes the configs pane.
`void`	`make_details_pane()` This makes the details pane.
`void`	`make_tabbed_pane()` This makes the details pane.
`void`	`removeTreeModelListener(javax.swing.event.TreeModelListener l)` removeTreeModelListener not fully implemented.
`Device`	`search(int num)` The search method searches for devices with a particular device number.
`java.lang.String`	`toString()` This returns a brief textual representation of the Device
`java.lang.String`	`toString(int indent)` This returns a textual representation of the Device and its children.
`void`	`valueForPathChanged(javax.swing.tree.TreePath path, java.lang.Object NewValue)` valueForPathChanged not fully implemented.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Class Configuration

java.lang.Object

  +--Configuration

public class Configuration

extends java.lang.Object

Class used to store information about a configuration of a device

See Also:

Device

Field Summary
`boolean`	`active` true if this is the active configuration of the device.
`int`	`attr` The attributes of this configuration.
`java.util.Vector`	`if_list`
`int`	`num` The number of this configuration.
`int`	`num_ifs` The number of different interfaces that this configuration has.

Constructor Summary
`Configuration()`

Method Summary
`boolean`	`bus_powered()` bool that is derived from the attr integer.
`boolean`	`self_powered()` bool that is derived from the attr integer.
`java.lang.String`	`toString()`
`boolean`	`wakeup_capable()` bool that is derived from the attr integer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Class Interface

java.lang.Object

  +--javax.swing.table.AbstractTableModel

        +--Interface

All Implemented Interfaces:

java.io.Serializable, javax.swing.table.TableModel

public class Interface

extends javax.swing.table.AbstractTableModel

Class used to store information about an interface (part of a cfg of a device.

Field Summary
`int`	`alt` The alternate that is being described.
`int`	`alt_cls` The alternate's class as an int.
`java.lang.String`	`alt_cls_string` The alternate's class as a String.
`java.lang.String`	`alt_driver` The alternate's driver as an int.
`int`	`alt_protocol` The alternate's protocol as an int.
`int`	`alt_subclass` The alternate's subclass as an int.
`java.util.Vector`	`endpoint_list` List of the endpoints for this interface.
`int`	`num_eps` The number of endpoints in this interface.
`int`	`number` The interface number.

Fields inherited from class javax.swing.table.AbstractTableModel

listenerList

Constructor Summary
`Interface()` Constructor...

Method Summary
`int`	`getColumnCount()` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.
`java.lang.String`	`getColumnName(int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.
`int`	`getRowCount()` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.
`java.lang.Object`	`getValueAt(int row, int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.
`boolean`	`isCellEditable(int row, int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.
`void`	`setValueAt(java.lang.Object value, int row, int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Configurations Page for a device.

Methods inherited from class javax.swing.table.AbstractTableModel

addTableModelListener, findColumn, fireTableCellUpdated, fireTableChanged, fireTableDataChanged, fireTableRowsDeleted, fireTableRowsInserted, fireTableRowsUpdated, fireTableStructureChanged, getColumnClass, getListeners, removeTableModelListener

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Class Endpoint

java.lang.Object

  +--javax.swing.table.AbstractTableModel

        +--Endpoint

All Implemented Interfaces:

java.io.Serializable, javax.swing.table.TableModel

public class Endpoint

extends javax.swing.table.AbstractTableModel

Class used to store information about device.

See Also:

Serialized Form

Field Summary
`int`	`address` The address of the endpoint.
`int`	`attr` The type of transfer as an int.
`java.lang.String`	`attr_string` The type of transfer as an String.
`boolean`	`input` The endpoint is an input.
`int`	`interval` The interval in ms between the polling of interrupt endpoints.
`int`	`max_packet_size` The maximum packet size of this endpoint.
`int`	`monitoring_level` The level at which this endpoint is currently monitored.
`int`	`number` The endpoint number
`boolean`	`output` The endpoint is an input.
`java.util.Vector`	`URB_list` The list of URBs monitored on this device

Fields inherited from class javax.swing.table.AbstractTableModel

listenerList

Constructor Summary
`Endpoint(Device d)`

Method Summary
`int`	`getColumnCount()` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.
`java.lang.String`	`getColumnName(int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Deatils Page for a device.
`int`	`getRowCount()` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.
`java.lang.Object`	`getValueAt(int row, int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.
`boolean`	`isCellEditable(int row, int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.
`void`	`set_monitoring_level(int val)`
`void`	`setValueAt(java.lang.Object value, int row, int col)` In Order to Extend AbstractTableModel so that the table of endpoints can be shown on the Details Page for a device.

Methods inherited from class javax.swing.table.AbstractTableModel

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Class URB

java.lang.Object

  +--URB

public class URB

extends java.lang.Object

Class used to store details of a URB that occurred in a Linux USB subsytem. The exact configuration of this URB object relates to the attributes that this monitoring application needs to maintain. This is not a definition of what a URB contains.

Field Summary
`int`	`actual_length` This is the actual length of the transmitted URB
`Configuration`	`cfg` Device Configuration - at time of load.
`byte[]`	`data` The Vector containing the data
`boolean`	`data_present` True if data is present in this URB
`Device`	`dev` The Device that this URB was sent/recieved on
`int`	`devnum` The actual device number as passed in ***M
`Endpoint`	`ep` The Endpoint that this URB was sent/recieved on
`int`	`epnum` The actual Endpoint number as passed in ***M
`int`	`error_count` Error Count - the number of transmission errors.
`int`	`hour` the hour that this URB was completed
`long`	`millisecond` the millisecond that this URB was completed
`int`	`minute` the minute that this URB was completed
`int`	`pipe` The Pipe information: This contains the direction and type of pipe information.
`int`	`second` the second that this URB was completed
`boolean`	`showing_data` True if data is present in this URB
`int`	`size` The size of the transmitted buffer.
`int`	`status` Status:
`int`	`transfer_flags` transfer flags
`javax.swing.JFrame`	`window` A JFrame placeholder to store the Frame showing the contents of the URB.

Constructor Summary
`URB(java.lang.String str, byte[] rec_data, Bus bus)` The constructor that creates the URB structures from the line in the ***M file in the USB Monitoring part of the /proc file system.

Method Summary
`java.lang.String`	`toString()`

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

[1] Manufacturers of these systems include Intel, Compaq, CATC, and Catalyst Enterprises

[2] A minor update to the 1.0 specification released primarily to clarify issues relating to hardware device design.

[3] This is derived from a maximum of 500mA current from a supply that can vary 4.75V – 5.25V as specified in [2] p.134-139

[4] This situation was not necessarily intended. It is primarily a result of Intel refusing to licence the UHCI design very widely prompting the development of OHCI by competitors. Intel has now agreed to licence the USB 2.0 EHCI design freely [33].

[5] Apple Computers Inc usually refers this to.

[7] According to Apple Corp. over 125 peripherals are available at this time, in the year 2000 over 12 million IEE 1394 devices were shipped.

[8] DOS is the Disk Operating System format common on floppy disks. EXT2 is the most common native Linux hard drive filesystem.

[9] USB support was started in the previous 2.2 kernel (starting at 2.2.7). However development soon moved to the 2.3 experimental kernel tree. USB usage on 2.2 kernels was possible via the use of a kernel back-port patch that some distributions used to allow basic USB mouse and Keyboard use. USB usage beyond this was possible but not easy.

[10] This is the validation for the accuracy of the timestamp taken at time of monitoring.

[11] Valid values for return status are detailed in [15].