LWN.net Logo

 


 
Summary page
Return to the Kernel page
 
Recent Features

LWN.net Weekly Edition for March 18, 2004

LWN.net Weekly Edition for March 11, 2004

The annotated SCO stock price chart

A grumpy editor's calendar search

LWN.net Weekly Edition for March 4, 2004

Printable page
 

 

Driver porting: compiling external modules

This article is part of the LWN Porting Drivers to 2.6 series.
The 2.5 development series saw extensive changes to the kernel build mechanism and the complete replacement of the module loading code. One result of these changes is that compiling loadable modules has gotten a bit more complicated. In the 2.4 days, a makefile for an external module could be put together in just about any old way; typically a form like the following was used:

    KERNELDIR = /usr/src/linux
    CFLAGS = -D__KERNEL__ -DMODULE -I$(KERNELDIR)/include -O

    all: module.o

Real-world makefiles, of course, tended to be a bit more complicated, but the job of creating a loadable module was handled in a single, simple compilation step. All you really needed was a handy set of kernel headers to compile against.

With the 2.6 kernel, you still need those headers. You also, however, need a configured kernel source tree and a set of makefile rules describing how modules are built. There's a few reasons for this:

  • The new module loader needs to have some extra symbols defined at compilation time. Among other things, it needs to have the KBUILD_BASENAME and KBUILD_MODNAME symbols defined.

  • All loadable modules now need to go through a linking step - even those which are built from a single source file. The link brings in init/vermagic.o from the kernel source tree; this object creates a special section in the loadable module describing the environment in which it was built. It includes the compiler version used, whether the kernel was built for SMP, whether kernel preemption is enabled, the architecture which was compiled for, and, of course, the kernel version. A difference in any of these parameters can render a module incompatible with a given running kernel; rather than fail in mysterious ways, the new module loader opts to detect these compatibilities and refuse to load the module.

    As of this writing (2.5.59), the "vermagic" scheme is fallible in that it assumes a match between the kernel's vermagic.o file and the way the module is being built. That will normally be the case, but people who change compiler versions or perform some sort of compilation trickery could get burned.

  • The new symbol versioning scheme ("modversions") requires a separate post-compile processing step and yet another linkable object to hold the symbol checksums.

One could certainly, with some effort, write a new, standalone makefile which would handle the above issues. But that solution, along with being a pain, is also brittle; as soon as the module build process changes again, the makefile will break. Eventually that process will stabilize, but, for a while, further changes are almost guaranteed.

So, now that you are convinced that you want to use the kernel build system for external modules, how is that to be done? The first step is to learn how kernel makefiles work in general; makefiles.txt from a recent kernel's Documentation/kbuild directory is recommended reading. The makefile magic needed for a simple kernel module is minimal, however. In fact, for a single-file module, a single-line makefile will suffice:

	obj-m := module.o
(where module is replaced with the actual name of the resulting module, of course). The kernel build system, on seeing that declaration, will compile module.o from module.c, link it with vermagic.o, and leave the result in module.ko, which can then be loaded into the kernel.

A multi-file module is almost as easy:

	obj-m := module.o
	module-objs := file1.o file2.o 
In this case, file1.c and file2.c will be compiled, then linked into module.ko.

Of course, all this assumes that you can get the kernel build system to read and deal with your makefile. The magic command to make that happen is something like the following:

    make -C /path/to/source SUBDIRS=$PWD modules
Where /path/to/source is the path to the source directory for the (configured and built) target kernel. This command causes make to head over to the kernel source to find the top-level makefile; it then moves back to the original directory to build the module of interest.

Of course, typing that command could get tiresome after a while. A trick posted by Gerd Knorr can make things a little easier, though. By looking for a symbol defined by the kernel build process, a makefile can determine whether it has been read directly, or by way of the kernel build system. So the following will build a module against the source for the currently running kernel:

    ifneq ($(KERNELRELEASE),)
    obj-m	:= module.o

    else
    KDIR	:= /lib/modules/$(shell uname -r)/build
    PWD		:= $(shell pwd)

    default:
	$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
    endif

Now a simple "make" will suffice. The makefile will be read twice; the first time it will simply invoke the kernel build system, while the actual work will get done in the second pass. A makefile written in this way is simple, and it should be robust with regard to kernel build changes.


Post a comment

  Driver porting: compiling external modules
(Posted Feb 6, 2003 6:05 UTC (Thu) by rfunk) (Post reply)

For the make trickery in the latter part of this article, it's important to note that the original make command assumes the kernel source is in /usr/src/linux/:
make -C /usr/src/linux SUBDIRS=$PWD modules
While the makefile that allows a shorter command line assumes the kernel source is in a directory like /lib/modules/2.5.59/build/, due to the following line:
KDIR := /lib/modules/$(shell uname -r)/build

I believe there was some minor controversy on the linux-kernel mailing list recently over which location is more appropriate.

  Path to the kernel source
(Posted Feb 6, 2003 14:44 UTC (Thu) by corbet) (Post reply)

Actually, /usr/src/linux is pretty much deprecated by the Prime Penguin himself; you're supposed to keep your kernel sources somewhere else. I got lazy and used it in the example, mostly because it's shorter to type than the /lib/modules path. The latter is the better way to go, however, especially in scripts or makefiles - it "automatically" points to the right source tree, unless you move your trees around.

  Driver porting: compiling external modules
(Posted Feb 6, 2003 11:03 UTC (Thu) by raarts) (Post reply)

Thanks for this article.
This is why I subscribe to LWN.

Ron Arts

  What about cross-compilation?
(Posted Feb 6, 2003 18:35 UTC (Thu) by sjmadsen) (Post reply)

The Makefile trickery at the end isn't going to work in cross-compiler enviornments. My company is building a product that uses Linux as the embedded OS, but builds typically take place on Solaris.

Even if we were building on Linux, it's unlikely that the OS on the build machine is going to match the embedded environment.

  What about cross-compilation?
(Posted Feb 6, 2003 21:02 UTC (Thu) by Peter) (Post reply)

The Makefile trickery at the end isn't going to work in cross-compiler enviornments. My company is building a product that uses Linux as the embedded OS, but builds typically take place on Solaris.

True enough. In fact, there is no automated way for the computer to read your mind and know, for a specific module build, where the matching kernel tree resides. If you don't give the computer any more information, one reasonable guess is "wherever the currently-running kernel was built, assuming it was built on this machine". In fact, I can't think of a better default guess. But if this turns out to be wrong - for various reasons, including cross-compilation - you are going to have to specify the source location.

Putting that location in the Makefile as a special macro like KERNELDIR makes it possible to override on the command line: 'make KERNELDIR=...'. The other option, of course, is to work to get your module integrated into the official kernel tree, at which time you are rid of this headache once and for all.

  What about cross-compilation?
(Posted Feb 7, 2003 15:50 UTC (Fri) by dwmw2) (Post reply)

It works perfectly for cross-compilation for me. If you override CROSS_COMPILE (or have it set in your kernel's top-level Makefile) that works just as it always did:@
make KDIR=/local/arm-kernel CROSS_COMPILE=arm-linux-gnu-

Note also that the article is somewhat misleading -- the Makefile fragment

> KERNELDIR = /usr/src/linux
> CFLAGS = -D__KERNEL__ -DMODULE -I$(KERNELDIR)/include -O
>
> all: module.o

... was _always_ broken and nonportable -- building using the kernel makefiles was the only way to get it working portably since about the 2.0 kernel. It doesn't kernel+arch-specific CFLAGS like -mregparm= -mno-gp-opt -mno-implicit-fp right.

  Hardware vendor supplied modules
(Posted Feb 6, 2003 21:02 UTC (Thu) by pradu) (Post reply)

I see a problem with hardware (and software) vendors modules that are supplied in source code (see for instance VMWare kernel modules). If you can't compile a kernel module without the kernel source tree and on a different machine (maybe with different compiler/processor and whatnot) from the distibutor kernel, you can't use such modules.

Or I am missing something?

  Hardware vendor supplied modules
(Posted Feb 8, 2003 0:28 UTC (Sat) by Peter) (Post reply)

I see a problem with hardware (and software) vendors modules that are supplied in source code (see for instance VMWare kernel modules). If you can't compile a kernel module without the kernel source tree and on a different machine (maybe with different compiler/processor and whatnot) from the distibutor kernel, you can't use such modules.

There are a few possibilities here:

  • completely disallow external modules - then, VMWare and co. would have to provide a patch to integrate their modules with the kernel source proper, and each customer would have to apply this patch and rebuild the kernel. (Or, alternatively, VMWare could ship with a custom kernel, but then they'd have to follow the GPL and open-source their module.)
  • have VMWare and their ilk each track all the major vendor kernels and release modules to go with each one. Many proprietary module vendors do this today. It's a horrible hack but it does work for some people.
  • freeze the kernel/module interface, somehow nullify or abstract away all differences that affect it, and maintain this situation for a given length of time (say, a stable series). That would include gcc 2.95 vs. gcc 3.2, preemptible vs. non-preemptible, UP vs. SMP, 386 vs. 586 vs. K7, highmem vs. non-highmem, and a few other details, on the x86 platform. This is the Microsoft / commercial Unix solution, and the Linux people refuse to bother with it. If you think this is the way to go, please be prepared to present your patch which does all this without affecting efficiency or maintainability of the source base. Alternately, Google for 'UDI unix linux' and see what happened the last time this was proposed.
  • take some sort of snapshot of the state of the build environment when you build a kernel, and make this snapshot available somehow for when you want to build an external module. This snapshot would consist of kernel headers, .config file, exact compiler and its flags, and some sort of glue to plug the external module in to all this.

At present, we have the latter option, in the form of a live source tree. Note that if you can locate the live source tree you wish to build against, and the compiler you used to build the kernel, and any other variables you passed in when you built the kernel (like a CROSS_COMPILE variable), you can use this method just fine.

If you don't know this information, unfortunately there is no way for the computer to divine it for you. What if I build 10 different kernels on my box, destined for 10 different machines, then I try to build an external module? How is the system supposed to know which of my 10 kernels the module is for? Barring a direct DWIM neural interface (for which Linux drivers are still sadly lacking), I'm gonna have to point my module at the proper kernel tree, and re-set-up any custom variables I set the first time.

  Hardware vendor supplied modules
(Posted Feb 13, 2003 22:53 UTC (Thu) by mmarq) (Post reply)

>freeze the kernel/module interface, somehaw nullify or abstract away all differences...

IN OTHER WORDS, BUILD A IN KERNEL API/ABI (like in LSB)!!!

THE PROBLEM WHIT "UDI" WAS NOT THE IDEA IN ITSELF, BUT "HOW AND WHO" WAS IN CONTROL...IN THE BEGINING LINUS WAS A PROMOTER OF IT...

YOU ALREADY HAVE A "ABSTRACT INTERFACE" IN KERNEL IN THE FORM OF I2O...THE PROBLEM IS THAT YOU NEED "SPECIAL HARDWARE" TO GET ALONG WITH IT...

The current "state of the art" is, as specified in your explanation, there is no simple answer to any question... be it cross-compilation, be it building external modules, or whatever related to hardware. There's almost 100% certainty now that you run against "gcc 2.95 vs. gcc 3.2, preemptible vs. non-preemptible, UP vs. SMP, 386 vs. 586 vs. K7, highmem vs. non-highmem"....

BUILDING GOOD AND COMPLETE WIDE SUPPORTE FOR A PIECE OF HARDWARE IN EXTERNAL MODULES, "IS FOR SURE NOW" ONLY FOR A SKILLED AND WELL DOCUMENTED "COMERCIAL" TEAM...(how about that????)

If you have to have a good and extensive support of hardware in the form of in kernel loadable modules, it would mean that the source of the kernel could easely be 10 times of current size, cutting out those that dont have broadband( more than half of the world), or...

...LINUS GETS FIRED FROM TRANSMETA OR DIES OF EXAUSTION!...OR FOR STABLE KERNEL 2.8/3.0 YOU HAVE A "DEVELOPMENT CICLE OF 4 YEARS",...OR WORST THAN EVERYTHING ELSE BEFORE!!!...YOU SEE THE KERNEL HIGHJACKED IN THE FORM OF NVIDEA-KERNEL, ATI-KERNEL, OR MORE COMPLET HP/COMPAQ-KERNEL, DELL_KERNEL, GATEWAY-KERNEL, LIVING IN THE DUST THE LESS COMPLIENT RH-KERNEL, MANDRAKE_KERNEL, SUSE_KERNEL, VMWARE_KERNEL!!!!...

IS THE LINUX KERNEL HEADING FOR A "ABISM", OR IS MY IMAGINATION?????????

  Hardware vendor supplied modules
(Posted Feb 8, 2003 15:12 UTC (Sat) by jschrod) (Post reply)

Is there any vendor who doesn't distribute source code and config/{autoconf,version}.h to its kernel? I always thought, on different machine, one installs said kernel source code, adds the config files, and there you go. At least, this worked for me all the time.

Cheers, Joachim

  Driver porting: compiling external modules
(Posted Feb 13, 2003 0:17 UTC (Thu) by pedretti) (Post reply)

This is probably a dumb question, but what is the module that you insmod? When I compile the hello_world I get a hello_world.o and a hello_world.ko. Both of these can be insmod'ed but the hello_world.o give an error "no version magic, tainting kernel" -- so I assume the hello_world.ko is the one I want to use?

  Driver porting: compiling external modules
(Posted Apr 30, 2003 20:19 UTC (Wed) by krobidea) (Post reply)

It seems that this 2.5 method of compiling external modules does not work on older kernel versions. I tried the example .c and Makefile, and got the following results:

- RedHat 7.3 out of the box (2.4.18-3 w/.config or any .o files). Compilation fails, thinking that module support is not compiled in.

- Version 2.4.20 I built and installed. Compilation failed, *** No rule to make target `modules'. This happens when make changed back to my working directory and did a make xxx modules. The kernel source base Makefile appears to be different in the 2.5.x versions.

So, will external modules require that the source be present, built and running? I don't believe any RedHat distributions will work.

What about the 2.4 or 2.2 target modules issue?

  Driver porting: compiling external modules
(Posted Feb 13, 2004 14:25 UTC (Fri) by grisu1976) (Post reply)

What about compiling a module that is composed of multiple files? Especially if you use global variables across all files using the extern storage specifiers referencing the global variable definition in one file? I do this and getting a "is COMMON symbol" warning for that variable and at module loading (the .ko file) a "Invalid module format" error. If i use the static storage specifier in all files (for the source definition and the extern references), module loading works fine, but the nm command shows that there are as many symbols as files...

Is there a compiler parameter to achieve global variables across multiple files of a module? It's strange, because in kernel 2.4 my module worked properly

  Driver porting: compiling external modules
(Posted Feb 19, 2004 20:13 UTC (Thu) by jdeas) (Post reply)

I am new to this.
I have a custom driver I want to port to 2.6
The interrupt and work queue info seems good but
I am having trouble with the make file.

Compiling using a standard make command and Makefile
does not understand (can not find) 'obj-m' . Is my basic way of compiling
now wrong?
obj-m:=Mydriver.o give the above error

  Driver porting: compiling external modules
(Posted Mar 9, 2004 18:04 UTC (Tue) by liangjz) (Post reply)

I'm trying to port a 2.4 driver to 2.6 and am having trouble getting the makefile to call nasm.

In the 2.4 driver, I simply had a build rule that said

assembly_stuff.o: assembly_stuff.asm
nasm $(NASM_FLAGS) assembly_stuff.asm


I tried doing the same thing with the 2.6 driver and added

my_driver-objs := assembly_stuff.o

The problem is that the makefile insists that it needs a assembly_stuff.s, presumably to send through GAS.

How can I get around this?

  problem when compiling external modules
(Posted Mar 10, 2004 21:53 UTC (Wed) by madmax1984) (Post reply)

i encountered the following problem when trying
to compile a hello.o type module for 2.6.1:
after the initial setbacks of having to figure
out about init_module, cleanup_module, SUBDIRS
thingie and all it still wouldn't work:
insmod hello.ko would return -1: invalid format;
insmod hello.o would work warning/error free
(really odd) but i would get a seg fault when
trying to rmmod hello.

after quite a bit of googleing i found the
solution: recompile the kernel with the running
config (the last kernel i had built was for my
gateway and used no modules, had no module
support - which might have been the problem -
although my running kernel had module support)

hope this is useful

Copyright (©) 2003, Eklektix, Inc.
Linux (®) is a registered trademark of Linus Torvalds
Powered by Rackspace Managed Hosting.