Driver porting: low-level memory allocation

Allocation flags

The GFP_BUFFER allocation flag is gone (it was actually removed in 2.4.6). That will bother few people, since almost nobody used it. There are two new flags which have replaced it: GFP_NOIO and GFP_NOFS. The GFP_NOIO flag allows sleeping, but no I/O operations will be started to help satisfy the request. GFP_NOFS is a bit less restrictive; some I/O operations can be started (writing to a swap area, for example), but no filesystem operations will be performed.

For reference, here is the full set of allocation flags, from the most restrictive to the least::

• GFP_ATOMIC: a high-priority allocation which will not sleep; this is the flag to use in interrupt handlers and other non-blocking situations.

• GFP_NOIO: blocking is possible, but no I/O will be performed.

• GFP_NOFS: no filesystem operations will be performed.

• GFP_KERNEL: a regular, blocking allocation.

• GFP_USER: a blocking allocation for user-space pages.

• GFP_HIGHUSER: for allocating user-space pages where high memory may be used.

The __GFP_DMA and __GFP_HIGHMEM flags still exist and may be added to the above to direct an allocation to a particular memory zone. In addition, 2.5.69 added some new modifiers:

• __GFP_REPEAT This flag tells the page allocater to "try harder," repeating failed allocation attempts if need be. Allocations can still fail, but failure should be less likely.

• __GFP_NOFAIL Try even harder; allocations with this flag must not fail. Needless to say, such an allocation could take a long time to satisfy.

• __GFP_NORETRY Failed allocations should not be retried; instead, a failure status will be returned to the caller immediately.

The __GFP_NOFAIL flag is sure to be tempting to programmers who would rather not code failure paths, but that temptation should be resisted most of the time. Only allocations which truly cannot be allowed to fail should use this flag.

Page-level allocation

    struct page *alloc_pages_node(int node_id, 
                                  unsigned int gfp_mask, 
				  unsigned int order);

The memory allocator now distinguishes between "hot" and "cold" pages. A hot page is one that is likely to be represented in the processor's cache; cold pages, instead, must be fetched from RAM. In general, it is preferable to use hot pages whenever possible, since they are already cached. Even if the page is to be overwritten immediately (usually the case with memory allocations, after all), hot pages are better - overwriting them will not push some other, perhaps useful, data from the cache. So alloc_pages() and friends will return hot pages when they are available.

On occasion, however, a cold page is preferable. In particular, pages which will be overwritten via a DMA read from a device might as well be cold, since their cache data will be invalidated anyway. In this sort of situation, the __GFP_COLD flag should be passed into the allocation.

Of course, this whole scheme depends on the memory allocator knowing which pages are likely to be hot. Normally, order-zero allocations (i.e. single pages) are assumed to be hot. If you know the state of a page you are freeing, you can tell the allocator explicitly with one of the following:

    void free_hot_page(struct page *page);
    void free_cold_page(struct page *page);

These functions only work with order-zero allocations; the hot/cold status of larger blocks is not tracked.

Memory pools

To work with mempools, your code should include <linux/mempool.h>. A mempool is created with mempool_create():

    mempool_t *mempool_create(int min_nr, 
                              mempool_alloc_t *alloc_fn,
    			      mempool_free_t *free_fn,
			      void *pool_data);

    typedef void *(mempool_alloc_t)(int gfp_mask, void *pool_data);
    typedef void (mempool_free_t)(void *element, void *pool_data);

The allocation function should take care not to sleep unless __GFP_WAIT is set in the given gfp_mask. In all of the above cases, pool_data is a private pointer that may be used by the allocation and deallocation functions.

Creators of mempools will often want to use the slab allocator to do the actual object allocation and deallocation. To do that, create the slab, pass it in to mempool_create() as the pool_data value, and give mempool_alloc_slab and mempool_free_slab as the allocation and deallocation functions.

A mempool may be returned to the system by passing it to mempool_destroy(). You must have returned all items to the pool before destroying it, or the mempool code will get upset and oops the system.

Allocating and freeing objects from the mempool is done with:

    void *mempool_alloc(mempool_t *pool, int gfp_mask);
    void mempool_free(void *element, mempool_t *pool);

mempool_alloc() will first call the pool's allocation function to satisfy the request; the pre-allocated pool will only be used if the allocation function fails. The allocation may sleep if the given gfp_mask allows it; it can also fail if memory is tight and the preallocated pool has been exhausted.

Finally, a pool can be resized, if necessary, with:

    int mempool_resize(mempool_t *pool, int new_min_nr, int gfp_mask);

This function will change the size of the pre-allocated pool, using the given gfp_mask to allocate more memory if need be. Note that, as of 2.5.60, mempool_resize() is disabled in the source, since nobody is actually using it.