In this example I will use Ubuntu 18.04 LTS(AMD64) Desktop with Linux 5.0.0 Kernel and the 5.4.2 Kenel source file as an example to build a linux kernel with a customized system call, and then update the linux kernel of the Desktop. The road to a successful compilation and load can be arduous, so be careful, which should make the process much easier for you.
Video Tutorial
If you prefer a video tutorial, I’ve made one and you can go to YouTube for it.
Get the Kernel Source
You can download the kernel from the website The Linux Kernel Archives.
Or Simply
$ wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.4.2.tar.xz
After downloading the kernel, you should get a linux-x.x.tar.xz(in my case, linux-5.4). You should move the file to your home directory and then extract it.
$ mv linux-5.4.tar.xz ~/ $ tar -xvf linux-5.4.tar.xz
Now, a linux-5.4 will be in your home directory, to make the later process easier, we can create a soft link to the directory.
$ ln -s ~/linux-5.4 ~/linux
Install the Compile Tools
$ sudo apt-get install git build-essential kernel-package fakeroot libncurses5-dev libssl-dev ccache libncurses-dev bison flex gcc make git vim
In the later process, in case there is some tool missing, you can just run the apt-get install command. For example
With the gcc: not found error, you should run the following command.
$ sudo apt-get install gcc
While this time
You should run
$ sudo apt-get install flex
To note, flex and bison are 2 most easily forgotten packages.
Transfer the Current Kernel Configuration
Firstly, clean all the previous configurations of the kernel source code
$ cd linux $ make mrproper
To make it easier, we can transfer our current kernel configuration to the new kernel.
$ cp /boot/config-`uname -r` .config $ yes '' | make oldconfig
The above commands should make your life easier. No extra steps are required
Add Customized System Call
Now with all of this configured, let’s add some small feature to our kernel. This syscall is about the kernel ram page management and it will allow our kernel to print the number of page faults in the whole system, the number of page faults and dirty pages in the system log.
Add the System Call Number
I am using an x64 Ubuntu, so I will not be modifying the syscall_32.tbl in the same directory.
$ vim ~/linux/arch/x86/entry/syscalls/syscall_64.tbl
Then find an unused syscall number in the gap of the common/64 and x32 ABI’s. To know more about how to add a syscall number in should refer to [Ref 2].
In my case, I will use 428.
Since this is only a simple program that is not architecture-specific, I will just use a ‘common’ ABI.
Also add it to the file syscall_32.tbl
$ vim ~/linux/arch/x86/entry/syscalls/syscall_32.tbl
Then we can add it to the unistd.h file as a micro.
$ sudo vim ~/linux/include/uapi/asm-generic/unistd.h
In the vim editor, you can use the ?__NR_syscalls to search for the pattern. If there are many matches, you can simply hit Enter and then use ‘n’ to search up and ‘N'(Shift + ‘n’) to search down. When you are in the right position use ‘i’ to insert your syscall above this string and then modify the syscall number. For example, it was originally #define __NR_syscalls 436, I modified it to #define __NR_syscalls 437, and added my own syscall number right above it. When the modification is finished, hit ESC key to quit the insert mode. Then use ‘:x’ command to save the changes and exit.
Deal with the Page Faults Info Print
Now it’s time to insert our code to the kernel.
Add Global Page Counter
$ vim ~/linux/include/linux/mm.h
Use ‘:$’ vim command to go to the end of the file and insert the following c declaration right above the #endif directive. This will be our variable to count the global page faults.
extern unsigned long pfcount;
Then EXC, ‘:x’ to quit insert mode, save and exit.
Add Counter for task_struct
The task_struct structure is an abstraction for Linux processes. What we will do next is to add a page fault counter for the process structure.
$ vim ~/linux/include/linux/sched.h
Use ‘?task_struct’, hit enter and then use ‘N'(Shift + ‘n’) to search for the structure definition.
WRONG: Now declare our member variable at the beginning of the structure definition. Then save and exit. (Kidding, DO NOT DO THIS!!!) As you can see from the comment below my the #ifdef statement. thread_info should be declared before everything else in the struct.
CORRECT: So the correct thing to do is to scroll down and you can add it at the same place as me.
Initialize the Task Specific Page Fault Counter
Open the file fork.c
$ vim ~/linux/kernel/fork.c
And search for ‘static struct task_struct *dup_task_struct’
Now after the `tsk = alloc_task_struct_node(node)` statement and its judger that detects if tsk is NULL, insert
tsk->pf = 0;
It should be like the following
Increment the Counters Whenever Page Faults Occur
$ vim ~/linux/arch/x86/mm/fault.c
Search for the __do_page_fault function
Perform the forward declaration for the external variable pfcount, and then increment both pfcount and tsk->pf,
unsigned long pfcount; .... pfcount++; current->pf++;
Implement the System Call Function
$ vim ~/linux/kernel/sys.c
Append the following Syscall function to the end of the file, don’t miss the return 0.
SYSCALL_DEFINE0(mysyscall)
{
printk("Number of dirty pages per process: \n");
struct task_struct *t;
// Dirty pages of each process
for_each_process(t)
{
printk("%s [%d]: %lu\n", t->comm, t->pid, t->nr_dirtied);
}
printk("Number of page faults (total): %lu\n", pfcount);
printk("Number of page faults (current): %lu\n", current->pf);
return 0;
}
Don’t forget the declaration in ~/linux/include/linux/syscalls.h
$ vim ~/linux/include/linux/syscalls.h
Add the following line at almost the end of the file.
asmlinkage int sys_mysyscall(void);
And for compatibility,
$ vim kernel/sys_ni.c
Add the following to the file
COND_SYSCALL(mysyscall);
Compile the Kernel
Now it’s time to compile our kernel, simply
$ nohup make -j12 &
nohup will take the program running in the background and the output is kept as a nohup.out log file. The -j12 option means that I use 12 logical processors since I want to make full use of my 6-core 12-process CPU. To see the nohup.out log file
$ cd ~/linux $ cat nohup.out
You can have a look at it from time to time to make sure it is working correctly.
The log should be updating once in a while.
Install the Kernel
$ sudo make modules_install -j 12 $ sudo make install -j 12 $ sudo update-grub
Verify
To verify that our kernel module is working, you should firstly reboot the system.
$ sudo reboot
Write a Test Program
#define _GNU_SOURCE
#include <linux/unistd.h>
#include <sys/syscall.h>
#include <stdio.h>
#include <string.h>
#define __NR_mysyscall 428
int main()
{
syscall(__NR_mysyscall); /* or syscall(428) */
return 0;
}
Then compile the test program
$ gcc -o test test.c
Then run it
$ ./test
To view the output
$ dmesg
Recompile
In case your compile process fails, every time you retry, the recompile process can be like the following
$ cd ~/linux $ make mrproper $ cp /boot/config-$(uname -r) .config $ yes '' | make oldconfig $ nohup make -j12 & $ sudo make modules_install -j 12 $ sudo make install -j 12 $ sudo update-grub
Wiki: Linux Architecture
What is the difference between i386, x86, x86_64, amd64
- x86-64 is the 64bit version instruction set of x86
- x86 is the 32-bit instruction set for Intel CPUs
- i386 is a former version of x86 in the early ages of Intel
- AMD64 and Intel 64 are synonyms for x86-64
- IA-64 is a little-used instruction set, proposed by Intel, a disruptive 64-bit instruction set, while AMD64 is an extension to x86. History turns out that AMD64 wins the market.
But in Linux, the situation is a little bit different, x86 is denoted by i386, while x86-64 is denoted by x86. Since x86_64 is an extension to x86, so it is a general x86 architecture. In this way, it makes sense to use x86 for x86_64.
References
- How to Compile a Custom Linux Kernel on Unbuntu 19.04: https://medium.com/@simonconnah/how-to-compile-a-custom-linux-kernel-on-ubuntu-19-04-9b623ece5922
- How to add a new Syscall: https://www.kernel.org/doc/html/v5.1/process/adding-syscalls.html
- https://www.cyberciti.biz/faq/creating-soft-link-or-symbolic-link/
- https://www.maketecheasier.com/build-custom-kernel-ubuntu/
- https://www.kernel.org/
- ABI: https://stackoverflow.com/questions/49557623/parameters-of-files-syscall-32-tbl-syscall-64-tbl-in-build-linux-kernel
- More on ABI: https://stackoverflow.com/questions/19648470/what-does-the-name-parameter-mean-in-the-syscall-32-tbl-linux-file
- About x86:https://elixir.bootlin.com/linux/latest/source/arch/x86/Kconfig
- Merge i386 and x86_64 into x86: https://lwn.net/Articles/258127/
- x86-64 Wiki: https://en.wikipedia.org/wiki/X86-64
Furthermore, there is a doc provided in kernel version 3.17 for my OS course. Although not update to date, still inspirint