Creating your own Linux Shell

¡Hi there! This blog will address programming a brand new Linux Shell, as well as some basics linux concepts.

However, before starting coding…

The first step to build a shell is understanding the concept of Linux Kernel and what a Shell does.

The kernel of an operative system is its central part, which performs key operations for any computer.

Among other things, the Linux Kernel handles:

• Process Scheduling: a CPU core can only handle 1 process at the time, so the Kernel has to schedule, decide which process will use the CPU processing power, and store in RAM memory information of the processes scheduled.
• Memory management: memory is limited and has to be shared along processes efficiently.
• Provides a file system on disk, allowing to create, update, and delete files.
• Creating and termination of processes.
• Access to hardware devices.
• Permission management.

More relevant information to keep in mind

1.

“A system call is a controlled entry point into the kernel, allowing a process to request that the kernel perform some action on the process’s behalf. The kernel makes a range of services accessible to programs via the system call application programming interface (API). These services include, for example, creating a new process, performing I/O, and creating a pipe for interprocess communication.” Kerrisk, Michael, 2010, The Linux Programming Interface, Pag 46.

2. Process are primitive units for the allocation of system resources. They have their own memory space, process identifier (ID), and a ‘parent’ process.

Process run programs; There can be multiple processes running the same program, but each process has its own copy of the program within its own address space and runs it independently of the other copies.

Additionally, they can create ‘child’ processes. by using fork.

3. When any program starts, there are three streams of data that are initialized: stdin, stdout and stderr. These are called ‘Standard Streams’. These streams are declared in stdio.h.

• The standard input stream is the normal source of input for the program. STDIN_FILENO
• The standard output stream is used for normal output from the program.
  STDOUT_FILENO
• The standard error stream is used for error messages and diagnostics issued by the program. STDERR_FILENO

These represent a buffered streams in the file, and contain a file descriptor. We’ll work with these in our Shell.

4. The linux Kernel has some programs related to the core of the operative system called built-ins. They’re stored in /usr/bin. These programs will always available in RAM so that accessing them is bit fast when compared to external commands which are stored on hard disk.

Also, folders like /usr/bin, /usr/local/bin, /usr/local/sbin, and /usr/sbin store executable programs in our system. All of their addresses are store in an environment variable called $PATH. We can check them by using the ‘echo $PATH’ command.

5. In Unix based systems, environment variables are a set of names values, stored within the system. They are used in programs launched in the shell (built-ins). They allow you to customize how the system works and the behavior of the applications on the system.

That was a lot ¿wasn’t it?

Please take a couple of minutes to process all this information before moving on.

…

Let’s talk about the Shell now

We can start guessing what a shell is ¿Can’t we?

A shell is a program working as the middle man between the Kernel and the user.

It’s a command line interpreter which receives inputs from the user and passes them onto the Kernel.

There are a lot of different Linux Shell types, and all of them have the same functionality (except for specific features in each version).

Also, there are 2 ways executing our Shell: interactive and non-interactive. That is, commands can be typed directly to the running shell (interactive) or can be put into a file and the file can be executed directly by the shell (non-interactive).

Non-interactive Shell

The program detects if the program is sent through the built in echo with pipes | to execute it. The shell separates the two modes by utilizing isatty() function. isatty() returns 1 if fd is an open file descriptor referring to
a terminal; otherwise 0 is returned and this mode is activated.

Interactive Shell

The interactive mode allows us to input and execute commands for an infinite amount of times as long as the shell is running and it’s not ‘exit’ or ‘EOF’ (Ctrl + d).

Our interactive Shell will have an infinite loop, where we’ll print the Shell prompt symbol ($) with the write system call.

There our program will scan for inputs (stdin) using the getline function.

We’ll scan in case there are empty spaces. However, if we do get a valid input we’ll separate (tokenize) each word of the input string with the strtok function.

This function will set certain inputs (spaces, \n, \t) replace them with NULLs, and jump from NULL to NULL. Now we’ll have an array of strings, where each word is a position in the array. This way it’ll recognize the different commands with their respective flags.

Now we’ll compare the position [0][0] of the array with a ‘/’. If it’s true, we’ll check with stat if it is an executable command. Either way it’ll execute the command or print error and go back to the first instance of the program. Please check the flow diagram above for a clear understanding of the program.

If the position [0][0] of the array is not ‘/’ we’ll check if it’s a built-in (using the function execute_builtins), and will execute the built-in in case it exist.

If it’s not a built-in program, we’ll search $PATH(using the function get_path), tokenize $PATH (tok_path) and will get to a loop that concatenates each position of the $PATH with the command line arguments in position 0. This ay we’ll check if the input is an executable (check_executable).

If it’s executable, our shell will execute the command using the function execute. In case it’s not executable, the shell should print error and start our program again.

There’s also the possibility that the user send our program a command like: ‘$/bin/ls’. In this case the program will follow the process described above. When it reaches stat, will check if the input is valid, then ask if it’s a built-in keep

You can see the program flow in the chart above.

Let’s examine the flow of a our program in a practical way…

ls -l *.c

This shell reads ‘ls -l *.c’ from the command line and verifies if it contains a path . This is not the case, the path is obtained from environ and tokenized so it can be concatenated with the first argument until the file is found.

Then it executes by creating a child process, the child makes a system call with execve, it’s is in charge of executing the command with their respective arguments.

We can finally see our Shell's output: it lists in long format all files ending in .c. (due to the *.c wildcard).

.

Personal observations of the coding process.

This has been a very changeling project for us to code. However, our team composed by Marcela, Felipe, Andres, Maria Fernanda and I could program a couple of different version of the Shell, fix compilation errors and memory leaks.

I’d recommend to anyone attempting to build their own Shell to work with a small team, use Valgrind to solve memory leaks, and take into the account both the interactive and non-interactive modes since the very inception of the program.

.

Bibliography

Kerrisk, Michael, 2010, The Linux Programming Interface.
Shotts, William, 2019, The Linux Command Line
Wikipedia, Kernel (computer science)
GNU.org, Manuals
GNU.org, The GNU C Library Reference Manual