Friday, September 01, 2006

Linux kernel linked List Implementation

Linked list is a very important data structure which allows large number of storage with efficiant manipulation on data.Most of the kernel code has been written with help of this data structure.So most of the kernel code (data structure) complexity depends on the implementation of that data structure. You could write your own code but the lack of bad design could give you bad performance. A good operating system should give not only stability but also good performance. So i was eager to know how linux implements linked list. Is it circular singly linked list or doubly linked list or circular doubly linked list ?
does It has some libray file where most of the functions would have been declared? Yes my search ends at
"/usr/src/linux/include/linux/list.h".
I could see all types of fuctions are declared here. Before discuss all these function i should tell somthing about how linux kernel adopts strategy to build efficiant implementation.
It has a unique style to implement the traversing,adding,deleting node into a circular link list.As it is circular , you don't need think about head node and last node.You just simple pick up a node and follow the pointers untill you get back the same original.This approch removes the extra head node implementation.So each routine simple needs a pointer to a single element in the list and it could be any element. You may be wondered to see the implementation.Normally we used to declared linked list as like
struct my_list{
int data,
struct my_list *prev;
struct my_list *next;
};
But if want to adopt linux implementation style then you could write like that
struct my_list{
struct list_head list; //linux kernel list implementation//
int data;
};
struct list_head { //Declared in list.h file
struct list_head *next;
struct list_head *prev;
}

Now note the name of the structure "list_node". Linux developers has taken this name in mind from the fact that there is no head node in the list. All node could be acted as a head node .The next pointer points the next element and prev pointer will point the previous pointer.Thanks the linux claver list implementation.You could forget about firat node and last node concept.Just just compare list as a big cycle with no first and last.There are many clever implementation of each function. Please go through the list.h.It is well documented.I thnik i couldn't make better documentation than this. I am sure , you can digest it within one hour.
Now some big questions may come:-)

1.) why should i learn linux kernel linked list implementation strategy though i could write my own code or i can take linked list library from somewhere?

2) How can i use linux linked list library in my user space application? What would be the benefit that i could get if i use this library?
Yes , I am trying to give answer of these two question. If you are not satisfied with my answer . please leave a comment...
Answer No 1:
1) If you want to see yourself as a future linux kernel hacker,you must learn this strategy. I can give a prof from a book "linux Kernel Development" by renowned kernel hacker Robert Love.See the Apendix-A
"All new User must use the exiting interfase,we are serious about this, do not reinvent the wheel"

2) Create various type of Data Structure: You can build any data structure as in your mind.
3) Portability: Otherwise it would not have been placed into main linux kerenl source tree.
4) East to learn. readibility: Yes, It is easy to learn, you can learn all function with one hour rigorious study.
5) Complexity: You would be wonderd that all these functions are O(1), means they execute in costant time regardless of the size of the list.

Answer: Q.No.2:
With some minor modification, You could include this list.h header file into your userspace application. It's a single header file. very handy! You need to do
a)Remove #ifdef __KERNE__ and its #endif
b)Remove all #include line
c)Remove rcu related functions(many of them there)

Yes , I am working on this list to publist my own list.h header file. I have already implemeted generic implementation of my own circular doubly linked list and have written same program with help of "list.h" file and compare

You could also ............

P.S: The kernel uses linked list to store the task list, Each process's task_struct is an element in the linked list.
look at this link...You will automatically understand how many times list_add function have been called:-)

Resource: Section
Yes i used to read from good article.Try to understand from the good article.And ofcouse i should share with you.Have a look
Ref 1: "Linux Kernel Development"-Robert Love Appendix -A
Ref 2: Linux kernel linked list for user space
Ref 3: Linux Kernel Linked List Explained
Happy Hacking...

Thursday, August 31, 2006

[off-topic]Hope One Day I Would Be..

I am alwayes very much keen to do my Ph.D in Operating System( It could be Real Time Opearating System also). I don't know when i will get chance to fulfill my dream. But i should do..i must do.. so now i have to prepare myself. But How? While I was thinking , i thouht first i have to find out what should i need? ...i could prepare a list like ..i need extensive knowledge , i have to do hard work, i have to read countless books or I have to ........................................................But first i could guess that , i should know how to do research ...while i was searching , i got a fantastic link which i should share with you...Have a look on this link
Advice on Reserch and Writing
You could visit this page. It's one of my research interest area...
CS Columbia University's Professor Erez Zadok Page

Tuesday, August 29, 2006

How to install GNU/Linux without any removal media

If you want to install GNU/Linux withoutany removal media(CD, USB, Floppy) then please cheack out
this given link.You need to just copy the (small) Linux installer on a hard disk and run it from your previous operating system. Finish using the network. That's all. In some cases you can substitute the network by a (big) hard disk if your network is not good..

LINK

Monday, August 28, 2006

Intrusion Detection System

We were searching for an efficient intrusion detection system. .We wanted to develop some differnt kind of system which can perform as a detection and prevention system.After a long time discussion and searching in the net, we finalized ythis approch..

In our system , We have two module. One from userspace called detection system.Another from kernel space called prevention system.System acts like

When packets come from the network to NIC, we capture the packets and send to the userspace program.
Still we are thinking to avoid IP stack for passing each packets to user space.Any way , a user space detection program checks the each and every packets and try to match with predefined rule. If this match happens then detection system calculates the rate of packets whichs falls under this rules. If it overtakes the thresold value then this packet address would be written into a configuration file and immediately send to the kernel module which is ruuning in kernel space. So next time if this type of packets is comming into the NIC, before passing to the network stack, our kernel module will drop theses packets. This procedure will occour recuresively and best way to name this project .....

" Knowledge based intrusion detection and prevention system"
While searching , i got very good metirial from net which i should mention here for further look up:

linux 2.4 Packet Filtering
Writing Network device Driver
Inside the Linux Packet Filter Part 1
Inside the Linux Packet Filter Part II

i will add more......

Friday, August 11, 2006

[off topic]Planning to go pondichery

Today , i decided to go pondichey as a weekend trip with anubhav and his friends.Last one year i am staying chennai but didn't go anywhere. Though i am not feeling well but i don't want to miss this trip.Next year,i would be a married person.So i would not get a chance to enjoy with friends.

Ok. Hope my trip will give me best enjoyment. Oh i didn't tell my parent that i am going. i thought i will tell when i will reach pondichery.

Wait for next trip blog....

bye bye

Monday, August 07, 2006

Kernel Hacking Free Course 2006

If you are interested in installing and using a Unix system based on the Linux kernel, you won't have any problem in finding all the documentation you need. In fact, there exist several excellent books that teach how to install Linux and how to run it. Furthermore, many courses are taught in many countries on the same topics.

If your problem is how to modify the Linux kernel and how to evaluate the impact of the changes you made (we call this kind of programmer's behavior kernel hacking) then you won't find as much documentation as in the previous case. In fact, you can find almost anything on the net, except that is scattered around many places and not all the documents you will find are outstanding, while many are obsolete.

So, the ambitious goal of this Kernel hacking course is to describe in a coherent way all you should know to become a reasonable kernel hacker[Written by famous author-Daniel P. Bovet and Marco Cesati, the authors of the popular O'Reilly's book "Understanding the Linux kernel"]

Have a look on this link.

Happy Hacking...

Monday, July 17, 2006

A small tips for C/C++ programming with Vi Editor

I am using vi editor to write my C program. I found some good feature with Vi editor which will help programmer to write C program clean and readble. Also reduce time to debug the C/C++ source file.

Start C style Indent in Vi Editor
:set cindent

ctags
just make a command " ctags *" on your source directory. Ctags command will create one file called tags which contains defined
macros defined by #define
enumerated values
function definitions, prototypes, and declarations
class, enum, struct, and union names
namespaces
typedefs
variables (definitions and declarations)
class, struct, and union members

Now open vi editor with source file name and put
: tag [i.e if i have a function fun() then it sould be :tag fun or just pressed ,it will list down all the tag name and hit enter . It willl jump where fun() has been defined. Just put ctrl+T to come back.

Syntax highlight:

:syntax on

Auto make

If you prepare makefile for your program then vi will allow you to run your program within vi. Just type
:make
[for more details :h make]

Thursday, July 13, 2006

Device Driver Kit for Linux

Do you want to write Device Driver under linux? What's the easyest way to do that...I have searched couple of time in net to get any device driver kit[ ofcource free] where everything sould be configured and preloaded. You need not to worry about any setup to code for your new device driver. I got one like windriver from Jungo but it is propitary.I don't have money to pay. So i stopped when Gregkh released first LDDK with 2.6.16.18 kernel and full LDD3 book.
You could visit his home page.
If you want to get full iso image then visit this site

Thursday, June 22, 2006

Create your own Loopback Device

A loopback device enables you to simulate a block device using an ordinary disk file.Data can be written or read from a file name disk-image rather than to and from the tracks and sector of an actual physical disk drive or disk partition.(ofcourcd disk-image must reside on an actual disk,which must be larger than the simulated disk). Loopback devices are named like /dev/loop0, /dev/loop1, and so on.Each can be used as a single block device at one time. Be sure when you are stting up loopback device you should be logged as a super user.

This loopback device can be used as any other block device(like your hard disk). You can create file system and mount this files system as you would mount the file system on a normal disk or partition.Such type of file system is basically called virtual file system.
Construct the virtual File System and Mount it with loopback device(Follow this step)

1.Create any 10MB file using dd command to hold the virtual file system. You can create this file anywhere as you like but i like to create in /tmp.
[suman@linux tmp]#dd if=/dev/zero of=/tmp/disk-image count=20480
your output should be like

20480+0 records in
20480+0 records out
10485760 bytes (10 MB) copied, 0.218746 seconds, 47.9 MB/s
2.cheack with ls -l command
[suman@linux tmp]# ls -l /tmp/disk-image
-rw-rw-r-- 1 suman suman 10485760 Jun 22 15:43 /tmp/disk-image
3.Now create a file System (i prefered ext2 File System..U can choose as you like) by this command
suman@ajit program]$ mke2fs -q /tmp/disk-image
It will ask for proceed just put y
/tmp/disk-image is not a block special device.
Proceed anyway? (y,n) y
4.Mount the file system using loopback device. Before that you need to create one mount point for the virtual file system.
[suman@linux tmp]# mkdir /tmp/virtual_fs
[suman@linux tmp]# mount -o loop=/dev/loop0 /tmp/disk-image /tmp/virtual_fs

5.If you don't see any error, you have successfully mounted the file System . Now cheack it through this command
[suman@linux tmp]#df -h /tmp/virtual_fs
you would see
Filesystem Size Used Avail Use% Mounted on
/tmp/disk-image 9.7M 92K 9.1M 1% /tmp/virtual_fs
6. Create one test file to test whether it is working correctly or not
[suman@linux tmp]# cd /tmp/virtual_fs
[suman@linux tmp]#echo 'I have done' >test.txt
[suman@linux tmp]#ls -l
lost+found test.txt
[suman@linux tmp]#cat test.txt
I have done
......lost+found is a direcxtory that was automatically added by mke2fs.

If want to unmount the file system just put this command
[suman@linux tmp]# umount virtual_fs

Why should i need loopback device? This question may arriase..let me tell one situation and explain me how we can use loopback device..

You have one CDROM image and you are using frequently. You need to just copy the whole CDROM image into new loopback device...you can do like that
[suman@linux]$ cp /dev/cdrom /tmp/disk-image
or
[suman@linux]$dd if=/dev/cdrom of=/tmp/disk-image

This may take several time depends on the size of your CD ROM. The resulting image file could be as large as the contents of the CDROM.
Now you can mount the CDROM image with out having the original CDROM in your CDROM.For example
[suman@linux]# mount -o loop=/dev/loop0 /tmp/disk-image /mnt/cdrom

You can access much faster than CDROM beacause whole CDROM content is in Hard Disk. Note most CDROM use the file System type iso9660.

You are playing a game which always needs your CDROM. You could not play without CDROM and sometimes you are seeing that your game is going to slow as you expected. Well create a loopback device and copy the CROM contents and mount it......and Enjoy...You are not accessing CDROM phisically which could save your CDROM life.
If you have any comments any question and also if you found any error about this mini tutorial please mail me and please leave your suggestion in comment section that i can improve later. I always apologies my poor english expression. forgive me..

Wednesday, June 21, 2006

kernel Status

Tuesday, June 20, 2006

Linux kernel Develoment

Now a days i am little bit busy. I am learning how to write device driver under linux(maily char and USB). i have spend lots of time to search good meterial (for beginner) from internet.I started with LKMPG 2.6.x from tldp.org. Next I should read LDD3 by rubini(reading..)I listed some online metirial which i found during my study. Hope you could save some time to finding these metirial. I have given all the downloadable link. You just have to click the link and save it. Second tutorial is a link for entire book. You could get chapterwise pdf file from this site.

Tutorial No. 1
The Linux Kernel Module Programming Guide
Tutorial No. 2
Linux Device Driver by Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman(2.6.x kernel) Tutorial No.3
Writing Character Device Driver for Linux
Tutorial No. 4
Programming Guide for Linux USB Device Drivers

Those who are realy interested in kernel development and device driver here is great resource list.

Kernel Resource List

Happy Hacking!

Sunday, June 18, 2006

Running FreeBSD inside Fedora Core 5: QEMU A processor Emulator


Processor emulator is one kind of software which will allow one operating system to run in another operating system.We don't need to reboot the system and write boot sector in MBR.

It is very much usefull for testing cross platform application. Say you want to build one application which should run GNU/Linux,Micro$oft,FreeBSD and MAC OS. Then probabily you would need four different system or one system with four operating System.But every timr you have to reboot your system and test your application. Virtualization(i.e Processor Emulator) will help you to run four operating system in one platform. Just you have to switch one OS to onather by clicking your mouse.
There are many such type of application,both are open source and propitary. Here I have given some list , If you know another please update it.

Open Source:Xen,Bochs,QEMU,PeerPC etc
Propitary:Vmware,Micro$oft Virual PC etc

I have successfully installed freeBSD under my Fedora core 5 through QEMU.(Just for showing how it works) Here i am writing very small HOW-TO for Fedora code 5.
(As it is small , I am going to install from RPM not from source)
Step 1: First Get RPM fro FC5(Link)
Step 2: go to root by typing su - and your password
[suman@ajit ~]$ su -
Password:
Step 3: Install RPM
[suman@ajit qemu]# rpm -ivh qemu-0.8.1-1.fc5.i386.rpm
Step 4:put this command
[suman@ajit ~]$ qemu-img create linux.img 2G
after successfull creation of linux.img file it should shows this message
"Formating 'linux.img', fmt=raw, size=2097152 kB"

Step 5: If you download iso image into your home directory then give this command
[suman@ajit ~]$ qemu -boot d -cdrom ~/6.1-RELEASE-i386-bootonly.iso -hda linux.img
Or
If you have CDROM then give this command
[suman@ajit ~]$ qemu -boot d -cdrom /dev/cdrom -hda linux.img


Congratulation! You will see that a freeBSD is booting from a separate window from your present Operating System..

Step 6:Relax and enjoy multiple Operating System from your system...

In this tutorial I didn't include any comment regarding any command. It is your task to read and understand .I am hereby giving some link from internet which will help you to understand..best of Luck!!
link1
Link2
Let me know if you have any comments regarding this mini tutorial..

Friday, June 09, 2006

GDB: A GNU Debugger

I recently wanted to learn GDB(Briefly). I searched in google with keyword " GDB tutorial"and google gave me this peter page. If you are writing program in linux it would be helpful to you. Peter started this tutorial with memory layout of a program and discussed gdb command with example. Any one who wants to know gdb could start from this tutorial. Anyway peter has also other good stuf . You can visit his site.

More inforation about GDB:
Official GDB User Manual
Norman Matloff tutorial on GDBPrinceton University GDB tutorial
RMS's gdb debugger tutorial
uchicago University tutorial




Wednesday, May 31, 2006

two handful commands

Do you want to make shortcut in your command ?Then run this command into your command line
command format: alias [shoutcut-command]=[command]
example:
>alias=c=clear
Do you want to revoke this alias? Run this command in to your coomand line
>unalias c

How to open CDROM with command line?
Run this comand
>eject /dev/cdrom or > eject /mnt/cdrom or just run > eject .Bash shell will take CDROM device as a default device.

how to close the CDROM Device with command line?
> eject -t /dev/cdrom or simply put >eject -t

Wednesday, May 24, 2006

Know about web syndication

RSS is a family of web feed formats, specified in XML and used for Web syndication. RSS is used by (among other things) news websites, weblogs and podcasting. The abbreviation is variously used to refer to the following standards:
Really Simple Syndication (RSS 2.0)
Rich Site Summary (RSS 0.91, RSS 1.0)
RDF Site Summary (RSS 0.9 and 1.0)

Web feeds provide web content or summaries of web content together with links to the full versions of the content, and other metadata. RSS, in particular, delivers this information as an XML file called an RSS feed, webfeed, RSS stream, or RSS channel. In addition to facilitating syndication, web feeds allow a website's frequent readers to track updates on the site using an aggregator.

(PS: Content has been taken from wikipedia.).

I am using suse 10 with kde 3.5. I used "akregator" to get the feeds from site.

visit Redhat site for more information.

Further reading

Friday, May 19, 2006

how to write a system tray application?


The system tray is a cross-desktop method of notifying the user. It will be very usefull for user. Suppose if you have mailclient[Kmail] and if it is in system tray then whenever new mail will come, it will pop up a new windows with notification of new mail. But question is how to write these kind of application? Well i was searching internet for document. Yesterday i got one document from redhat site . It is in perl language.Search the document
Procedure:
1.Your system should have perl and gtk 2.0.
2. We need two library called
a)gnome-python2-libegg-2.10.0-2.1
b)gnome-python2-extras-2.10.0-6
Just Download this two rpm and installl with --nodeps option like
suman@>rpm -ivh gnome-python2-extras-2.10.0-6.i386.rpm gnome-python2-libegg-2.10.0-2.1.i386.rpm --nodeps
Open a vi editor with file name test.py
#! /usr/bin/python
import gtk
import egg.trayicon
t = egg.trayicon.TrayIcon("MyFirstTrayIcon")
t.add(gtk.Label("Hello"))
t.show_all()
gtk.main()
Copy this code to the file. And run as suman> python test.py
Thats's all !!! A "Test" system tray icon will be shown in your desktop system tray.

After long time back

Oh! I am writing my blog after 13 days. I went to attend my sister marraige. I couldn't get time to write. Well after comming back to my office, I started coustomizing the Open Source ERP product called Compiere . I successfully made some small changes with modify the source code.

Open Source News:
Okippi is the successor to Blue Security's Blue forge project..Now it is started...

Friday, May 05, 2006

Open Source Shooping cart application

If you are looking for good open source shopping cart software,visit Interchange. It worked so well that some company began an enterprise-wide migration to open source software that has Linux running everywhere, from the servers to the desktops.

Top 50 Open Source Application

Now we can do all our work with open source application. Sometimes i have seen that people don't find or don't know which open source applications are available to do certain job. Here is a list of 50 top open source application by devid uhlman which we can use for beginner to enterprise level.This is a pdf file. I am giving here linkwhere you can download. This pdf also contains the regarding application site name where you can get the software.Have a look ,Have a fun!

http://www.uversainc.com/download/top50.pdf

Thursday, May 04, 2006

OCFS2 project from Oracle

OCFS2 is the next generation of the Oracle Cluster File System for Linux. It is an extent based, POSIX compliant file system. Unlike the previous release (OCFS), OCFS2 is a general-purpose file system that can be used for shared Oracle home installations making management of Oracle Real Application Cluster (RAC) installations even easier. Among the new features and benefits are:

  • Node and architecture local files using Context Dependent Symbolic Links (CDSL)
  • Network based pluggable DLM
  • Improved journaling / node recovery using the Linux Kernel "JBD" subsystem
  • Improved performance of meta-data operations (space allocation, locking, etc).
  • Improved data caching / locking (for files such as oracle binaries, libraries, etc)
sources: http://oss.oracle.com/projects/ocfs2/

OCFS2 will only work on the 2.6 series of the Linux kernel. As a result, OCFS2 packages are only available for 2.6 based distributions.
SuSE Linux Exterprise Server 9:Bundeled,need configuration
Red Hat Enterprise Linux 4:not bundeled ,need to download and configure
Ubuntu:Bundeled,need configuration

User Guide