Saturday, November 1, 2008

[Software] Install VMware Tools on Fedora 9

I bought a new PC with Window XP operating system. Like what I did on my old computer, I decided to install VMware workstation on Windows XP since I also need Linux/Unix environment for software development. I downloaded Fedora 9 ISO image from redhat.com and used it to build the linux guest OS on the VMware workstation. Everything for Fedora 9 installation went smoothly. After that, I began to install VMware Tools as usual (you can find a guide from VMware):

1) Select "Install VMware tools" on VMware, it will mount VMware Tools to "/media/VMware Tools" of the guest Linux OS,
2) start a terminal, type in "su", enter the root password,
3) rpm -Uhv /media/VMware\ Tools/VMware*.rpm, install the VMware Tools using RPM (or you can use extract the gz file using tar). Note, there's a space between VMware and Tools, so you need a backslash,
4) after installation, you should have a script file installed: /usr/bin/vmware-config-tools.pl
5) run rmware-config-tools.pl, it will ask you several questions, such as "which compile command to use", "kernel headers directory" etc.

However, when it goes to "kernel headers directory", it complained that the default directory /usr/src/include/linux does not exist or the kernel headers are not matched to the kernel you are running. In order to solve this problem, start a new terminal and run "rpm -qal | grep kernel", it will tell you what kernels have been installed on your guest OS and where they are located (for my case, it's under /lib/modules/2.6.6.xxx/include).

After changing the directory, it still doesn't work and continues complaining "mismatch" messages. The reason is that I upgraded the kernel but didn't reboot it. So the running kernel and the header files installed are not "synchronized" yet. Restart the OS and re-run vmware-config-tools.pl, now it's ok.

The last step of vmware configure is to set the dispaly resolution. After that, you need to restart X as it suggests. Unfortunately, another problem emerged: the mouse position messed up. I clicked the menu but it's not selected. I googled this problem, and found it's a xorg-x11-drv-vmmouse driver bug for Fedora 9 only. A work around is given in this link:

Edit /etc/X11/xorg.conf in the guest machine. Create the section if necessary:

Section "ServerFlags"
Option "NoAutoAddDevices"
EndSection

Restart X, everything goes well! Now you can enjoy the features provided by VMware, such as copy&paste between host and guest OSs, share files and folders etc.

Monday, August 25, 2008

Embedded C programming - for beginners (1)

Here the beginner is me. :-) If you never touched embedded C programming before and want to learn it, join me. I have some experiences to share with you. We can teach and learn ourselves.

Embedded programming is always hardware related. You may at least need to buy a develop kit or emulator. The price is not that cheap (usually costs you several hundred bucks, with those bucks you can send roses to your GF everyday for two months). Can we save those bucks and still learn the fundamentals of embedded C programming? Congratualtion, you got it!

Thanks to Keil - An ARM company. If you never heared about Keil, congratulation again! you are the beginner and this article is right for you! Keil provides a great Evaluation Software supporting different paltforms but with very few limitations. It includes the assembler, compiler, linker, debugger, and IDE. Everything you need for embedded C learning is here. How to get it? Go to this webpage of Keil (http://www.keil.com/demo/) and you can get it free (you just need to fill out a simple form)!

Ok, next I'll show you how to write your first embedded C program.

Sunday, July 6, 2008

Run Javascript under Windows using Mozilla's interpreter - SpiderMonkey

Javascript is widely used in Web programming. Microsoft's Internet Explorer and Mozilla's Firefox both include its Javascript interpreter in the browsers. However, they also limit your capability of running Javascript in a web-only environment.

The good news is, Mozilla's interpreter was released as an open source. You can find it from http://www.mozilla.org/js/. It has two versions, one is written in C and is called SpiderMonkey. The other is written in Java and called Rhino.

Today, I installed SpiderMonkey on a windows xp computer. The installation is very simple:
1) Install Mozilla Developer Center - MozillaBuild
2) Go to SpiderMonkey's website to get the latestest version and build it under MozillaBuild.

After installation, you will get the executable interpreter js.exe in a directory whose name depends on the system you're building on (you can find this direcotry under mozilla/js/src/ with a suffix .OBJ).

If you are not accustomed to the MozillaBuild environment (unix-like) and is comfortable with Windows/Dos, you can modify the system environment by adding the path of js.exe into the PATH variable (right click "My computer" in Windows Explorer, and then select Advanced -> Environment Variables). Then under the DOS prompt environment (in Windows, Start -> Run -> cmd) , you can use "js file.js" to run your javascripts.

Tuesday, April 8, 2008

[Programming] Threads (4) - synchronization

Threads (4) - synchronization

Previously we described the issue of joinable and detached thread scheduling. In practice, the situation is not that simple. Most of the threaded programs are concurrent programs, which means the operating system will run multiple threads at the same time. However, there's no way to know when a thread will be scheduled to run by the system. The order of thread's running is not deterministic.

Another big problem of threaded programming is that, unlike processes, threads share the same address space. This is one of the powerful aspects of threads - if one thread changes the value of a variable, other threads will see this modification subsequently. However, this sharing could also be dangerous without careful design and implemenation.

The third thing is that debugging a threaded program is not easy as debugging a single-thread program - you may not reproduce the same result in a multi-threaded program.

Therefore, the synchronization among threads is a very important issue. In order to understand the details of synchronization, let's first learn some new concepts one by one: critical section, atomic operation, race condition, mutex, semaphore, deadlock.

Critical section: it means a resource (usually implemented by a sequence of code) that should be exclusively used by a program for a period of its execution. In simple words, if a thread begins executing a critical section code, it must continue to the end without being canceled.

Race condition: since a critical section can only be exclusively used, when multi threads occur concurrently, they will compete for the critical section. In this situation, we say there is a race condition.

In order to eliminate race conditions and ensure a critical section being implemented successfully, we need a way to make operations atomic, which is described below.

Atomic operation: when we speak of an atomy, we usually mean that it is a tiny particle indivisible. So here the atomic operation has the similar meaning: once the operation starts, it will not be canceled, pasued or interrupted until it completes; at the meantime, no other operation will take place either.In multi-threaded programming, the operating system will provide and guarantee some operations to be atomic.

(to be continued)

Wednesday, April 2, 2008

[Wireless] China 3G phone (TD-SCDMA)

In the area of 3G, China proposed and submitted its own standard (TD-SCDMA) to ITU-R in 1998 and got it approved as one of the 3G standardized technologies in the world. After that, China has been working on this standard for about 10 years. In order to support this self-own standard, China government postponed the 3G licence from year to year, and China Mobile Group (the world's largest mobile phone operator ranked by number of subscribers, with over 376.38 million customers by the end of Feb, 2008) was selected to build pre-commercial networks for TD-SCDMA system. Many of my previous colleagues in China participated in this project.

Yesterday (Apr 1, 2008), a video clip about China 3G/TD-SCDMA phone was released in sina.com (the largest Chinese-language infotainment web portal). In this clip, a video call, a piece of stream media and some java supported applications are demonstrated on a TD-SCDMA phone (the model is U980 of ZTE). The video call looks not bad. Although UMTS (WCDMA), cdma2000 EVDO, and even HSDPA have been widely deployed by some service providers in the United states, as I know no one supports video call yet.

The Olympics game is coming soon. Although China said TD-SCDMA might not make Olympics deadline, it's still very possible that TD-SCDMA phones will be offered on hand and showcased at that time.

Monday, March 31, 2008

[Programming] Threads (3) - joinable or detached thread

Last time we've seen it's very easy to create a new thread in Linux by using the POSIX thread API functino pthread_create(). But wait, that example code has a bug. How does Linux schedule those two threads, namely, the main thread and the new created thread?

The answer is Nothing! If the new created thread terminates before the main thread, everything is ok. But it's very possible that the main thread stops first, especially when you are creating a new thread to do some time-cost calculation or operation. In this case, the memory used to passing parameters to thread function will be deallocated when the main thread is done, while the created thread may still be accessing this block of memory.

So we need a mechanism to avoid this. The POSIX thread standard provides a function called pthread_joint(). It's similar to the wait() function that a parent process used to collect child processes. With pthread_join(), the main thread will wait until the new created thread terminates, then the main thread returns. In addition, the pthread_join() also provides a way for the main thread to read return values from the created thread.

The prototype of pthread_join() is
int pthread_join(pthread_t thread, void **thread_return);

pthread_join() looks good but it doesn't suit for all cases. Sometimes we may just want the main thread to call a new thread, but don't need the main thread to wait for the return of the new thread. For example, we create a second thread to print some documents. It's not necessary to let the main thread wait, it can continue to serve other jobs while the print is onging. In this situation, we can make the thread detached instead of joinable. In order to do this, we need to set the second argument in the pthread_create() function, i.e. thread attributes.

As we did in the previous blog, setting thread attributes to NULL means the default attribute (thread is joinable) is used. If we need detached thread, we can use pthread_attr_init() function to initialize it, then call pthread_attr_setdetachstate() to set the attribute as detached. Indeed, POSIX thread provides a set of functions to set different attributes to a thread, such as scheculed, inherited etc. But for most applications, the detached attribute is typically of interest.

A typical program setting thread attributes to detached is as follows:

{
...
pthread_attr_t threadAttr;
pthread_t thread;

pthread_attr_init(&threadAttr);
pthread_attr_setdetachstate(&threadAttr, PTHREAD_CREATE_DETACHED);
pthread_create(&thread, &threadAttr, thread_function, (void *)thread_arguments);
pthread_attr_destroy(&threadAttr);
...
}

Thursday, March 20, 2008

[Programming] Threads (2) - Thread Creation

In Linux/Unix, threads are implemented using POSIX standard API (known as pthreads). All functions and data types related to threads are declared in the header file .

To create a thread, use the pthread_create() function. The prototype of pthread_create() is
#include <pthread.h>
int pthread_create(pthread_t *thread, pthread_attr_t *attr, void *(*start_routine)(void *), void *arg);


The function is not hard to understand
1) the first argument is used to store the thread ID of the new created thread
2) the second argument is a pointer to a thread attribute object, you can use NULL as default or call pthread_attr_init() to initialize the attribue you want
3) the third argument is a pointer to the thread function, i.e., the function used for the new created thread. Don't be scared by the format, it's just an ordinary function pointer.
4) the last argument is used to pass parameters to the thread function, i.e. the thread function uses this passed value as its arguments

We'll use a simple example to show how it works. The code with comments is attached below



/** simple_thread.c */
#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>


void * thread_function(void * dumb);
int main()
{

// pthread_t is the thread type defined in POSIX thread
pthread_t thread_id;

int res;

/** create the new thread
the new thread ID is stored in thread_id
the thread's attribute is set to default(NULL)
it will call the thread_function()
nothing is passed to the thread_function
*/

res = pthread_create(&thread_id, NULL, &thread_function, NULL);
if (res!=0)
{

perror("Thread creation failed.");
exit(EXIT_FAILURE);
}

return 0;
}

void * thread_function(void * dumb)
{

printf("the new thread is created.\n");
return NULL;
}







Notice, since pthread functions are not included in the standard C library, you should add the pthread library yourself

cc -o simple_thread simple_thread.c -lpthread

Wednesday, March 12, 2008

[Wireless] New contender in Mobile TV - WiMAX

The latest news about WiMAX is that NextWave plans to introduce Mobile TV over WiMAX, which is currently named as MXtv. It will support up to 30 frames/second of video, and provides up to 45 mobile TV channels with 300kbps in 10 MHz. Nextwave will develop this MXtv with its partner Huawei Technologies USA and Alcatel-Lucnet. The news said they will demonstrate the technology at CTIA Wireless 2008 next month in Las Vegas.

In recent years, IP TV gains a lot of attention. In the wireline domain, some technologies have been proposed to provide the "Final Mile" data stream transmission for the consumer's premises, such as Passive Optical Network, Ethernet in the First Mile over copper. In the wireless domain, Mobile TV can be supported by two-way cellular network (such as 3G) or one-way dedicated broadcast network (such as satellite). Now WiMAX becomes a new platform for Mobile TV. No matter which one dominates or some of them share the market, the new Multimedia era is running to use.

[Programming] Threads (1)

Another important concept in multi-tasking operating system is threads. We have learned that a program can fork a child process to do a different thing. However, at times the overhead of creating a new child process is heavy, especially for some embedded systems which only have limited resources for use. The fact behind this is that a new copy of the original process will be created when forking, such as the memory space and file descriptors. That's what we mentioned before that child process is schedueled independently of the parent process.

In order to realize multi-tasking with light overhead, some new concepts are introduced to multi-tasking programming, such as light-weight process and threads. Sometimes people just call threads as light-weight processes for the sake of simplicity, although they are not the same thing (for details, you can refer to Light-weight process in wikipedia).

So in the view of the resource usage, the new created thread does not copy the original system resources used by the creating thread. It shares the same memory space (heap), global variables, file descriptors, signal handlers and other system resources as the original. However, threads have their own stack space.

If we take a look at the thread in the view of implementation, a thread exists within a process, or more precisely, a thread is a sequence of control within a process. Indeed, a process has at least one thread of execution (also called primary or main thread). This primary thread can create additional threads, and all these threads run the same program in the same process, but they may execute a different part of the program at any time.

Tuesday, March 11, 2008

[Programming] Processes (3) - Signals

A Signal is a special message sent to a process to respond to some conditions. Some examples of the conditions are: illegal instructions, broken pipe, termination etc. Those signals are defined in the header file - singal.h. They are all with prefix SIG. The following table is a list of some signals.

Signal NameDescription
SIGILLIllegal instruction, indicates the program's stack is corrupted
SIGINTTerminal interrupt from keyboard by pressing Ctrl+C
SIGKILLEnds a process immediately, which cannot be caught or ignored
SIGPIPEAttempts to write on a broken pipe
SIGQUITQuit from keyboard
SIGTERMRequests a process to terminate


For a full list of signals, you can find it by typing the command below:

% man 7 signal

Generally speaking, signals can be used in the next three situations:
1) send a signal to processes in response to some errors, such as SIGBUS(bus error), SIGSEGV(segmentation violation)
2) send a signal to another process, such as SIGTERM(terminate), SIGKILL(kill the process)
3) send a command to a running program using user-defined signals, such as SIGUSR1

When a process receives a signal, it will take some actions immediately - either to call another function or ignore it (ignore is also a kind of action). When it calls another function, the current executing program will be paused and the new function will be executed. When the function returns, the paused program will resume again.

You can aslo set or modify the actions to be taken on receipt of signals by using function sigaction().

[Programming] Processes (2) - Zombie processes

Last time we mentioned that we can use fork() to create child processes. However, you should keep this in mind: the child process is schedulued independently of the parent process. It implies that you cannot predict which process will execute before or after another, namely, the order of the occurrence of multi processes cannot be forecast. This situation also occurs in multi-threaded programming.

Therefore, some advanced, even sophisticated mechanisms must be used in mutlitasking programming to control the programs running in the desired order.

For example, in some cases you may want the parent process to wait until its child processes have completed. This can be done using the wait family functions, such as wait(), waitpid(), wait3() etc.

However, we need to pay attention to the occurrence of "Zombie" processes. When a child process terminates, its association (such as exit status) with its parent does not vanish immediately, it is still in the system. In this case, this terminated child process becomes a "Zombie" process. It is the responsibility of the parent process to clean up the zombie child processes. The wait() function we mentioned above can do this clean up work.

If you have too many zombie processes in the system, you may suffer slower performance since they consume resources. So you need to avoid zombie processes. Fortunately, when the parent process exits, a special process called init() with process ID of 1 will do the clean up work automatically for you.

You may ask why we don't notify the parent process when a child terminates. Great, you make it! In Linux, child process can send signals to parent when it terminates. We'll touch it in the next post.

Sunday, March 9, 2008

[Programming] Type casting problem in function's argument

Recently someone asked a question of C programming on a discussion board: he has some codes which handle a fixed point array by passing the name of the array to a function, for example:


/** Code 1 */
// declaration of the function
void f(int *pt);

// an array of fixed point integer
int A[10];

// call the function
f(A);




The above codes work successfully. Now he tries to extend the code to a floating point array, as follows:

double B[10];
f((int*)B);

The compiler didn't complain any error, but the output result is something weird. For example, run the following codes on gcc or visual c, you suppose to get output as 3, but the result is -266631570,



/** Code 2 */
#include <stdio.h>
void test_fun(int* px);
int main()
{
double a[10];

a[0] = 3.14159;
test_fun((int*)a);
return 0;
}

void test_fun(int *px)
{
printf("%d\n", *px);
}







We all know that we can type-cast a double variable to an integer variable like this:




/** Code 3 */
double d=3.14;
int c;
c=(int)d;




And we can pass an array to a function by the array's name if they are in the same type (as the above code 1). But why cannot we do the similar thing for passing a floating point array's name to a funciton which accepts fixed point variables?

The answer to this question is: floating point and fixed point numbers are stored in different formats. For floating point number, it conforms with IEEE754. For example, for a 32-bit computer, a single-precision binary floating point number is stored in the format of sign(1 bit) + exponent (8 bits) + fraction (23 bits). So if you simply pass the array's name of floating point numbers as function's argument to an array that accommodates fixed point numbers, the function will read the 32-bit floating point number in fixed point format. The compiler doesn't complain because, yes, you can read it. But the result cannot be what you expect.

The solution to this question may be: 1) cast each element of the floating point array to a fixed point array first, then call the function; 2) re-write the function to support floating point, such as using templates...

Friday, February 29, 2008

[Wireless] Sprint nosedives! Then, WiMAX???

Sprint-Nextel lost more than 20 percent of their stocks value after it annnounced fourth-quarter earning (or loss?) yesterday. The shares of its WiMAX partner - Clearwire, also dropped more than 10 percent in two days.

Now in market, most wireless firms are betting on 3G wireless networks. Even the new handset player, Apple, is going to launch a 3G-capable iPhone this year. Although 3G system was not deployed as fast as it was expected initially, it finally occupies a part of the wireless market and becomes bigger in years. Although WiMAX is newly designed with more advanced technologies, it may have a disadvantage than 3G in time-to-market.

So, who will drive wireless to the MAX? LTE or WiMAX? I bet nobody can answer this. What we know is: Sprint plans to launch commerical WiMAX service in Q2 2008. Ericsson plans to invest 1.5 billion dollars into next generation wireless systems (LTE or 4G) in three years.

But I'll say, no matter it's LTE, WiMAX or 4G, the winner is always OFDM+MIMO. :-)

Tuesday, February 26, 2008

[Wireless] Free global Wi-Fi network?

Today I read an article "Is a free global Wi-Fi network possible?". It said a company called WeFi is hoping to provide a virtual global wi-fi network which is free to allow every to log on.

Actually, WeFi is not the first one working on this. You can find free Wi-Fi hotspots from some websites, such as gWifi.net, free-hotspot.com etc. The basic idea of this kind of free wi-fi network is to find unsecured hotspots to access in. However, the distinction of WeFi is that it provides a software that can automatically connect your device to the best available hot spot.So it saves time of switiching among different hotspots if there are some.

Nevertheless, the ratio of unsecured (free) access spots to all spots in a typical area is not as high as WeFi said to be 50 percent. By checking the hotspots map provieded by WeFi, I found this ratio is about 20 percent on average. We know current 802.11 products all support WEP or WPA and the vendor encourage people to set up a log-in password for data protection. Only some businesses (such as Panera Bread‎) may have open wi-fi spots. For individual wi-fi hotspot owner, unless he is generosity and can bear the low data rate as well as the risk of privacy intrusion, he may make his wi-fi open and free. My two cents: just finding and utilizing the existing open wi-fi resources may not make this virtual free wi-fi network long-lived. As a wi-fi user, I definitely hope the Internet access be free. But, I also know another phrase "There is no such thing as a free lunch."

Friday, February 22, 2008

[Programming] Processes (1)

From today, we will digress to processes and threads programming.

One standard definition of process (IEEE Std 1003.1) is, "an address space with one or more threads executing within that address space, and the required system resources for those threads". It's like a circular defintion, because in order to understand this definition, you have to know what are threads first. This may not be a problem for those computer gurus, but for beginners, it's better to find a simple explanation, as follows.

The description of process in wikipedia is easy to understand: an instance of a computer program that is being sequentially executed. Further to say, a computer program is just a passive collection of instructions, while a process is the actual execution of those instructions. This is described from the view of the user. On the other hand, in the perspective of operating system, you can say process is the operating system's basic scheduling unit.

So what elements does a process consist of? It's easy to answer, since process is actaully a program. It contains: program code, data, variables, current context (execution status), working directory, access rights (mode and ownership), memory and other system resources allocated to the process etc. For Linux system, processes may share code and system libraries.

In Linux/Unix, you can
a) start a new process by the system() function;
b) replace the current process with a new process by the set of exec() funtions (execl, execv, execlp etc.);
c) or duplicate a new process by the fork() function.

In practice, system() is rarely used to start new processes, because it invokes the new program using a shell, which may cause security problems. So it's preferable to use exec() and fork(). The difference between exec() and fork() is: fork() creates a new child process which is an exact copy of its parent process, but exec() causes a process to cease being an instance of one program and to instead become an instance of another problem.

The flows of the system(), exec() and fork() call are shown in the following figures.


Thursday, February 21, 2008

[Programming] Sockets programming - multiple clients - prologue

Prologue

Previously we only talked about a very simple case: the server processes one client's request before accepting a connection to the next client. If the processing time for each client is long, and if multiple clients are coming, then the new clients have to wait a long time to be served. Obviously, this is not a good design for the case of multiple concurrent clients.

Several solutions are used in practice, such as:
1) Multi-process method (forked server processes)
2) Multi-thread method (threaded server processes)
3) Select
4) Poll

The first multi-process method calls fork() to create a child process. The main server blocks on accept(), waiting for a new connection. Once a new client comes in, the server calls fork(), the new child process will handel the new connection and the main server is able to serve other new incoming requests.

However, this method suffers the disadvantage that sharing information becomes more complex (you need to use message queues, semaphores etc), and it requires more CPU resource to start and manage a new process for each request.

The second multi-threaded solution offers the lightweight advantages of the first multi-process method. However, multi-threaded program is not easy to debug.

The third and fourth approaches, select() or poll(), offer a different method to block execution of the server until a new event occurs. They usually only uese one process to serve multiple client's request. The server will not need shared memory or semaphore for different clients to communicate. For example. select() works by blocing until something happens.

Oh, hold on, please. You interrupted me, because you already got lost by those new terminologies, such as process, thread, shared memory, semaphore. Yes, in order to understand those, we need to first learn somthing about process/thread. We will cover this in the next several posts.

Tuesday, February 19, 2008

[Programming] Sockets programming on Linux/Unix

Last time we introduced the fundamental of sockets programming and gave an example (WinSock) on the Windows platform.
In deed, socket interface was first introduced in UNIX. In UNIX, socket interface is actually an extension of the concept of a pipe.
However, sockets are created and used differently from pipes because they make a clear distinction between client and server, and they can implement multiple clients attached to a single server.

Again, let's briefly go over the principles of sockets programming:
1) First, a server application creates a socket using the system call socket.
2) Next, ther server process gives the socket a name, using the system call bind.
3) Then the server process waits for a client to connect to the named socket using the system call listen, which creats a queue for incoming connections.
4) For incoming connections, the server can accept them using the system call accept.
5) When the server calls accept, a new socket is created that is distinct from the named socket. This new socket is used only for communication with this particular client. The named socket remains for further connections from other clients.
6) Once a connection is established, a two-way communication will be available between the server and the client. They can communicate with each other using sytem call read/write for AF_UNIX protocol or send/recv for AF_INET protocol.
7) After communication, a system call close or shutdown is called to release the named socket.

Client side:
1) The client creates an unamed socket by calling socket.
2) Then the client uses system call connect to establish with the server by using the server's named socket as an address.
3) If the connection is accepted by the server, then the client can communicate with server using read/write for AF_UNIX protocol or send/recv for AF_INET protocol.
4) Similarly, when communication is finished, a call of close/shutdown is called.

Here's a very simple example of socket server and client programs.



// Local server

#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main()
{
int server_socketfd, client_socketfd;
int server_len, client_len;
struct sockaddr_un server_address;
struct sockaddr_un client_address;

unlink("server_socket");
server_socketfd = socket(AF_UNIX, SOCK_STREAM, 0);

server_address.sun_family = AF_UNIX;
strcpy(server_address.sun_path, "server_socket");
server_len = sizeof(server_address);
bind(server_socketfd, (struct sockaddr *)&server_address, server_len);

listen(server_socketfd, 10);
while(1)
{
char s_sendbuf[] = "Welcome to server.";
char s_recvbuf[255];
printf("Server waiting\n");

client_len = sizeof(client_address);
client_socketfd = accept(server_socketfd, (struct sockaddr *)&client_address, &client_len);


read(client_socketfd, s_recvbuf, 255);
printf("Message received from client: %s\n", s_recvbuf);
write(client_socketfd, s_sendbuf, strlen(s_sendbuf)+1);
close(client_socketfd);
}
}


// Local client


#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>

int main()
{
int socketfd;
int length, result;
struct sockaddr_un address;
char sendbuf[] = "Hi, this is client 1.";
char recvbuf[255];

socketfd = socket(AF_UNIX, SOCK_STREAM, 0);

address.sun_family = AF_UNIX;
strcpy(address.sun_path, "server_socket");
length = sizeof(address);

result = connect(socketfd, (struct sockaddr*)&address, length);

if(result==-1)
{
perror("Error:client 1");
exit(1);
}

write(socketfd, sendbuf, strlen(sendbuf)+1);
read(socketfd, recvbuf,255);
printf("Greeting from server: %s\n", recvbuf);
close(socketfd);
exit(0);
}

Monday, February 18, 2008

[Security] Winpcap installation problem: Error opening file for writing c:\windows\system32\drivers\npf.sys

Recently a friend asked me to help him install the winpcap on his computer. Winpcap is the packet capture and network monitoring library for Windows. Some network analyzer (such as Wireshark, the new version of Ethereal) or url sniffer (such as Url helper) need winpcap.

However, my friend got an error when installing the winpcap. He tried both the old 3.x version and the new 4.x version of winpcap, but everytime he got an error message "error opening for writing, c:\windows\system32\drivers\npf.sys". We googled the internet and found many people suffered the same problem and no solution was provided so far. Some people said it's due to the insufficient priveledge of the user account to install a driver file, but my friend's account is the administrator. Some other people said you may need to first delete the old npf.sys in the system32\driver folder and then reinstall it again, but we checked this folder and didn't find a file called npf.sys existing. We also closed the firewall and anti-virus software, but the problem is still there.

Finally, we made another try by copying a virus-free npf.sys from another computer to my friend's computer. Then, we found the reason. The system prompts that we cannot copy npf.sys (34,064 bytes) to npf.sys (0 bytes). But we just mentioned that we already did the check, and we didn't have a file named as npf.sys installed. So the only reason will be: there's a sub-directory called npf.sys existing. We checked again, and Bingo, we got it! There's a directory called c:\windows\system32\drivers\NPF.SYS existing on the computer (notice, it's not a file, but a directory). This NPF.SYS directory is empty and with name in upper case. I suspect there's some other softwares (with high probability to be a trojan or spyware) installed this NPF.SYS directory into the system32\drivers folder (they pretended to be a file of winpcap, then they could capture or sniff your password over the network). Later, it may be removed by anti-virus programs. But, it's not cleaned up, the resudual NPF.SYS directory is still on your system.

In summary, if you encounterred the same question (Error opening file for writing c:\windows\system32\drivers\npf.sys), try to first check if you have a NPF.SYS directory installed under \windows\system32\drivers.

Friday, January 25, 2008

[Programming] Quick guide for network programming (2)

Source code for Server's socket:


#include <winsock2.h>
#include <stdio.h>

void main()
{
WORD wVersionRequested;
WSADATA wsaData;
int err;

wVersionRequested = MAKEWORD( 1, 1 );
// call WSAStartup, initialize winsock.dll and negotiate the socket version
err = WSAStartup( wVersionRequested, &wsaData );
if ( err != 0 ) {
// Tell the user that we could not find a usable WinSock DLL.
return;
}


if ( LOBYTE( wsaData.wVersion ) != 1 ||
HIBYTE( wsaData.wVersion ) != 1 ) {
// Tell the user that we could not find a usable WinSock DLL.
WSACleanup( );
return;
}

// a) create a socket
SOCKET sockSrv = socket(AF_INET, SOCK_STREAM, 0);
// b) bind
SOCKADDR_IN addrSrv;
addrSrv.sin_addr.S_un.S_addr = htonl(INADDR_ANY); // INADDR_ANY use any IP address of the host, you can use inet_addr("127.0.0.1"); to designate a specific IP
addrSrv.sin_family = AF_INET;
addrSrv.sin_port = htons(6123); // use a port number larger than 1024
bind(sockSrv, (SOCKADDR *)&addrSrv, sizeof(SOCKADDR));
// c) listen
listen(sockSrv, 5);
// d) accept
SOCKADDR_IN addrClient;
int len = sizeof(SOCKADDR);
while(1) // waiting for requests from clients
{
SOCKET sockConn = accept(sockSrv, (SOCKADDR *)&addrClient, &len); // create a new socket for this connection
// e) send/recv
char sendBuff[100];
sprintf(sendBuff, "Welcome %s to server", inet_ntoa(addrClient.sin_addr));
send(sockConn, sendBuff, strlen(sendBuff)+1, 0);

char recvBuff[100];
recv(sockConn, recvBuff, 100, 0);
printf("%s\n", recvBuff);
// f) close the current socket and wait for another request
closesocket(sockConn); //release the socket for this connection
}

}



Source code for Client's Socket programming:


#include <winsock2.h>
#include <stdio.h>

void main()
{
WORD wVersionRequested;
WSADATA wsaData;
int err;

wVersionRequested = MAKEWORD( 1, 1 );
// call WSAStartup, initialize winsock.dll and negotiate the socket version
err = WSAStartup( wVersionRequested, &wsaData );
if ( err != 0 ) {
// Tell the user that we could not find a usable WinSock DLL.
return;
}


if ( LOBYTE( wsaData.wVersion ) != 1 ||
HIBYTE( wsaData.wVersion ) != 1 ) {
// Tell the user that we could not find a usable WinSock DLL.
WSACleanup( );
return;
}
// a) create a socket
SOCKET sockClient = socket(AF_INET, SOCK_STREAM, 0);
// b) connect
SOCKADDR_IN addrClient;
addrClient.sin_addr.S_un.S_addr = inet_addr("127.0.0.1"); // loopback address
addrClient.sin_family = AF_INET;
addrClient.sin_port = htons(6123);
connect(sockClient, (SOCKADDR *)&addrClient, sizeof(SOCKADDR));
// c) send/recv
char recvBuff[100];
recv(sockClient, recvBuff, 100, 0);
printf("%s\n", recvBuff);
send(sockClient, "This is from client B", strlen("This is from client B")+1, 0);

// d) close
closesocket(sockClient);
WSACleanup();
}

[Programming] Quick guide for network programming (1)



When we talk about network programming, basically it includes two parts: one is the socket programming, another one is the multi-threaded programming.


I'll give a step-by-step guidance for network programming in my blog. We'll first talk about socket programming, then go to the multi-threaded programming. I'll use the windows system as a demonstration. A TCP server will be first started up, then a TCP cilent will communicate with the server from the loopback IP address 127.0.0.1 with port number 6123. (I suppose you have already known the fundamental of network protocols, such as TCP/UDP, IP, port number etc.)

1) What is socket?
Shortly put, a socket is a combination of an IP address and a port number of a two-way communication link between two programs on the network.

2) Types of sockets:
For old version of socket, it provides two types of sockets:
"Stream Socket": provide connection-oriented, reliable data transmission service, use TCP protocol
"Datagram Socket": provide connetionless service, packets may not be received in order, use UDP protocol

Now some new types of sockets have been added, such as: SOCK_RDM (Reliable message datagram), SOCK_SEQPACKET(pseudo-stream packet based on datagrams)

3) Flow of the stream socket communication

On the server side, there are seven steps:
a) create a socket (socket)
b) bind the socket with a local IP address and a port (bind)
c) set socket to 'listen' mode (listen)
d) wait for client's request, when it arrives, accept the request and return
a new socket for the connection (accept)
e) communicate with client using the new returned socket (send/recv)
f) return, wait for another client's request
g) close the socket (close)

On the client side, there are four steps:
a) create a socket (socket)
b) send request to the server ((connect))
c) communicate with server (send/recv)
d) close the socket (close)

4) Step-by-step programming

Follow the steps in 3), we first write the code for a TCP server. We use visual c++. You can find the source codes in the next post. Here is the explanation of the code. It's better to read the code and the following descriptions simultaneously.

<1> create a Win32 Console Application, set project name to TCPServer

<2> write a main function

<3> initialize the win32 socket DLL, call function WSAStartup();

int WSAStartup(
WORD wVersionRequested,
LPWSADATA lpWSAData
);
WSAStartup has two jobs to do:
i) initiates use of Ws2_32.dll: if WinSock.dll or lower network subsystem is not initialized correctly or is not found, will return WSASYSNOTREADY.
ii) negotiate the socket version:
if requested version < MAX version supported by the WinSock.dll, use min(requested version, MAX version)
if requested version < MIN version supported by the WinSock.dll, return WSAVERNOTSUPPORTED, then call function WSACleanUP() to release resource that has been allocated for WSAStartup

The WSADATA or *LPWSADATA used in WSAStartup is a structure, it's defined as
typedef struct WSAData
{
WORD wVersion; // winsock version
WORD wHighVersion; // highest version in the current winsock library
char szDescription[WSADESCRIPTION_LEN+1];// for special use
char szSystemStatus[WSASYS_STATUS_LEN+1];// for special use
unsigned short iMaxSockets; // maximum num of sockets that can be
// opened, don't use it
unsighed short iMaxUdpDg; // maximum length of packet, don't use it
char FAR * lpVendorInfo; // reserved for vendor, never been used
} WSADATA, *LPWSADATA;


<4> create a socket

SOCKET WSAAPI socket(
int af, // for tcp-ip, it must be AF_INET or PF_INET
int type, // for winsock 1.1, only SOCKET_STREAM or SOCKET_DGRAM
int protocol // protocol to be used, is specific to the address family and
// socket type, set to 0 for default
);

If no error occurs, it will return a descriptor referencing the new socket. Otherwise, return INVALID_SOCKET.

<5> bind

int bind(
SOCKET s, // descriptor identifying an unbound socket
const struct sockaddr* name, // the address to assign to the socket,
// it may be varied with different
// address families, so we should use the next
// parameter to indicate the
// length of this address structure
int namelen // length of the value, in bytes
);


The sockaddr is defined as:
struct sockaddr {
u_short sa_family;
char sa_data[14];
};

For different address family, we should use differnt structure to replace this sockaddr.
For TCP/IP, sockaddr_in is used.
The definition of sockaddr_in is:

struct sockaddr_in {
short sin_family; // for IP address, it's AF_INET
u_short sin_port; // port number
struct in_addr sin_addr; // IP address of the host,
// should use "network byte order"
char sin_zero[8]; // padding bytes, to make the length equals to the
// sockaddr structure
};
where the "network byte order" is the same as the well-known big_endian order.

It will return 0, if successful; otherwise, return SOCKET_ERROR

<6> listen

int listen(
SOCKET s, // descriptor identifying a socket
int backlog // max length of the queue of pending connections
);

<7> accept

SOCKET accept(
SOCKET s, // identifying a socket
struct sockaddr* addr, // used to store the client's ip addr and port num
int* addrlen // length of the
);

<8> send/recv
int send(
SOCKET s,
const char* buf,
int len,
int flags
);

<9> Include the header file , add ws2_32.lib to the project, the operation in Visual C++ is:
project -> setting -> link -> Object/library modules, append ws2_32.lib to the end


<10> Then let's go to the client side. First add a new project to the current workspace, name it as "TCPClient".

<11> Similar as the Server code, first call WSAStartup to initialize the ws2_32.dll and negotiate the socket version, then create a socket

<12> connect

int connect(
SOCKET s,
const struct sockaddr* name,
int namelen
);

<13> After connect, call function send/recv, and finnaly call closesocket and WSACleanup as in server side.

<14> Include the header file and add ws2_32.lib to this project too

<15> Compile all files, run TCPServer first, then run TCPClient, we will get the following results, as shown in the figure

5) Some functions are used in the program for the conversion of the address, they are
htons(): converts a u_short from host to TCP/IP network byte order
htonl(): converts a u_long from host to TCP/IP network byte order
ntohs(): converts a u_short from TCP/IP network byte order to host byte order
ntohl(): converts a u_long from TCP/IP network byte order to host byte order
inet_ntoa(): converts an (Ipv4) Internet network address into a string in Internet standard dotted-decimal format
inet_addr(): converts a string containing an (Ipv4) Internet Protocol dotted address into a proper address for the IN_ADDR structure


Thursday, January 24, 2008

[Algorithm] Reverse words in a string

This is an interview problem. Given a string, such as "tell me something", output the string as "something me tell", i.e. reverse every word in the string. Here is a demo program in C code, it's easy to understand with the comments given in the code. The basic idea is, scan the string from the end, find the beginning and the end of each word, then write to a temporary buffer.





#include <iostream.h>
#include <stdlib.h>
#include <string.h>

void ReserveWordsInString(char string1[])
{
// get the length of the input string
int length=0;
while (string1[length]!='\0')
length++;
// allocate memory space for a temporary buffer, used to store the temporary result
char *buffer= (char *)malloc(length+1);

// declare four indicators
// MoveInd: from length to 0, used to traverse the input string
// EndInd: indicate the end of a word that is to be reversed in the input string
// BeginInd: indicate the begin of the word that is to be reversed in the input string
// WriteInd: the indicator of the writing position in the temporary buffer
int MoveInd, BeginInd, EndInd, WriteInd;
// initialize the MoveInd to the end of the input string, WriteInd to the begin of the temporary buffer
MoveInd = length-1;
WriteInd = 0;

while (MoveInd>=0)
{
if (string1[MoveInd]!=' ') // if it's a word charactor
{
EndInd=MoveInd;
// scan the current word, find the next non-word charactor
while(MoveInd>=0 && string1[MoveInd]!=' ')
{
MoveInd--;
}
// copy the word to the temporary buffer
for (BeginInd=MoveInd+1; BeginInd <= EndInd; BeginInd++, WriteInd++)
{
buffer[WriteInd]=string1[BeginInd];
}

}
else
{
// write the non-word charactor directly to the temporary buffer
buffer[WriteInd++]=string1[MoveInd--];
}
}
// it is better to append \0 to the end of the buffer
buffer[length]='\0';

// copy the buffer to the input string for output
for (int i=0; i<length; i++)
string1[i]=buffer[i];

// free the allocated memory for the buffer
free(buffer);
}
void main()
{
char string1[]="This is a C program, for word reverse.";
// this is a tricky method in getting the size of an array, explanation is below the code
int length = sizeof(string1)/sizeof(string1[0]);

// Call the function
ReserveWordsInString(string1);

// output the result
for (int i=0; i<length; i++)
{
cout<<string1[i];
}
cout<<endl;
}





Notes: In this program, we encountered a problem, how to get the length of an array or a string?

A well-known and tricky method is to use "sizeof(array)/sizeof(array[0]);", this works in the main program. However, if the array is the input argument of a function, this one doesn't work. Because in this case, the type of the array identifier "decays" into a pointer to the base type, and its value is set to the address of the first element in the array.
So, my opinion is, always traversing the array from the beginning to the end to get the length, like "for(int i=0; array[i]!='\0'; i++)".

A more decent algorithm is: first reverse the whole string in characters, e.g. reverse "tell me something" to "gnihtemos em llet", then using the similar method as above to locate the beginning and end of each word, then reverse each word in characters, e.g. reverse "gnihtemos" to "something". This is an in-place method, which avoids using the extra temporaray buffer.