Protocol Programming

• Learn how to use UNIX socket interface to set up Inter-Process Communication (IPC) between processes in the network.
• Learn the relationship between the transport protocols and the socket function calls.
• Learn how to set up time-out in UNIX and use select system call to listen to multiple inputs (from users, timer, and network).
• Protocol Programming project:
  Design and implement a Multimedia Intelligent Network Service (MINS)
  
  which allows users without the knowledge of others' whereabouts to communicate in "real-time" with text and graphics!

References:

"UNIX Network Programming," by W. Richard Stevens, Prentice Hall, 1990
ISBN 0-13-949876-1.

"An Introductory 4.3BSD Interprocess Communication Tutorial" in Unix
Programmer's Manual Supplementary Documents 1, PS1:7

Protocol Implementation

• Stepwise refinement (transformation)

Protocol spec. (such as CFSMs) Æ code

• Automatic generation of program skeletons
• Implementation choices
  --modular structure
  procedures, processes, protocol entities, management functions,...
  --interface between protocol layers
  * internal
  * accessible to "user" or other layers
  --error handling of peer entity, user, others
  --buffer management (passing msgs between layers)
  --use of inter-task communications libraries (e.g., unix domain socket)
  e.g. Berkeley Socket, system V Transport Layer Interface (TLI) on UNIX.
  WinSock on Window 3.1, ...

UNIX Workstation Network Interface

Relation to Message Formats of TCP/UDP/IP

Berkeley Socket--An Application Program Interface

A set of library function calls forms an abstraction called socket to facilitate the programming involving inter-process communication (IPC).

Idea Æ The socket mimics the file I/O operations:
sending message--write(socket, msgbuf, strlen(msgbuf))
receiving message--read(socket, msgbuf, Maxmsglength)

However the creation of sockets is different, depending on the types of IPC:

• For process-to-process communication within a UNIX machine.
  a path name is used as an address to identify a socket,
  e.g. /tmp/cs522.chow.UA2Gui
  The space this type of socket addresses can be in is called UNIX domain .
  This type of socket is also called UNIX domain socket.
• For process-to-process communication between processes at two different machines using internet (TCP/UDP/IP) protocols.
  The 4-byte internet address of the host and a 2-byte port id is used to identify
  the socket, e.g., 128.198.162.62 and portId 21 identify the socket to which
  the FTP process on sanluis listen.
  See /etc/services for the designated port numbers.
  The space this type of socket addresses can be in is called Internet domain .
  This type of socket is also called Internet domain socket.

Socket Creation

#include <sys/types.h>

#include <sys/socket.h>

int socket(int family, int type, int protocol); // socket function call return fd

/* use man to see the description of each of the IPC routines and parameters.*/

Typical calls for Internet Domain sockets are

sockfd = socket(AF_INET, SOCK_STREAM, 0)
for byte stream connection-oriented service, default protocol is TCP.

sockfd=socket(AF_INET, SOCK_DGRAM, 0)
for datagram connectionless service, default protocol is UDP.

Typical calls for UNIX Domain sockets are

sockfd=socket(AF_UNIX, SOCK_STREAM, 0)
for byte stream connection-oriented service, default protocol is UNIX internal protocol.

sockfd=socket(AF_UNIX, SOCK_DGRAM, 0)
for datagram connectionless service, default protocol is UNIX internal protocol.

The return value is a file descriptor for later reference.

If socket() fails, return value is negative.

Flow Chart of Datagram Setup

Bind Internet Domain Socket to Address

int bind(int sockfd , struct sockaddr * myaddr , int addrlen );

Before the bind() call, specify the proper socket address in struct sockaddr:

For a receiving Internet domain socket,

struct sockaddr_in from;
int length;

from.sin_family = AF_INET;
from.sin_addr.s_addr = INADDR_ANY; /* ask system to assign a port no.*/
from.sin_port = 0

/* 0< portno.< 1024 are reserved for privilege process */
/* to find the assigned port no., call getsockname()*/

bind(sockfd, &from, sizeof (from));
length = sizeof(from);
getsockname(sock, &from, &length)

printf("socket assigned port=%d\n", ntohs(from.sin_port)

Receiving Internet Datagram Messages

int sockfd;

struct sockaddr_in from;
char frombuf[2048], toBuf[2048];

/* after bind the socket to address */
while (flag) {
n = recvfrom(sockfd, fromBuf, sizeof(fromBuf), 0, &from, &length);
fromBuf[n] = 0; /* null terminate */
printf("received msg=%s\n", fromBuf);
/* do some processing according to the request */
/* use the returned sender's socket address in the from structure to */
/* send the reply message */
sprintf(toBuf, "received message: %s\n", fromBuf);
if (sendto(sockfd, toBuf, strlen(toBuf), 0, &from, sizeof(from)) < 0)
perror("sending datagram message");
}
close(sockfd);

~cs522/project/ipc contains sample programs, idgr.c (receiver), idgs.c (sender for I386, Alpha, and idgs5.c (sender for Solaris `sys V' UNIX on SPARC, zeppo).

Sending Internet Datagram Messages

int sockfd;

struct sockaddr_in to;
struct hostent *hp, *gethostbyname();
char msgbuf[1024];

sockfd=socket(AF_INET, SOCK_DGRAM, 0)
toaddr.sin_family = AF_INET;

/* hostname and portno of the receiver may be entered from command line */
hp = gethostbyname(receiver_hostname); /* host name of receiver process */

bcopy(hp->h_addr, &to.sin_addr, hp->h_length); /* copy the 4 byte addr*/
/* sys V unix use memcpy(&to.sin_addr, hp->h_addr, hp->h_length); */

to.sin_port = htons(toPort_no); /* port number of the receiver process */
strcpy(msgbuf, "how are you?");
if (sendto(sockfd, msgbuf, strlen(msgbuf), 0, &to, sizeof(to)) < 0)
perror("sending datagram message");

Running idgr and idgs

zeppo> cd ~cs522/project/ipc/SPARC

zeppo> idgr

socket has port #46972

The no. of bytes received=20

received msg=This is packet one!

rcvd sockaddr_in: Domain=2, Hostname=sanluis, Port=2727, Address=128.198.2.62,

sanluis> cd ~cs522/project/ipc/I386

sanluis> idgs zeppo 46972

socket has port #2727

sending message: This is packet one!

number of bytes received=39

received ack msg=received message: This is packet one!

You can copy idgr.c, idgs.c, idgs5.c, and Makefile to you local directory.

sanluis> make //You have to create I386 and SPARC

zeppo> make idg-SPARC // subdirectories for the executable code.

Receiver: Bind UNIX Domain Socket to Address

int sockfd;

struct sockaddr_un addr, to;
char frombuf[2048], toBuf[2048];

addr.sun_family = AF_UNIX;

sprintf(addr.sun_path, "/tmp/cs522.%s.server", getlogin());

unlink(addr.sun_path); /* if previous incarnation of socket still exists, kill it*/

if (bind(sockfd, &addr, sizeof(struct sockaddr_un))) {

perror("binding name to datagram socket");

exit(1);

}

/* ls -F /tmp you will find a file with the same name there */

/* the socket file has srwxr-xr-x as access right and 0 length */

Why we put the login name in the socket pathname?

In what situation this will not uniquely identify the socket?

Receiver: Waiting/Reply UNIX Domain Datagram

to.sun_family = AF_UNIX;

sprintf(to.sun_path, "/tmp/cs522.%s.client", getlogin());

printf("unix domain datagram client uses sun_path=%s\n", to.sun_path);

/* receive message */

while (flag) {

n = recv(sock, fromBuf, sizeof(fromBuf), 0);

if (n < 0) perror("receiving datagram message");

fromBuf[n] = 0; /* null terminate */

printf("The no. of bytes received=%d\n", n);

printf("received msg=%s", fromBuf);

/* formulate the response */

sprintf(toBuf, "received message: %s\n", fromBuf);

if (sendto(sock, toBuf, strlen(toBuf), 0, &to, sizeof(to)) < 0)

perror("sending datagram message");

}

unlink(addr.sun_path); /* clean up by removing the socket file descriptor */

close(sock);

Sender: Sending UNIX Domain Datagram

/* create name structure with wildcard using INADDR_ANY */

addr.sun_family = AF_UNIX;

sprintf(addr.sun_path, "/tmp/cs522.%s.client", getlogin());

unlink(addr.sun_path);

if (bind(sock, &addr, sizeof(struct sockaddr_un))) {

perror("binding name to datagram socket");

exit(1);

}

/*send message */

sprintf(toBuf, "This is packet one!\n");

if (sendto(sock, toBuf, strlen(toBuf), 0, &to, sizeof(to)) < 0) {

perror("sending datagram message"); exit(1);l}

n = recv(sock, fromBuf, 1024, 0);

if (n < 0) perror("receiving datagram message");

fromBuf[n] = 0; /* null terminate */

printf("received ack msg=%s\n", fromBuf);

unlink(addr.sun_path);

close(sock);

Flow Chart of Stream Connection Setup

Stream Connections

int sockfd, newsockfd;

if ((sockfd=socket(AF_INET, SOCK_STREAM, 0)) < 0)
perror("socket creation error");

if (bind(sockfd,...) <0) perror("binding error");

if (listen(sockfd, 5) < 0) perror("listen error");
/* allow 5 connection request in queue*/

for (;;) {

newsockfd = accept(sockfd, peer, addrlen);
if (newsockfd < 0) perror("accept error");
if (fork() == 0) { /* child process */
close(sockfd);
doit(newsockfd); /* process the request, first use read()*/
exit(0);
}
close(newsockfd);
}

The decision of using fork() depends on whether the complexity of the task worth the overhead of forking process.

Sample programs, istr.c, ists.c, in ~cs522/project/ipc.

Running istr and ists

chico> istr

before getsockname socket has port #0

socket has port #36085

rcvd msg-->test

What is your reply ("$" to break connection)?ok

rcvd msg-->hi

What is your reply ("$" to break connection)?$

sanluis> ists chico 36085

msg for receiver("$" to exit):test

reply msg=ok

msg for receiver("$" to exit):hi

receiver breaks the connection, exiting...

Protocol Programming Project Phase 1

Exercise 1: Compare the performance of UNIX and Internet Domain IPC.

Goal: Learn how to write distributed programs using socket for IPC.

Task: Using SOCK_DGRAM. Write a sender program that takes the UNIX domain socket pathname of a receiver process, sends consecutive 10000 small messages, "packet 1" ... "packet 10000" to the receiver on the same machine, then wait for the first ack message, print out the ack message, then exit.
Write the receiver program that takes the UNIX domain socket pathname of the sender, enter a loop that receives a message, prints out the message content, then reply with an "ACK" message, (note it will send 10000 acks.)
Compile the sender and receiver on both wetterhorn and zeppo.
Time the execution of sender on the zeppo and wetterhorn using time sender

Exercise 2: Repeat the above exercise on Internet Domain IPC, i.e., sender and receiver at different machine. The sender takes the hostname and port number of receiver as input. The receiver prints out the port it was allocated.
Time the execution of sender with the following four situations:
a) sender on zeppo; receiver on zeppo.
b) sender on zeppo; receiver on wetterhorn.
c) sender on wetterhorn; receiver on zeppo.
d) sender on zeppo; receiver on wetterhorn.

Exercise 3: Repeat the above task by redesigning sender & receiver using SOCK_STREAM.

Report and discuss your timing results on those ten cases.

Note that you may lose messages in some of these cases.

Exercise 4: Describe what happen if multiple senders try to send messages to a receiver in both the STREAM and DATAGRAM cases.

Exercise 5: Byte ordering problem: In ~cs522/project/ipc/pens.c and penr.c

a message structure is defined as follows:

struct Message_header {

char sender[10]; /* sender login */
int sessionID; /* starting from1 */}

struct Pen_Message {

struct Message_header header;

unsigned char msg_type;
unsigned char size; /* 1x1, 2x2, 4x4, 8x8, or 16x16 pixel size dot */
short x;

short y;} pen;

pen.x = 2; pen.y=32; strcpy(pen.header.sender, getlogin()); pen.header.sessionID = 1; pen.msg_type = 3; pen.size =8;

Use pens to send out this pen message on wetterhorn,

Use penr receive and print out all the fields in this pen message on zeppo.

See if the pen.x and pen.y changed. Use script to document your result.

Discuss why there is such a change.

Mail me your timing results and analysis, and include the pathnames to your programs (change the access right so that I can read them).

Handle multiple inputs and time-out

There is a special bit-vector type, fd_set, allowing you to indicate which file descriptors you would like to pay attention to.

#include <sys/types.h>

#include <sys/time.h>

FD_ZERO(fd_set * fdset ); /* clear all bits in fdset */

FD_SET(int fd , fd_set * fdset ); /*turn the bit for fd on in fdset */

FD_CLR(int fd , fd_set * fdset ); /* turn the bit for fd off in fdset */

FD_ISSET(int fd , fd_set * fdset ); /* test the bit for fd in fdset */

struct timeval {

long tv_sec; /* seconds */

long tv_usec; /* microsecond */

}

int select(int maxfdpl , fd_set * readfds , fd_set * writefds , fd_set * exceptfds ,
struct timeval * timeout );

maxfdpl: the number of file descriptors to be checked.

readfds: the input channels (file descriptors) to receive incoming msgs.

writedfds: the output channels (file descriptors) to send msgs.

exceptfds: the execptional signal channels (file descriptors).

timeout: a pointer to data that specify the time-out value.

If (timeout == NULL) select() will wait indefinitely until some of the channels have "actions"

If (timeout NULL) select() will return when

• some of channels indicates "actions", returns value of select() will be the number of file descriptors that have "actions". The fd_set* will be overwritten with the bits set on for those with actions and bits set off for those without.
• time expires and time-out event occurs, return value is 0.

I/O multiplexing using select()

fd_set readfds;

int unix_d_sockfd, maxfdpl;

int inet_d_sockfd;

struct timeval timeout;

/* assume you have create the above sockets properly */

timeout.tv_sec = 3; /* three seconds and */
timeout.tv_usec= atol("500000"); /* 500000 m seconds, use atol() to convert str*/
maxfdpl=(unix_d_sockfd>inet_d_sockfd)?unix_d_sockfd+1: inet_d_sockfd+1;

while (quitflag) {
FD_ZERO(&readfds); FD_SET(0, &readfds); /* listen to stdin */
FD_SET(unix_d_sockfd, &readfds); /* fd_set need to be reset, why? */
FD_SET(inet_d_sockfd, &readfds); /* because select() overwrites the value*/
if ((i=select(maxfdpl, &readfds, 0, 0, &timeout)) < 0) {
perror("select error"); exit(1); }
if (i==0) { printf("time-out"); /* return value of select() is 0*/ }
if (FD_ISSET(0, &readfds)) { scanf("%s\n", buf); /* user input */ }
if (FD_ISSET(unix_d_sockfd, &readfds)) { /* unix_d_sockfd receives msg */}
if (FD_ISSET(inet_d_sockfd, &readfds)) { /* inet_d_sockfd receives msg */}

} /* See sample program ~cs522/project/gui/ua.c for more details */

Mapping CFSM to C-code

How to implement the following simple protocol using datagram sockets.

Goal: Learn how to use select and time-out.

D represents any string user type in except `q'.

When time-out happens, print out "time out n time" where n the number of time-outs during the sending of a message.

Sample of C-code implements Machine M

int state = 1;

int quitflag = 1;

int retry = 1;

while (quitflag) do {

switch (state) {

case 1: /* state 1 */
fflush(stdin); scanf("%s", buf);
if (strcmp(buf, "q") ==0) {
/* userinput(q) */
state=4;
} else {/* userinput(D), Dq */
state=2;}
break;

case 3: /* state 3 */
/* set time-out value */
/* set fdest */
if ((i=select(sinfd+1, fdset, 0, 0, timeout)) < 0) {
perror("select"); exit(1);}
if (i==0) /* timeout */
if (retry == 5) state=4;
else {retry++; state=2;}

Sending/Receiving Different Message Types

struct PenRecord {

unsigned char msgType;

unsigned char size;

short X;

short Y;};

struct ConnRecord {

unsigned char msgType;

unsigned char size;

char name[20];

char originator[20];};

union Record { /* see ~cs522/project/mins/msg.h for detail */

PenRecord pr;

ConnRecord nr;

ACKRecord ackr;

...} r;

r.pr.msgType = PEN; r.pr.X=320, r.pr.Y=600; r.pr.size=4;

r.nr.msgType=CONN; strcpy(r.nr.originator, getlogin()); strcpy(r.nr.name, dst_usr)

How to know the type of received msgs?

struct sockaddr_un toGui;

struct sockaddr_un fromGui;

n = recv(sockinet, &(r.pr.msgType), sizeof(r), 0);

switch(r.pr.msgType) {

case PEN:

printf("receiving pen msg, X=%d, Y=%d\n", r.pr.X, r.pr.Y);

if (sendto(socktoGui, &(r.pr.msgType), sizeof(r), 0, &toGui, sizeof(toGui)) < 0)

perror("sending datagram msg"); /* relay the msg to GUI process */

break;

case CONN:

printf("receiving connect msg, originator =%s, dst user=%s\n",

r.nr.originator, r.nr.name);

/* update table, and send conn msg to Gui */

break;}

Protocol Programming Project Phase 2
Due day:

Implement the alternating-bit protocol using internet datagram sockets.

Goal: Learn how to use select and time-out.

Sender and receiver should print out the state changes and msg exchanged.

C1 and C2 can be the same program and take command line parameter to behave normally or lose every other message.

Simulate this protocol on four machines with C1 lose odd msgs. Use script to capture the state changes on Sender and the delivered msgs on receiver side.

Send path name of the code and script to me.

Possible declaration of msgs record:

struct Msg {

boolean alternating-bit;

char buf[128];}

Protocol Programming Project

Protocol Programming Phase 3, Final Phase

The preliminary version of the graphical user interface program, sd , for the protocol is ready for you to test your user agent and intelligent server programs.

The source of sd is in the directory ~cs522/project/sharedraw

It uses the InterViews graphic package to construct the graphic user interface.

In the same directory there are a simplified version of user agent program, ua.c, and a dummy datagram server, is.c, which receives and replies with simple ack message.

For testing, is is started first, followed by ua and sd .

sd listens to a UNIX socket, which is identified by the path
/tmp/cs522.<login>.UA2Gui

ua listens to a UNIX socket, which is identified by the path
/tmp/cs522.<login>.Gui2UA

After each session, you should remove those paths, otherwise next time you execute the sd and ua you will get socket in use error.

Send me email if you find bugs in sd .

The current version of sd did not support multiple sessions (to multiple users). It only support shared editing among two users.

Protocol Programming Phase 3.1

Document your design of UA and IS using CFSM model. Describe the message formats used among UAs and IS. Due day 10/24

The database updates, which are triggered by the reception of "login" or connect request messages, should be documented as internal transitions, e.g.,
(S0, S1, +CR(org=$u1, dst=$u2)) // Sn is the state of the CFSM represents IS.
// org and dst are the fields in the CR (connect request) message
// $u1 and $u2 are values of the fields.
(S1, S2, $u2 entry exists in database and db.host($u2)=$h, db.portId($u2)=$n)
// $h is the value in the host field of the $u2 database entry;
// $u2 uses the most recent context of the CFSM path for its value.
(S1, S3, $u2 entry does not exist in database)
(S2, S4, -CR(org=$u1, dst=$u2)/($h, $n)) // define your own notation
// here, I use /($h, $n) to specify the socket of destination UA
(S4, S5, timeout) // How to distinguish time-outs for different requests
(S4, S6, +ACK) // is a challenging problem.
(S4, S7, -NAK)...

When the user is connected and a third party tries to connect, UA should relay the connection request and let the user decide using sharedraw interface.

On 10/26, we will set up the MINS project demo schedule.

Lessons Learned in Exercises 1&2

• Datagram service lost packets
• DEC->SUN3, using stream, when the sender executes close(socketfd) before receiver finishes receiving message, the receiver receives signal and exits.
• TCP flow control did not help to pace the sending of faster machine (DEC).
  It caused the receiver to hang.
• Exercise 2 not only shows the byte ordering problem but also show the memory align problem in defining message structure.

blanca> DEC/idgs2 atwood 2126

socket has port #2967

hostname=blanca

sending message: blanca sessionID=1 msg_type=2 size=1 x=200 y=100

size of pen structure=24

Message:blanca

62 6c 61 6e 63 61 00 00 00 00 00 00 01 00 00 00 02 01 c8 00 64 00 00 00

atwood> SUN3/idgr2

socket has port #2126

received number of bytes=20

size of pen structure=20

Message:blanca

62 6c 61 6e 63 61 00 00 00 00 00 00 01 00 00 00 02 01 ffffffc8 00

parse: get rcv_name=blanca, SessionID=256 msg_type=0, size=0 x=513 y=-1433