You are here: Home / ECPE 170 / Labs / Lab 7: Network Socket Programming (Basic)

Lab 7: Network Socket Programming (Basic)

Overview

In this lab, you will use the Python programming language to implement a simple HTTP client that can download files from a webserver.

 

References

There are a number of official and un-official references for Python that you may find useful. Be warned, however, that Python syntax and libraries have changed between version 2.x and 3.x. Thus, for this lab, be sure any reference you use is for Python 3.x.

 

Lab - Getting Started

To begin this lab, start by obtaining the necessary boilerplate code from here.  Enter the class repository:

Extract and copy the files you want to your lab folder and change the directory to lab07:

unix>  cd ~/2022_spring_ecpe170/lab07

 

Python Configuration

Ensure that the latest version (3.8+) of Python is installed on your computer.

unix>  sudo apt install python3

Verify that Python is working. Ask it to print the version number, which will be useful in debugging errors/quirks later: Note: This is a CAPITAL letter V!

unix>  python3 -V

The output should be similar to: Python 3.8.x

 

Python IDE: PyCharm

If you want an friendly IDE to write Python code in, complete with handy features such as code completion and auto-indenting, install the free Community Edition of PyCharm.  

PyCharm

Installation Procedure:  (alas, PyCharm is not included in the standard package library)

  1. Install a necessary package: sudo apt-get install python3-distutils
  2. Download the Community Edition for Linux (select the corresponding tab) from http://www.jetbrains.com/pycharm/download/
  3. Extract the .tar.gz archive file that was downloaded
  4. On the command line, navigate to the extracted archive folder and enter the /bin subdirectory inside
  5. Run the installer script at the command line:  ./pycharm.sh
  6. Create a desktop icon for the program:  Configure->Create Desktop Entry
  7. By default, PyCharm uses Python 2.7.   Switch the default interpreter to use Python 3.8.x
    1. Configure->Settings ->Project Interpreter, then click the gears button and add a new environment using the Python 3.8 interpreter

 

Note: I  will still run your Python programs at the command line, not through the IDE.  So, don't embed any custom settings in the IDE project properties.

 

Lab Part 1 - Demo Client and Server

A demo Python 3 client application and server application are provided for your use.  Download the programs to your computer:

wget https://ecs-network.serv.pacific.edu/ecpe-170/lab/client.py
wget https://ecs-network.serv.pacific.edu/ecpe-170/lab/server.py

Mark the files as "executable" so you can run it as a program. (The + symbol means 'add flag', and 'x' represents the executable flag)

unix>  chmod +x client.py
unix>  chmod +x server.py 

Now, in two separate terminal windows, execute the demonstration program:

Terminal Windows 1:
(Run the server program and tell the server to listen on port 5678)

unix>  ./server.py 5678

Terminal Window 2:
(Run the client program, and tell the client to connect to the server located at IP 127.0.0.1 and listening on port 5678. That IP is equivalent to 'localhost', i.e. your computer) 

unix>  ./client.py 127.0.0.1 5678

 

Lab Report:
(1) What is first line of the python script that starts with #! doing?  Where in ECPE 170 have you seen this before? 

  

Lab Part 2 - HTTP Basic Skills

Before writing a program that communicates with an HTTP server, you are going to manually test your knowledge of HTTP.  The netcat client program allows you to open a TCP socket to a port and send ASCII characters. It will print both the characters that you send and the characters that the server sends.

To invoke the netcat client to connect to www.google.com on port 80:

unix>  netcat -C www.google.com 80

The -C argument specifies that when you hit the enter key on the keyboard, netcat will send the \r\n (carriage return, line feed) sequence of two characters, which is required for the HTTP protocol.

Once the connection to the web server is open, you can send an HTTP request.  Here is an example HTTP request to download the file at http://www.google.com/about/

GET /about/ HTTP/1.1
Host: www.google.com
Connection: close 
 
<<SERVER RESPONSE STARTS HERE>> 

Note that the HTTP client (in this case, you!) must send an extra blank line after the last request line. This trigger tells the web server to begin processing the request. (Technically, the web server is looking for a \r\n\r\n sequence of characters).   After the request is sent, the reply should immediately follow on the same connection.

 

Lab Report:
(2) Document the HTTP request and the server response when you manually download the HTML file at http://spaceref.com via Netcat.
(By "document", you should provide the full client request and a partial server response (top 40-50 lines is sufficient for me to tell if you downloaded the right file). The script utility can make this capture easy for you - see below.) 
(3) Document the HTTP request and the server response when you manually download the HTML file at http://neverssl.com/ via Netcat
(4) Document the HTTP request and the server response when you manually download the PNG image file at http://spaceref.com/img/logo_320red.png via Netcat
Note: Is there a good reason why it doesn't make sense to include the server response (at least, the data portion) in your lab report?  On a related note, if your Terminal window hangs during this step, at least you'll know why!

Requirements for your HTTP request:

  1. Use the HTTP 1.1 protocol
  2. Specify the Host field, which is the domain name of the server that should answer your request. (In HTTP/1.1, there could be multiple servers -- for example, gmail.google.com and www.google.com -- listening on the same IP address).
  3. Specify that the web server close the socket connection immediately after sending the requested file. (This allows for a more simple client implementation.)

 

Tip: Tired of important text scrolling off the top of your terminal window? Adjust the "scrollback" option. Go to Edit->Preferences->Scrolling and uncheck the limit on the "Limit Scrollback" option (or at least set it to a large fixed number - 10000 lines?)

 

Lab Part 3 - HTTP Download Client

Write a Python3 program called display.py to retrieve files from a web server via HTTP. Although this program could retrieve files of any type, we will use it solely to retrieve image files. and then display them after they have been downloaded.  

Program requirements:

  1. Your program must download image files from a web server via HTTP using the Python3 programming language and the socket module.
    1. Warning: Python has a built-in HTTP client module and URL request module.  You *cannot* use these (or anything similar) for this lab, however, and zero points will be awarded if you do! The reason for not using this module is because it hides how the HTTP protocol works, and the purpose of this lab is to actually learn about the protocol operation.  Instead, you must use the lower-level socket module.
  2. The HTTP Request sent to the server must be printed out on the screen
  3. The HTTP Response received from the server must be printed out on the screen (just the header, not the data)
  4. The entire response from the server must be downloaded in increments of max_recv bytes per system call to recv(). (This is a variable defined in the boilerplate code, and is a reasonable value such as 64kB).
  5. Downloaded files must be saved to local disk with the same file name as on the server. 
  6. Downloaded files must be displayed on screen by invoking the eog image viewer utility (see code at the end of the boilerplate file to do this)
  7. Your program will take 2 command-line arguments: the web server port number, and the URL of the image to display, e.g.:
unix>  ./display.py --port=80 --url=http://ut-images.s3.amazonaws.com/wp-content/uploads/2009/09/SED_wall_1920x1200.jpg

Tip: There is boilerplate code provided for this exercise, and step-by-step instructions contained within.

Your program will be tested with the following image URL:

http://ut-images.s3.amazonaws.com/wp-content/uploads/2009/09/SED_wall_1920x1200.jpg

 

 

(Optional) Lab Report:
(1) How would you suggest improving this lab in future semesters?