Project 1 - HTTP Server

Project Objectives

In this project, you will be implementing a simple web server in the Python3 programming language. In doing so, you will gain: 

• Hands-on experience with the Hyper Text Transfer Protocol (HTTP)
• Hands-on experience with TCP sockets
• Hands-on experience with Python programming, including details such as file I/O, string parsing, and command-line argument parsing.

To support these objectives, you are not allowed to use any pre-built HTTP, URL-parsing, or socket management libraries.

Requirements

The HTTP 1.1 protocol is extremely detailed and complex.  You could spend a week reading RFC 2616 which formally describes the protocol. Fortunately, there is no need to do so for this project, as we are only going to be implementing a tiny subset of the standard. Your high-level goal is the following:

Implement a sufficient part of the HTTP 1.1 protocol in your web server such that a user could download the official "Project 1 demo website" (describe in Testing, below) using any of the following web browsers: Chrome, Firefox, Safari, and Internet Explorer. Your server should faithfully transmit any file requested by the web browser when downloading the demo website, which may include HTML, CSS, Javascript, or images. Once rendered by the browser, the website hosted by your web server should be indistinguishable when compared to the site hosted on the original web server. Any broken images, missing text, JavaScript warnings, etc... will be taken as an indication that your server is corrupting the files it transmits.  (Obviously, if the web server is corrupting files, then we cannot expect the web browser to be able to display a perfect-looking website)

You must use the Python 3 programming language, specifically version 3.4.x or newer, which should be widely available.
(Note: On most systems, the binary will be called python3.  To be safe, run python --version and python3 --version at the command line to see what you have.) 

You must develop your web server on a Linux operating system.  Either a virtual machine or dual-boot arrangement is acceptable. (Your assignments will be graded on a Ubuntu 14.04 LTS machine, if you are interested in developing in exactly the same environment.)

You must support web browsers sending requests using version 1.1 of the HTTP protocol, and accept connections (and deliver responses) via TCP.

You must support the GET request method from clients. Any other request that is not GET should produce a 501 "Not Implemented" error response. This is preferable to silently failing.

You must support HTTP status codes 200 (OK), 404 (Not found), and 501 (Request method not implemented) as possible responses by your server.

A single-threaded solution that handles requests sequentially is acceptable in Project 1 for full credit.  (Note, however, that later projects will involve converting your web server into a design that can handle concurrent requests, so a bit of advanced planning here may simplify work later...)

It is acceptable, in this solution, for your web server to close the connection upon completing a single request.  In fact, if you only have a single-threaded web server, this simplification is necessary. (Note, however, that the HTTP/1.1 standard allows a web browser to send multiple sequential requests in a single connection, so we will be implementing this functionality in a later project.)

Your web server should be runnable from a file called server.py.  You can import additional Python files (so that your entire project is not in a single file), but the user should not invoke these helper files directly.

Listening on port 80 requires root-level access.  Because I don't want to run your programs with root-level permissions while grading, you should instead listen on port 8080, which does not require root access.  In your web browser, access your server by specifying the port number in the URL, e.g. http://localhost:8080/file.html

You should use the argparse Python library instead of parsing the arguments yourself. Argparse will provide the --help and --version arguments for "free".

Example invocations:

$ ./server.py --help
usage: server.py [-h] [--version] [--base BASE_DIR] [--port PORT]

Web Server for COMP/ECPE 177

optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
--base BASE_DIR Base dir containing website
--port PORT Port number to listen on

$ ./server.py --base=/home/shafer/website/html --port=8080
<< Server runs and starts listening on port 8080 >> 

Restrictions

You cannot use the following Python built-in modules in this course. Zero points will be awarded for an assignment that uses: 

• http.client - FORBIDDEN!
• http.server - FORBIDDEN!
• socketserver - FORBIDDEN!
• urllib - FORBIDDEN!

In addition to the list above, you cannot use pre-written HTTP server, URL parsing, or socket management libraries obtained from other online sources.

This assignment is to completed individually. You can discuss problems and potential solutions with other students, but you cannot share completed programs or significant pieces of completed code.

See the honor code in the syllabus for specific rules on re-using code found online or in other references. (Specifically, the amount of code reused, and the policy for documenting the reuse)

Testing

The official "Project 1 Demo Website" is the five page design template (ignoring the sixth "Download Now!" link).   You can view the demo website live at http://luiszuno.com/themes/vintage/.  

This is a fake website provided as a design example.  However, for our purposes, it looks good, and provides all the types of files a real website would have, including:

• HTML files containing the page content
• Cascading style sheets (CSS) that specify format for the page content
• JavaScript programs that enable animation, drop-down menus, and other glitz
• Images in a variety of formats

Browse the real website and remember how it looks.  When you access the same files, using the same web browser, but delivered through *your* web server, you want the result to look the same!

To test, follow this process:

1. Downloadable the Demo Website from the Canvas Files section.  It should be called website.zip. 
2. Unzip the website archive. Remember where you put it.  (You can browse it to get a sense of the files and directory structure.  The "html" subdirectory has the important files.)
3. Run your web server.  Specify on the command line the root directory of your website in this demo site. For example: 
   ./server.py --base=/home/myusername/website/html --port=8080
4. Run your web browser and access the following URLs for testing.   Be sure to browse the site after it appears, to ensure the links work!
   http://127.0.0.1:8080/index.html
   http://localhost:8080/index.html 
   http://localhost:8080/   (No file name is specified here.  What should your server do?)

Tip:  It is *unlikely* that your web server will work perfectly on the first attempt.  I suggest identifying and accessing individual CSS files, JavaScript files, images, and HTML files.   Load one at a time.  Once you are convinced that each type of resource is being served correctly, then go and test the entire site.

Resources

See the main resource page for links that helped me when developing my solution.

See also: example Python client and example Python server

Checkpoints

This project has two weekly checkpoints due, in addition to the final project deadline. Checkpoints give the instructor an opportunity to review your in-progress work, and, if problems are found, provide helpful feedback in advance of the project deadline. Checkpoints are graded as full credit, half credit, or no credit.

• Checkpoint 1: 
• Server accepts request from client and displays text request to screen (or, if you are making good progress, your server can decode the HTTP request)
• Server uses argparse for command-line arguments, including port number.
• Checkpoint 2: 
• Server can successfully decode HTTP request from client and send a single reply file given a controlled test environment.  (Tip: If your server is "fragile", provide some information on how it should be successfully tested)

Submission

There are slight differences between Python 3.x versions (3.2, 3.3, and 3.4) To ensure I use the same version of Python while grading that you did during development, include the following version-checking code during your program's initialization.
Note: Replace "3,4" with the version number of Python that you used.

import sys
if not sys.version_info[:2] == (3,4):
 print("Error: need Python 3.4 to run program")
 sys.exit(1)
else:
 print("Using Python 3.4 to run program")

If your Python program is just a single .py file, simply upload it to Canvas directly.

Otherwise, if your program contains multiple source code files, create a .tar.gz compressed archive and upload that.  To create the archive, assuming your files are in the folder "project1", run:

$ tar -cvzf project1.tar.gz project1

Once created, upload this archive file to the corresponding Canvas assignment and submit.  To extract your archive, I will run:

$ tar -xvf project1.tar.gz