Project - Integration
In this portion of the project you will be integrating the software and hardware portions of your router. Your software router will configure the hardware and send/receive packets using a simple protocol for reading and writing registers. You will be responsible for modifying your router to communicate through the hardware registers and adding the necessary user interface support for accessing registers.
An example router bitfile has been provided in /exports/provided/reference-router/reference_router.bit. Although the integration process is entirely up to your group, you might consider first integrating your software against this reference router (so that you can fix any of your software problems without worrying about the hardware), and then switching the bitfile to use your hardware router (so that you can fix any of your hardware problems in isolation). The reference router implements the standard Register Map specifications.
For this part of the project you will abandon the VNS framework. Your router server software will run on the machine physically connected to the NetFPGA board and communicate with the board using a Linux driver. In order to make your router run in NetFPGA mode (also referred to as "CPUMODE" in the code), you will have to change the MODE option in the Makefile from MODE = $(MODE_VNS) to MODE = $(MODE_NETFPGA) and recompile.
After changing this option, your software router will no longer connect to VNS. This means that you will need to provide the configuration information for each of your router's interfaces. On startup, your router should read a hardware information file, cpuhw, located in the current directory. An entry in this file should list the interface name, IP address, mask and MAC. The interface name must be of the format eth0-eth3. There should be one entry for each active interface. The Network Topology page specifies which network each physical interface on the router is connected to.
eth0 192.168.1.1 255.255.255.192 00:ba:d0:ca:fe:01
Although you do not have to write the code that parses the hardware information file, you do need to insure that this information is correctly saved by your router. Thus, you should modify the function sr_intf_read_from_file() in the file sr_interface.c and ensure that each interface is saved after reading it from a file. The process of saving the data should be similar or identical to what you have already implemented in sr_integ_add_interface().
In addition to specifying the interface file, you will also want to supply a static routing table file. Refer to the Network Topology page in order to properly construct your routing table. Remember, the format for the rtable file is <subnet><next-hop IP><subnet netmask><interface>. An rtable for this assignment will look something like this:
shafer@netfpga01:~$ cat rtable 192.168.1.0 0.0.0.0 255.255.255.192 eth0 192.168.1.64 0.0.0.0 255.255.255.192 eth1 192.168.1.128 0.0.0.0 255.255.255.192 eth2 192.168.1.192 0.0.0.0 255.255.255.192 eth3 0.0.0.0 192.168.1.194 0.0.0.0 eth3
The first four entries tell the hardware to forward packets for the directly attached subnets to the appropriate Ethernet port. The final entry is a default route which tells your router where to forward any packets not matching any other rule. (This default route is optional and not very useful for this particular network topology, but if we connected the NetFPGA board to the campus network, we could name a campus router here.)
Note that your hardware should not make any assumptions about what will be in the routing table, which should start out empty when the FPGA is first started. The software will have to write the interface information and routing entries to the hardware before it can start processing packets correctly.
The command to run your software router and connect to the board is shown below. Note that since we are no longer using VNS, we no longer have to specify a VNS server, topology, username, or authentication key. Further, note that root access is required to use raw sockets in order to connect with the NetFPGA card. We have extended the op utility to provide this access to you. For this command to work, you must be inside your router directory and call the binary of your software router "sr".
shafer@netfpga01:~$ op sr -r <routing table file> -i <interface file>
Tip: Don't run this command in the background! (i.e. don't use the & symbol). op sr runs as root, and if you let this program run in the background, you will have no way to kill it later, short of rebooting the system.
Once connected, you should be able to access the hardware registers and if implemented correctly, send and receive packets as before. In a nutshell, the hardware supports eight packet channels. Four channels for the Ethernet ports (e-ports) and four matching CPU channels (c-ports). An incoming packet on an e-port will be routed to any one of the eight channels, according to the entries in the routing table and the destination IP address table (listing the IP addresses that need to be forwarded to the software). If it is routed to a c-port, the software will be able to read it in. To send a packet out from software, write it to the appropriate c-port and it will automatically be forwarded to the matching e-port and sent out.
Tip: Because you have been using your dedicated NetFPGA machine the entire semester, it is likely that packets have accidentally been sent to the CPU queues by mistake. You may wish to reboot your system and clear the PCI controller before beginning integration, to remove any stale packets or old (invalid) control data.
op nf_download </path/to/your/router/bitfile.bit>
Tip: Your software router must be run on the netfpga0x system for your group number. This is a 32-bit machine, not a 64-bit machine like the ecs-network server. Thus, you must compile your software router on a 32-bit machine. If you accidentally compile it on the wrong system, simply do a make clean and recompile.
Instead of sending and receiving packets through VNS, your router will send and receive them over raw packet sockets connected to the CPU queues on NetFPGA. To save time, this has already been provided by the functions sr_cpu_input and sr_cpu_output (found in sr_cpu_extension_nf2.c). The Raw Packet Sockets page gives a tutorial what is happening behind-the-scenes to use raw packet sockets with NetFPGA. Note that root access is required to use raw sockets. We have extended the op utility to provide this access to you.
You need to use the readReg() and writeReg() functions to access the hardware in order to initialize/modify/read the routing table and other hardware router structures. These functions can be found at /lib/C/common/nf2util.h (and nf2util.c). This is the same code that is used to produce the regread and regwrite utilities, but it will be more elegant if you integrate the functionality directly into your software router. The register specification is provided in the Register Map.
You should extend your user interface to be able to read and write the hardware registers. This will be invaluable for debugging, as it enables you to determine the status of all registers at run-time. You should extend your functions for displaying internal software router state (routing table, ARP table, interfaces) to also display internal hardware router state, so that you can determine if they are consistent.
You have been provided with a hardware bitfile that correctly implements the hardware router specification. (Look in /exports/provided/reference-router/reference_router.bit). Your software must operate correctly with the reference router to ensure conformance to the specifications and interoperability with our (and other groups') hardware routers.
Once your integrated router is functional, we encourage you to evaluate its performance. A small part of the project grade will be related to the relative performance of your router in several scenarios.
However, you should NOT worry about performance unless/until you have completed everything else related to this project!
Your software must be able to operate with this reference hardware (for interoperability) and with your group's hardware.
You hardware should perform all of the functions that were previously required of it. All other functions should be performed in software. Your integrated router must continue to support all of the functions that were previously required of the software (which may require some changes to the hardware). Note that the software must now correctly control the hardware, so you will need to extend your software implementation to be able to configure the hardware tables to mirror the software data structures which can be considered the "master copies". Note that the size of the hardware and software tables will be different, so you will need to make some decisions about what will be "cached" in the hardware. Your integrated router should allow for larger routing tables and ARP tables in software compared to in hardware. If the number of table entries is less than the hardware capacity, then all entries should be in hardware. (i.e. you should not rely on the software to perform all routing)
You will turn in the project report, software (C, ...) and, if applicable for your group, the hardware (Verilog) for your integrated router implementation using the normal Project Submission process.
Your integration project report should answer the following questions:
- How are the router tasks divided between hardware and software?
- What is the design of the interface(s) between hardware and software?
- What are the issues involved in managing the hardware?
- How do you update the routing table/ARP table/etc?
- How do you deal with different sized and formatted structures in hardware and software?
- (If applicable for your group) Did you need to modify your hardware and/or software for the integration? How?
- Did you need to make changes to your user interface to access the hardware?
- What issues did you not anticipate in integrating the hardware and software?
- How did you test software, hardware (if applicable), and your integrated router?
This project will be graded out of 100 points.
- Correctness with your hardware: 80 points (if your group implemented a hardware router)
- Correctness with reference hardware: 80 points (if your group did not implement a hardware router)
- Performance: 10 points
- Project Report: 10 points
This guideline is not binding; we reserve the right to assign whatever grade we believe is fair.